stop using mmap for zip I/O

Martin Buchholz martinrb at google.com
Mon Mar 2 23:12:08 UTC 2015


Our experience at Google is also that many people run into crashes when
overwriting zip files.  Our standard answer is "don't do that, then!"
 Instead of the "evil" overwriting cp program, make a temp file, then move
it into place.  More generally, follow a policy of "immutability" for file
contents.  But that's not necessarily easy advice to follow - sometimes you
have to use Other People's Broken Software.

Yes, we could prevent the crashes by using only ordinary file I/O, but
you'd still run into input zip file corruption, so the effect would end up
being similar (but more recoverable - a "friendlier" crash).

A zip file is a kind of filesystem, and recovery from zip file corruption
is similar to recovery from any other kind of filesystem corruption
(especially when your app is partially located on the filesystem in
question).

But turning the SEGV/SIGBUS into a ZipException would definitely be
progress.  It would be nice if hotspot's existing signal handler could
cooperate somehow with zip_util.c to make that happen, but I don't know how
to do that, especially not without increasing coupling between libraries
and VM.

On Fri, Feb 27, 2015 at 6:17 AM, Alan Bateman <Alan.Bateman at oracle.com>
wrote:

> On 27/02/2015 11:30, Andrew Haley wrote:
>
>> On 02/27/2015 09:51 AM, Seán Coffey wrote:
>>
>>> So we catch the signal, right?  Maybe there's something I'm missing...
>>>>
>>> We don't catch/detect such SEGV's. Would it make sense, is it possible ?
>>>
>> Sure.  There's some fiddly low-level code involved but it's perfectly
>> possible, at least on GNU/Linux.  Maybe it's not worth the effort,
>> though.
>>
>>  The closest is MappedByteBuffer and there is code in hotspot to handle
> the SIGBUS and recover (recover in the form of throw an Error). It should
> be possible to do the same for zip but I assume would require this code to
> go through the VM. Much easier if we just disable mapping the central
> directory. At one point then Sherman and I talked about only mapping the
> directory of rt.jar and other JAR files in the JDK, don't map for zip/JAR
> files elsewhere. This gets easier now because we don't have JAR files in
> the JDK image, hence we just need to flip the default on the property that
> enables/disable this. Better again is Sherman's patch to move from native
> completely, we've had good experience along these lines already with the
> NIO zip provider.
>
> -Alan
>



More information about the core-libs-dev mailing list