RFR [7] 8133206: "java.lang.OutOfMemoryError: unable to create new native thread" caused by upgrade to zlib 1.2.8

[Xueming Shen]

On 10/14/2015 09:02 AM, Nikolay Gorshkov wrote:
> Please, review the following fix in core-libs area for the bug
> https://bugs.openjdk.java.net/browse/JDK-8133206 :
>
> http://cr.openjdk.java.net/~nikgor/8133206/jdk7u-dev/webrev.00/
>
> The bug is a regression of the recent zlib 1.2.3 => 1.2.8 library
> update. One of our customers started to see significant increase
> of native memory usage in production, which actually led to startup
> failures of one of their service in 32 bit mode.
>
> The root cause is that inflate() method in zlib allocates a 32 KB
> window much more often than before (please, see the bug record for
> full details), and in some use cases like class loading these windows
> are then never freed.

Hi Nikolay,

Can you be more specific about the "class loading cases" above? Sounds
more like we have a memory leaking here (the real root cause)? for example
the inflateEnd() never gets called?

 From the doc/impl in inflate() it appears the proposed change should be
fine, though it's a little hacky, as you never know if it starts to return
Z_OK from some future release(s). Since the "current" implementation
never returns Z_OK, it might be worth considering to keep the Z_OK logic
asis in Inflater.c, together with the Z_BUF_ERROR, just in case?

I would be desired to add some words in Inflater.c to remind the
future maintainer why we switched from partial to finish and why to
check z_buf_error.

-------------------------------------------------------------------------------------------
inflate.c

    In this implementation, the flush parameter of inflate() only affects the
    return code (per zlib.h).  inflate() always writes as much as possible to
    strm->next_out, given the space available and the provided input--the effect
    documented in zlib.h of Z_SYNC_FLUSH.  Furthermore, inflate() always defers
    the allocation of and copying into a sliding window until necessary, which
    provides the effect documented in zlib.h for Z_FINISH when the entire input
    stream available.  So the only thing the flush parameter actually does is:
    when flush is set to Z_FINISH, inflate() cannot return Z_OK.  Instead it
    will return Z_BUF_ERROR if it has not reached the end of the stream.
...

     if (((in == 0 && out == 0) || flush == Z_FINISH) && ret == Z_OK)
         ret = Z_BUF_ERROR;
     return ret;
-------------------------------------------------------------------------------------------

-Sherman