RFR [7] 8133206: "java.lang.OutOfMemoryError: unable to create new native thread" caused by upgrade to zlib 1.2.8

[Nikolay Gorshkov]

Please, review the following fix in core-libs area for the bug
https://bugs.openjdk.java.net/browse/JDK-8133206 :

http://cr.openjdk.java.net/~nikgor/8133206/jdk7u-dev/webrev.00/

The bug is a regression of the recent zlib 1.2.3 => 1.2.8 library
update. One of our customers started to see significant increase
of native memory usage in production, which actually led to startup
failures of one of their service in 32 bit mode.

The root cause is that inflate() method in zlib allocates a 32 KB
window much more often than before (please, see the bug record for
full details), and in some use cases like class loading these windows
are then never freed.

The fix (its idea was proposed by Mark Adler) is to change the way
we call inflate() method of zlib library and process its return result.
I fixed 4 points in jdk code. There are several more points in libpng,
which is also a part of jdk code, but I didn't touch them - not sure
what is our policy about changing the code of 3rd party libraries.

The fix was verified:
- in the customer's test environment (the problem's gone)
- by running JCK-7
- by running regression tests for java/util/jar, java/util/zip,
   tools/jar, tools/pack200

The changeset is prepared for integration into jdk7u-dev. The only reason
for this is that the only reproducer we have is a complicated environment
that accepts JDK 7 only. I would say that with 90% probability the same
issue exists in both JDK 8 and 9, and the same (or very similar) fix is
applicable to them, but unfortunately I can't prove it - we don't have
a reproducer for JDK 8 and 9.

Please, reply to me directly since I'm not subscribed to these aliases.

Thanks,
Nikolay