High memory usage / leaks was: Best mailing list for JVM embedding

[Bernd Eckenfels]

Hello,

I think in the Java 9 timeframe the Zip implementation was changed to no longer mmap the Zip index table. So this was not visible in the heap but now is. (Although it seems to be a bit slower). That might be the reason for your regression. You could use the following System property in Java 8 to get the new method there as well (in Java 9 the property and the old method is gone)

https://bugs.openjdk.java.net/browse/JDK-8175192

Gruss
Bernd
--
http://bernd.eckenfels.net

________________________________
Von: core-libs-dev <core-libs-dev-bounces at openjdk.java.net> im Auftrag von Robert Marcano <robert at marcanoonline.com>
Gesendet: Donnerstag, Januar 24, 2019 5:26 PM
An: core-libs-dev at openjdk.java.net
Betreff: Re: High memory usage / leaks was: Best mailing list for JVM embedding

On 1/23/19 8:59 AM, Sean Mullan wrote:
> On 1/22/19 8:50 PM, Bernd Eckenfels wrote:
>> I don’t think the launcher is doing this, it is the class loader,
>> that’s nothing new. You can turn on verbose security debug to see it
>> in all versions.
>
> Yes, and it only verifies the signature(s) on the JAR. It doesn't
> validate the certificate chain.
>
> --Sean

I noticed that trying to identify the higher memory usage after what
looks like a big application is loaded. I am doing memory profiling and
notice JarFile taking more memory that on Java 8. Still need to detect
the real cause for an independent test case. We probably didn't notice
this slowdown before because Oracle's JNLP implementation was slow
enough at startup.

IMHO the class library should not do doing signature checks without
certificate validation, because it doesn't give any protection if the
signature is not verified, the only thing it could do now is to detect
some random bit flips, that maybe the Zip format CRC detect before that.
With no certificate verification the signature could be replaced by
anybody with bad intentions.

Maybe adding a constructor flag to URLClassloader to pass to JarFile to
skip verification and a system property to tell the Java startup code to
skip verification of java.class.path (just for compatibility with old
code that expect it to be done). There is precedent of other runtimes
that added options to disable this, like CLR [1] (Ii not only verified
signatures, It do CRL/OSCP checks too)

Our new launcher replacing JNLP now do signature verification in native
code, at download time, and install on a system area (not user home
directory), so signature verification at application launch is a
slowdown we want to avoid, but think on another kind of users, those
deploying to OS Stores (for example Windows Store), why add the slowdown
of verification when the application is verified by the store client at
install time?, this could help these situations too.

Note: This can be avoided removing the signatures of all JARs if you
distribute to an OS store, there are a few libraries that distribute
their JARs signed (The old Java Help framework comes to my mind right now)

[1]
https://blogs.msdn.microsoft.com/shawnfa/2007/05/07/bypassing-the-authenticode-signature-check-on-startup/

>
>>
>> --
>> https://Bernd.eckenfels.net
>>
>> ________________________________
>> Von: core-libs-dev <core-libs-dev-bounces at openjdk.java.net> im Auftrag
>> von Robert Marcano <robert at marcanoonline.com>
>> Gesendet: Mittwoch, Januar 23, 2019 2:18 AM
>> An: Alan Bateman
>> Cc: OpenJDK Dev list; core-libs-dev Libs
>> Betreff: Re: High memory usage / leaks was: Best mailing list for JVM
>> embedding
>>
>> On Tue, Jan 22, 2019, 5:53 AM Alan Bateman <Alan.Bateman at oracle.com
>> wrote:
>>
>>> On 22/01/2019 4:48 am, Robert Marcano wrote:
>>>>> :
>>>>>
>>>>> So the question now is, why signed jars could affect the memory usage
>>>>> of an application (we aren't doing JAR verification on our custom
>>>>> launcher, yet), just by being on the java.class.path? IIRC the
>>>>> initial application classpath JARs were never verified previously (by
>>>>> the java launcher alone, without JNLP around).
>>>>>
>>>>> Note: Tested with JARs signed with a self signed certificate and with
>>>>> one signed with a private CA. At most, signing the JARs could slow
>>>>> down the start up if it is now expected to these being verified by
>>>>> the java launcher (is it true?) but not higher memory usage and no
>>>>> reductions after a GC cycle but constants heap size increases.
>>> Signed JARs can be expensive to verify, esp. on first usage as the
>>> verification is likely to result in early loading of a lot of security
>>> classes and infrastructure. If you can narrow down the apparently memory
>>> leak to a small test case with analysis to suggest it's a JDK bug then
>>> it would be good to get a bug submitted.
>>>
>>> -Alan
>>
>>
>> Greeting. Sure, I will work on a distributable reproduction of the
>> problem
>> today but it is new to me that the java launcher do JARs verification
>> now.
>> If it is doing it I doesn't make sense to me, because a self signed or
>> unrecognized CA doesn't trigger a validation error.
>>
>>>