StackOverflowError - Java 9 Build 181

[Sean Mullan]

Cross-posting to security-dev as this is more relevant to that list and 
bcc-ing core-libs-dev.

I think this might be an issue with the JavaWebStart SecurityManager not 
being granted the proper permissions. It is possible that the deployment 
policy files are not being loaded or there is some other subtle 
bootstrapping issue. It should not result in a recursive loop of course, 
but there may be a workaround.

In the meantime, can you send me more information, preferably a test 
case and a log file with -Djava.security.debug=all enabled? (The latter 
will help analyze the recursion and see what security checks are failing 
and for which ProtectionDomains). Also, have you tested this on builds 
earlier than b181?

Thanks,
Sean

On 9/19/17 2:53 PM, Tom Hood wrote:
> I should add that we have not modified or overridden any policy files.
> Also, we are not using a custom security manager.
> 
> On Tue, Sep 19, 2017 at 11:52 AM, Tom Hood <tom.w.hood at gmail.com> wrote:
> 
>> Hi,
>>
>> I hit an infinite recursion loop probably related to PolicyFile that
>> exists in Java 9 build 181 for windows 64-bit.  It might be related to
>> JDK-8077418 <https://bugs.openjdk.java.net/browse/JDK-8077418>
>>
>> I haven't tracked down what is causing our webstart app to hit this
>> problem yet, but I thought I would let you know sooner than later.  Also,
>> it probably is not a problem for our particular application as I should be
>> able to set the security manager to null which I think/hope will bypass
>> this issue.  I will try today to reproduce it in our app so I can confirm
>> if setting security manager to null will work for us.
>>
>> The stack looks like the following: (with many repeat stacks omitted)
>>
>> Exception in thread "AWT-EventQueue-2" java.lang.StackOverflowError
>> at java.base/java.security.AccessController.doPrivileged(Native Method)
>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po
>> licyFile.java:1135)
>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po
>> licyFile.java:1082)
>> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil
>> e.java:1038)
>> at java.base/java.security.provider.ProtectionDomain.implies(Pr
>> otectionDomain.java:323)
>> at java.base/java.security.provider.ProtectionDomain.impliesWit
>> hAltFilePerm(ProtectionDomain.java:355)
>> at java.base/java.security.provider.AccessControlContext.checkP
>> ermission(AccessControlContext.java:450)
>> at java.base/java.security.provider.AccessController.checkPermi
>> ssion(AccessController.java:895)
>> at java.base/java.lang.SecurityManager.checkPermission(Security
>> Manager.java:558)
>> at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec
>> kPermission(JavaWebStartSecurity.java:237)
>> at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897)
>> at java.base/java.io.File.isDirectory(File.java:845)
>> at java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299)
>> at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb
>> ase(PolicyFile.java:1665)
>> at java.base/sun.security.provider.PolicyFile.access$700(Policy
>> File.java:263)
>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139)
>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136)
>> **** and again ****
>> at java.base/java.security.AccessController.doPrivileged(Native Method)
>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po
>> licyFile.java:1135)
>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po
>> licyFile.java:1082)
>> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil
>> e.java:1038)
>> at java.base/java.security.provider.ProtectionDomain.implies(Pr
>> otectionDomain.java:323)
>> at java.base/java.security.provider.ProtectionDomain.impliesWit
>> hAltFilePerm(ProtectionDomain.java:355)
>> at java.base/java.security.provider.AccessControlContext.checkP
>> ermission(AccessControlContext.java:450)
>> at java.base/java.security.provider.AccessController.checkPermi
>> ssion(AccessController.java:895)
>> at java.base/java.lang.SecurityManager.checkPermission(Security
>> Manager.java:558)
>> at jdk.javaws/com.sun.javaws.security.JavaWebStartSecurity.chec
>> kPermission(JavaWebStartSecurity.java:237)
>> at java.base/java.lang.SecurityManager.checkRead(SecurityManager.java:897)
>> at java.base/java.io.File.isDirectory(File.java:845)
>> at java.base/sun.net.www.ParseUtil.fileToEncodedURL(ParseUtil.java:299)
>> at java.base/sun.security.provider.PolicyFile.canonicalizeCodeb
>> ase(PolicyFile.java:1665)
>> at java.base/sun.security.provider.PolicyFile.access$700(Policy
>> File.java:263)
>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1139)
>> at java.base/sun.security.provider.PolicyFile$7.run(PolicyFile.java:1136)
>> **** above lines start the stack that repeats until overflow ****
>> at java.base/java.security.AccessController.doPrivileged(Native Method)
>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po
>> licyFile.java:1135)
>> at java.base/sun.security.provider.PolicyFile.getPermissions(Po
>> licyFile.java:1082)
>> at java.base/sun.security.provider.PolicyFile.implies(PolicyFil
>> e.java:1038)
>>
>> -- Tom
>>
>>