Gradle and changes to application class loader in Java 9.

[Peter Levart]

On 04/23/2015 06:18 AM, Luke Daley wrote:
> Hi all,
>
> I’ve been asked (by Alan Bateman) to move the discussion that started @ https://discuss.gradle.org/t/classcastexception-from-org-gradle-process-internal-child-bootstrapsecuritymanager/2443 and then moved to email due to technical reasons to this list.
>
> In summary, the way that Gradle mitigates the command line length limit when forking build processes breaks with 9. Mandy Chung’s original post at the link above describes why this breaks.
>
> The suggested alternative to the current “trick” of injecting into the application class loader via the security manager is to use a jar with a classpath attribute, thereby reducing the command line length. Gradle has been down this route before, but it broke some tools. One that we have a record of is GWT-Test (https://issues.gradle.org/browse/GRADLE-2223). It will potentially break anything that inspects `java.class.path` or that does the equivalent by `getURLs()` on the application class loader.
>
> We don’t really know what the practical impact of this change would be today, but aren’t willing to risk the breaking change to our existing users on < 9. Therefore, in lieu of options file support in 9, our intention is to use the classpath manifest jar approach only on 9. If our users are using any tools that rely on `java.class.path` they may be blocked from moving to 9.
>
> We are intending to make this change “soon”.
>
> —
>
> Join us for Gradle Summit 2015, June 11 - 12 in Santa Clara, CA: http://www.gradlesummit.com

Hi,

I think it would be nice to be able to spawn a Java subprocess and set 
it's system properties to huge strings without creating temporary files 
to communicate that information.

From: 
https://msdn.microsoft.com/en-us/library/windows/desktop/ms682653%28v=vs.85%29.aspx 


"The maximum size of a user-defined environment variable is 32,767 
characters. There is no technical limitation on the size of the 
environment block. However, there are practical limits depending on the 
mechanism used to access the block. For example, a batch file cannot set 
a variable that is longer than the maximum command line length. Windows 
Server 2003 and Windows XP:  The maximum size of the environment block 
for the process is 32,767 characters. Starting with Windows Vista and 
Windows Server 2008, there is no technical limitation on the size of the 
environment block."

I believe UNIXes have much larger limits for environment variables. So 
why not use environment variables to overcome command-line limits?

In case of gradle, I assume it is spawning subprocesses via 
java.lang.Process API right? Therefore there is no CMD.EXE limitation 
there. But 32k per environment variable might still be to small to stuff 
the entire class path into a single variable. So what about a scheme 
where "-Dsystem.property=value" syntax was enhanced to support 
"expansion" from environment variables. For example:

-Djava.class.path=/some/dir/some.jar:${env:CP1}:${env:CP2}:${env:CP3}

with environment variables defined as:

CP1=/some/dir1/some1.jar
CP2=/some/dir2/some2.jar
CP3=/some/dir3/some3.jar

...the above JVM argument would set java.class.path to:

/some/dir/some.jar:/some/dir1/some1.jar:/some/dir2/some2.jar:/some/dir3/some3.jar

Environment variables in JVM are admittedly lazily initialized when the 
program needs them (System.getenv()) and JVM boot-up sequence does not, 
currently. But in this scheme they would only be initialized during 
boot-up if any "-Dprop=value" contained the value with expansion syntax, 
so normally they would not be. There is some danger that some 
application already uses the ${env:XXX} syntax for it's own purposes, 
but for JDK9 I think, this could be tolerated.

The patch to introduce this is quite simple:

http://cr.openjdk.java.net/~plevart/jdk9-dev/SystemProperties.EnvExpansion/webrev.01/


When there's no expansion syntax used in system properties, the boot-up 
sequence amounts to loading just the following additional class:

+ [Loaded java.util.Hashtable$KeySet from java/util/Hashtable$KeySet.class]

When there is expansion syntax used in some system property, it 
additionally loads just the following:

+ [Loaded java.lang.ProcessEnvironment$ExternalData from 
java/lang/ProcessEnvironment$ExternalData.class]
+ [Loaded java.lang.ProcessEnvironment from 
java/lang/ProcessEnvironment.class]
+ [Loaded java.lang.ProcessEnvironment$StringEnvironment from 
java/lang/ProcessEnvironment$StringEnvironment.class]
+ [Loaded java.lang.ProcessEnvironment$Value from 
java/lang/ProcessEnvironment$Value.class]
+ [Loaded java.lang.ProcessEnvironment$Variable from 
java/lang/ProcessEnvironment$Variable.class]
+ [Loaded java.util.Collections$UnmodifiableMap from 
java/util/Collections$UnmodifiableMap.class]


What do you say?

Regards, Peter