Getting a live view of environment variables (Gradle and JDK 9)

David Holmes david.holmes at oracle.com
Thu May 18 09:30:27 UTC 2017


On 18/05/2017 6:19 PM, Cédric Champeau wrote:
>>
>>
>> Can you elaborate as to why specifying the "big kill switch"
>> --permit-illegal-access is not viable? Specifically if you use:
>>
>> -XX:+IgnoreUnrecognizedVMOptions --permit-illegal-access
>>
>> it should work on 9 and be ignored on earlier releases.
>>
>>
> mmm that's interesting, I actually forgot this flag exists. However, it's
> an OpenJDK/Oracle JDK specific flag, right? (Thinking of users running IBM
> JDK typically).

It's an OpenJDK flag, yes. But I would have expected there were already 
differences in how you launch an IBM JVM versus OpenJDK or Oracle JVM - no?

David

>
>> Thanks,
>> David
>>
>> On 11/05/2017 7:37 AM, Cédric Champeau wrote:
>>
>>> Hi all,
>>>
>>> I'm writing this on behalf of the Gradle team. This email is closely
>>> related to the other thread just posted today, but just a timeline
>>> coincidence (just like the email exchange I had today about this with Alan
>>> Bateman ;)) and not exactly the same issue.
>>>
>>> We are in the process of making sure Gradle runs properly on JDK 9, but
>>> there's an issue which is unresolved so far, and probably requires a new
>>> API. It's described at [1], and I have discussed this at Devoxx France
>>> with
>>> Rémi Forax who suggested to post something here.
>>>
>>> In short, Gradle is a build tool which supports building a variety of
>>> different things, from Java to C++. The JVM happens to be its runtime
>>> environment, and Gradle has what we call the Gradle Daemon [2] which is a
>>> long running process, benefiting from the JIT, aimed at effectively
>>> running
>>> builds. When the build starts, a client connects to the daemon and sends a
>>> "build request". A daemon will run a single build at a time, and there are
>>> several cases which would trigger a new daemon to spawn (the daemon JVM
>>> arguments are one) but the environment variables are *not*. Since the
>>> daemon is a long running process, it is possible that the environment
>>> variables are mutated between the moment the daemon was spawned (in a
>>> previous build) and the moment the build is executed.
>>>
>>> What we do, now, is to send the environment variables of the client to the
>>> daemon, which *mutates* the existing environment variables map provided by
>>> System.getenv. This is exactly what is described in [1] as being sneaky
>>> (it
>>> is) and broken in JDK 9 (since the underlying map doesn't exist anymore).
>>> However, there are valid use cases for this:
>>>
>>>    - in practice, environment variables are not immutable. It is
>>> especially
>>>    true for long running process.
>>>    - native programs can mutate the environment variables. Even if it's
>>> not
>>>    recommended, it is possible and legal.
>>>    - Gradle runs in a "blackbox": we don't know what plugins are doing.
>>>    Even if we provide an API which gives access to "environment
>>> variables",
>>>    and that those environment variables are not the ones returned by
>>>    System.getenv, plugin authors would have to use this new API to get
>>>    correct information. However, they may use libraries which access
>>>    System.getenv directly, or use native APIs which would get out-of-sync
>>>    information.
>>>    - we need to propagate the environment to forked process (typically,
>>>    forked compilers and worker daemons)
>>>
>>> This means that today, we use JNI to effectively mutate the environment
>>> variables of running process (that’s one of the purposes of the
>>> native-platform project). Then, we mutate the backing map of the JDK to
>>> reflect those changes, otherwise the mutation is not visible from Java
>>> code.
>>>
>>> What can we do now?
>>>
>>>    - Have the JDK honor the fact that environment variables *can* be
>>>    mutated, because it just happens. In short, don't create an immutable
>>> copy
>>>    of environment variables at startup, but provide a live view of the
>>>    environment variables (using the existing APIs, System.getenv, would
>>> be the
>>>    best thing because it would be immediately visible to all consumers,
>>>    including 3rd party code run in plugins). In addition (but not
>>> mandatory),
>>>    you could provide us with an API to set environment variables directly
>>> from
>>>    Java. This would avoid JNI calls to do this. However, it’s not
>>> mandatory,
>>>    because the live view of environment variables would just work in this
>>> case.
>>>    - Last, but we would really, really avoid to do this, spawn a new
>>> daemon
>>>    if we detect that the environment variables have changed (diff between
>>> what
>>>    the client has and the daemon sees). The major drawback of this
>>> approach is
>>>    that it kills performance, since a new daemon would have to be
>>> spawned. And
>>>    it is likely to do so each time something (through native code, for
>>>    example), mutates environment variables. A very simple example is
>>> the PWD environment
>>>    variables on Linux which contains the working directory. Basically
>>> changing
>>>    the directory would be enough to spawn a new daemon. Another example
>>> is the
>>>    TERM_SESSION_ID one, which means that 2 different terminals would force
>>>    us to spawn 2 different Gradle daemons. We could, of course, have a
>>> list of
>>>    “blessed” environments variables that we don’t trust, but it’s very
>>> easily
>>>    broken, and no good design. That’s why, even if it’s possible, we don’t
>>>    consider this a solution.
>>>
>>> Thanks for considering our request, which is currently a blocker for us
>>> (understand, have Gradle running properly under JDK 9).
>>>
>>> [1] https://github.com/adammurdoch/native-platform/issues/16
>>> [2] https://docs.gradle.org/current/userguide/gradle_daemon.html
>>>
>>>


More information about the core-libs-dev mailing list