MethodHandle initialization process - problem with JVM TI early VM start event

[David Holmes]

Hi Alan,

On 4/01/2017 6:45 PM, Alan Bateman wrote:
> On 04/01/2017 03:11, David Holmes wrote:
>
>> We have encountered a crash in a JVM TI test, starting with b148, that
>> is caused by the attempted use of MethodRefs and invoke_dynamic within
>> class loading that occurs from the "early VM start" JVM TI event
>> callback - which happens before the VM and core classes are fully
>> initialized. The class being loaded/initialized is in java.base and so
>> this is allowed. The class, java.net.Authenticator, was changed in
>> b148 to have static initialization involving method refs and hence the
>> problem was introduced.
>>
>> I've included a stack dump below.
>>
>> It is far from clear to me where responsibility for this lies, or even
>> how to narrow that down. Is it in the code of MethodHandleNatives? Or
>> is it in the VM linking/resolution code?
>>
>> Thoughts appreciated. :)
> Ugh, this looks like the callback for the VMStart event is triggering
> the execution of arbitrary code via its initializer. When the
> can_generate_early_vmstart capability is enabled then the VMStart event
> is posted before the system classes are initialized (before
> Threads::initialize_java_lang_classes is called). If the callbacks uses
> JNI FindClass to load classes in java.base (it's restricted to java.base
> in this phase) then it will trigger the execution of random bytecode
> with all sort of consequences. Even if the system classes are
> initialized then it can still cause all sorts of issues - e.g.
> triggering code in classes such as ResourceBundle and elsewhere that
> depend on the system class loader to be initialized. For the specific
> crash then it looks like MethodHandleNatives has asserts and so
> Class::desiredAssertionStatus will be invoked to test if assertions are
> enabled but, as I say, there are lots of other reasons that would cause
> random classes in java.base to fail when invoked before the system
> classes are initialized.

Right.

> The reason this event is called so early is to allow native (not java)
> agents get other events during startup. Also it's not new that this
> event is posted before the system classes are initialized, you'll see
> the same with JDK 8 and older because VMStart was always posted very
> early. In JDK 9 then the VMStart is only posted early when the
> can_generate_early_vmstart capability is enabled. This chage.
>
> As regards fixing this then I don't think it would be unreasonable to
> put some restrictions on what can be done in the VMStart callback when
> can_generate_early_vmstart capability is enabled. Maybe it could be just
> further re-wording of the paragraph on what can be done in this phase
> rather than trying to enforce it.

That is my thought too, that the spec needs to give less of the 
impression that it's okay to access java.base classes at this early VM 
start event, and basically say that any form of class-loading is not 
guaranteed to succeed and will quite likely crash the JVM.

The test can then be modified accordingly.

Thanks,
David

> -Alan