RFR(S) 8230956: Should disable Escape Analysis when JVMTI capability can_tag_objects is taken

[Vladimir Kozlov]

It is also not clear to me that it is bug. Based on all description this functionality is used to 
catch leaks in Java heap. But scalar replaced objects do not exists. JVMTI should not even see them.

Thanks,
Vladimir

On 9/24/19 3:37 PM, Vladimir Kozlov wrote:
> can_tag_objects is "always" capability.
> 
> If it is true then EA will be disabled in all cases when JVMTI agent is used. It is too broad.
> 
> Am I missing something?
> 
> Thanks,
> Vladimir
> 
> On 9/13/19 7:12 AM, Reingruber, Richard wrote:
>> Hi,
>>
>> could I please get reviews for
>>
>> Webrev: http://cr.openjdk.java.net/~rrich/webrevs/2019/8230956/webrev.0/
>> Bug:    https://bugs.openjdk.java.net/browse/JDK-8230956
>>
>> JVMTI provides functions to follow references beginning at the roots of the object graph and it
>> provides functions to iterate all objects on the heap[1][2]. These functions are means to access
>> objects which are otherwise local to a Java thread. In terms of escape analysis these local objects
>> escape through these JVMTI functions invalidating optimizations based on escape analysis.
>>
>> Example:
>>
>> - Let J be a JavaThread that calls a compiled method M with a NoEscape instance I of class C that is
>>    scalar replaced.
>>
>> - JVMTI agent A uses JVMTI FollowReferences() to iterate the objects in the object graph tagging all
>>    instances of C.
>>
>> - A uses GetObjectsWithTags() to retrieve the tagged instances of C.
>>
>> - Error: I is missing because its allocation was eliminated / scalar replaced.
>>
>> Agents are required to possess the capability can_tag_objects in order to call the JVMTI heap
>> functions that let objects escape.  Currently it is not possible to revert EA based optimizations
>> just before objects escape through JVMTI therefore escape analysis should be disabled as soon as the
>> JVMTI capability can_tag_objects is taken.
>>
>> But this is not sufficient, because there may be compiled frames on stack with EA based
>> optimizations when a JVMTI agent takes can_tag_objects (see included exclusive test cases), and then
>> it does not help to disable escape analysis or invalidate compiled methods with ea based
>> optimizations. In general it is still an improvement to do so. JDK-8227745 would be a complete
>> solution to the issue.
>>
>> An further improvement could be to invalidate methods compiled by c2 when can_tag_objects gets
>> added, but I'd rather suggest to integrated the implementation for JDK-8227745. Note also that after
>> calling JVMTI AddCapabilities(), even with an empty set of capabilities,
>> JvmtiExport::can_walk_any_space() will return true.
>>
>> I've run tier1 tests.
>>
>> Thanks, Richard.
>>
>> [1] https://docs.oracle.com/en/java/javase/11/docs/specs/jvmti.html#Heap
>> [2] https://docs.oracle.com/en/java/javase/11/docs/specs/jvmti.html#Heap_1_0
>>