RFR: 8038468: java/lang/instrument/ParallelTransformerLoader.sh fails with ClassCircularityError

[serguei.spitsyn at oracle.com]

Yumin,

On 10/24/14 4:08 PM, Yumin Qi wrote:
> Serguei,
>
>   Thanks for your comments.
>   This test happens intermittently, but now it can repeat with 8/9.
>   Loading TestClass1 in main thread while loading TestClass2 in 
> TestThread in parallel. They both will call transform since 
> TestClass[1-3] are loaded via agent. When loading TestClass2, it will 
> call loading TestClass3 in TestThread.
>   Note in the main thread, for loop:
>
>                   for (int i = 0; i < kNumIterations; i++)
>                 {
>                         // load some classes from multiple threads 
> (this thread and one other)
>                         Thread testThread = new TestThread(2);
>                         testThread.start();
>                         loadClasses(1);
>
>                         // log that it completed and reset for the 
> next iteration
>                         testThread.join();
>                         System.out.print(".");
> ParallelTransformerLoaderAgent.generateNewClassLoader();
>                 }
>
> The loader got renewed after testThread.join(). So both threads are 
> using the exact same class loader.

You are right, thanks.
It means that all three classes (TesClass1, TestClass2 and TestClass3)
are loaded by the same class loader in each iteration.

However, I see more cases when the TestClass3 gets loaded.
It happens in a CFLH event when any other class (not TestClass*) in the 
system is loaded.
The class loading thread can be any, not only "main" or "TestClass" thread.
I suspect this test case mostly targets class loading that happens on 
other threads.
It is because of the lines:
                         // In 160_03 and older, transform() is called
                         // with the "system_loader_lock" held and that
                         // prevents the bootstrap class loaded from
                         // running in parallel. If we add a slight sleep
                         // delay here when the transform() call is not
                         // main or TestThread, then the deadlock in
                         // 160_03 and older is much more reproducible.
                         if (!tName.equals("main") && 
!tName.equals("TestThread")) {
                             System.out.println("Thread '" + tName +
                                 "' has called transform()");
                             try {
                                 Thread.sleep(500);
                             } catch (InterruptedException ie) {
                             }
                         }

What about the following?

In the ParallelTransformerLoaderAgent.java  make this change:
               if (!tName.equals("main"))
                   => if (tName.equals("TestThread"))

Does such updated test still failing?

> After create a new class loader, next loop will use the loader. This 
> is why quite often on the stack trace we can see it resolves JarLoader$2.
>
> I am not quite understand the test case either. Loading TestClass3 
> inside transform using the same classloader will cause  call to 
> transform again and form a circle. Nonetheless, if we see TestClass2 
> already loaded, the loop will end but that still is a risk.

In fact, I don't like that the test loads the class TestClass3 at the 
TestClass3 CFLH event.
However, it is interesting to know why we did not see (is it the case?) 
this issue before.
Also, it is interesting why the test stops failing with you fix 
(replacing loader with SystemClassLoader).

The test case was added by Dan.
We may want to ask him to clarify the test case purpose.
(added Dan to the to-list)

Thanks,
Serguei

>
> Thanks
> Yumin
>
> On 10/24/2014 1:20 PM, serguei.spitsyn at oracle.com wrote:
>> Hi Yumin,
>>
>> Below is some analysis to make sure I understand the test scenario 
>> correctly.
>>
>> The ParallelTransformerLoaderApp.main() executes a 1000 iteration loop.
>> At each iteration it does:
>>   - creates and starts a new TestThread
>>   - loads TestClass1 with the current class loader: 
>> ParallelTransformerLoaderAgent.getClassLoader()
>>   - changes the current class loader with new one: 
>> ParallelTransformerLoaderAgent.generateNewClassLoader()
>>
>> The TestThread loads the TestClass2 concurrently with the main thread.
>>
>> At the CFLH events, the ParallelTransformerLoaderAgent does the class 
>> retransformation.
>> If the thread loading the class is not "main", it loads the class 
>> TestClass3
>> with the current class loader 
>> ParallelTransformerLoaderAgent.getClassLoader().
>>
>> Sometimes, the TestClass2 and TestClass3 are loaded by the same class 
>> loader recursively.
>> It happens if the class loader has not been changed between loading 
>> TestClass2 and TestClass3 classes.
>>
>> I'm not convinced yet the test is incorrect.
>> And it is not clear why do we get a ClassCircularityError.
>>
>> Please, let me know if the above understanding is wrong.
>> I also see the reply from David and share his concerns.
>>
>> It is not clear if this failure is a regression.
>> Did we observe this issue before?
>> If - NOT then when and why had this failure started to appear?
>>
>> Unfortunately, it is impossible to look at the test run history at 
>> the moment.
>> The Aurora is at a maintenance.
>>
>> Thanks,
>> Serguei
>>
>> On 10/13/14 3:58 PM, Yumin Qi wrote:
>>> bug: https://bugs.openjdk.java.net/browse/JDK-8038468
>>> webrev:*http://cr.openjdk.java.net/~minqi/8038468/webrev00/
>>>
>>> the bug marked as confidential so post the webrev internally.
>>>
>>> Problem: The test case tries to load a class from the same jar via 
>>> agent in the middle of loading another class from the jar via same 
>>> class loader in same thread. The call happens in transform which is 
>>> a rare case --- in middle of loading class, loading another class. 
>>> The result is a CircularityError. When first class is in loading, in 
>>> vm we put JarLoader$2 on place holder table, then we start the 
>>> defineClass, which calls transform, begins loading the second class 
>>> so go along the same routine for loading JarLoader$2 first, found it 
>>> already in placeholder table. A CircularityError is thrown.
>>> Fix: The test case should not call loading class with same class 
>>> loader in same thread from same jar in 'transform' method. I modify 
>>> it loading with system class loader and we expect see 
>>> ClassNotFoundException. Detail see bug comments.
>>>
>>> Thanks
>>> Yumin *
>>
>