RFR: 8038468: java/lang/instrument/ParallelTransformerLoader.sh fails with ClassCircularityError

[serguei.spitsyn at oracle.com]

Yumin,

Thank you for answering my questions.

Thanks,
Serguei

On 10/24/14 5:23 PM, Yumin Qi wrote:
>
> On 10/24/2014 4:57 PM, serguei.spitsyn at oracle.com wrote:
>> Yumin,
>>
>> On 10/24/14 4:08 PM, Yumin Qi wrote:
>>> Serguei,
>>>
>>>   Thanks for your comments.
>>>   This test happens intermittently, but now it can repeat with 8/9.
>>>   Loading TestClass1 in main thread while loading TestClass2 in 
>>> TestThread in parallel. They both will call transform since 
>>> TestClass[1-3] are loaded via agent. When loading TestClass2, it 
>>> will call loading TestClass3 in TestThread.
>>>   Note in the main thread, for loop:
>>>
>>>                   for (int i = 0; i < kNumIterations; i++)
>>>                 {
>>>                         // load some classes from multiple threads 
>>> (this thread and one other)
>>>                         Thread testThread = new TestThread(2);
>>>                         testThread.start();
>>>                         loadClasses(1);
>>>
>>>                         // log that it completed and reset for the 
>>> next iteration
>>>                         testThread.join();
>>>                         System.out.print(".");
>>> ParallelTransformerLoaderAgent.generateNewClassLoader();
>>>                 }
>>>
>>> The loader got renewed after testThread.join(). So both threads are 
>>> using the exact same class loader.
>>
>> You are right, thanks.
>> It means that all three classes (TesClass1, TestClass2 and TestClass3)
>> are loaded by the same class loader in each iteration.
>>
>> However, I see more cases when the TestClass3 gets loaded.
>> It happens in a CFLH event when any other class (not TestClass*) in 
>> the system is loaded.
>> The class loading thread can be any, not only "main" or "TestClass" 
>> thread.
>> I suspect this test case mostly targets class loading that happens on 
>> other threads.
>> It is because of the lines:
>>                         // In 160_03 and older, transform() is called
>>                         // with the "system_loader_lock" held and that
>>                         // prevents the bootstrap class loaded from
>>                         // running in parallel. If we add a slight sleep
>>                         // delay here when the transform() call is not
>>                         // main or TestThread, then the deadlock in
>>                         // 160_03 and older is much more reproducible.
>>                         if (!tName.equals("main") && 
>> !tName.equals("TestThread")) {
>>                             System.out.println("Thread '" + tName +
>>                                 "' has called transform()");
>>                             try {
>>                                 Thread.sleep(500);
>>                             } catch (InterruptedException ie) {
>>                             }
>>                         }
>>
>> What about the following?
>>
>> In the ParallelTransformerLoaderAgent.java  make this change:
>>               if (!tName.equals("main"))
>>                   => if (tName.equals("TestThread"))
>>
>> Does such updated test still failing?
>>
> It still fails though it takes longer time to fail. The other 
> question, from D. H. is that the exception stack trace is from 'main' 
> thread. That still not known how that happened.
>
>>> After create a new class loader, next loop will use the loader. This 
>>> is why quite often on the stack trace we can see it resolves 
>>> JarLoader$2.
>>>
>>> I am not quite understand the test case either. Loading TestClass3 
>>> inside transform using the same classloader will cause  call to 
>>> transform again and form a circle. Nonetheless, if we see TestClass2 
>>> already loaded, the loop will end but that still is a risk.
>>
>> In fact, I don't like that the test loads the class TestClass3 at the 
>> TestClass3 CFLH event.
>> However, it is interesting to know why we did not see (is it the 
>> case?) this issue before.
>> Also, it is interesting why the test stops failing with you fix 
>> (replacing loader with SystemClassLoader).
>>
> Since TestClass[1-3] are designed loaded by custom class loader, the 
> system class loader will not find it --- ClassNotFoundException thrown.
>
>> The test case was added by Dan.
>> We may want to ask him to clarify the test case purpose.
>> (added Dan to the to-list)
>>
> Looking forward to getting Dan's comments.
>
>> Thanks,
>> Serguei
>>
>>>
>>> Thanks
>>> Yumin
>>>
>>> On 10/24/2014 1:20 PM, serguei.spitsyn at oracle.com wrote:
>>>> Hi Yumin,
>>>>
>>>> Below is some analysis to make sure I understand the test scenario 
>>>> correctly.
>>>>
>>>> The ParallelTransformerLoaderApp.main() executes a 1000 iteration loop.
>>>> At each iteration it does:
>>>>   - creates and starts a new TestThread
>>>>   - loads TestClass1 with the current class loader: 
>>>> ParallelTransformerLoaderAgent.getClassLoader()
>>>>   - changes the current class loader with new one: 
>>>> ParallelTransformerLoaderAgent.generateNewClassLoader()
>>>>
>>>> The TestThread loads the TestClass2 concurrently with the main thread.
>>>>
>>>> At the CFLH events, the ParallelTransformerLoaderAgent does the 
>>>> class retransformation.
>>>> If the thread loading the class is not "main", it loads the class 
>>>> TestClass3
>>>> with the current class loader 
>>>> ParallelTransformerLoaderAgent.getClassLoader().
>>>>
>>>> Sometimes, the TestClass2 and TestClass3 are loaded by the same 
>>>> class loader recursively.
>>>> It happens if the class loader has not been changed between loading 
>>>> TestClass2 and TestClass3 classes.
>>>>
>>>> I'm not convinced yet the test is incorrect.
>>>> And it is not clear why do we get a ClassCircularityError.
>>>>
>>>> Please, let me know if the above understanding is wrong.
>>>> I also see the reply from David and share his concerns.
>>>>
>>>> It is not clear if this failure is a regression.
>>>> Did we observe this issue before?
>>>> If - NOT then when and why had this failure started to appear?
>>>>
>>>> Unfortunately, it is impossible to look at the test run history at 
>>>> the moment.
>>>> The Aurora is at a maintenance.
>>>>
>>>> Thanks,
>>>> Serguei
>>>>
>>>> On 10/13/14 3:58 PM, Yumin Qi wrote:
>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8038468
>>>>> webrev:*http://cr.openjdk.java.net/~minqi/8038468/webrev00/
>>>>>
>>>>> the bug marked as confidential so post the webrev internally.
>>>>>
>>>>> Problem: The test case tries to load a class from the same jar via 
>>>>> agent in the middle of loading another class from the jar via same 
>>>>> class loader in same thread. The call happens in transform which 
>>>>> is a rare case --- in middle of loading class, loading another 
>>>>> class. The result is a CircularityError. When first class is in 
>>>>> loading, in vm we put JarLoader$2 on place holder table, then we 
>>>>> start the defineClass, which calls transform, begins loading the 
>>>>> second class so go along the same routine for loading JarLoader$2 
>>>>> first, found it already in placeholder table. A CircularityError 
>>>>> is thrown.
>>>>> Fix: The test case should not call loading class with same class 
>>>>> loader in same thread from same jar in 'transform' method. I 
>>>>> modify it loading with system class loader and we expect see 
>>>>> ClassNotFoundException. Detail see bug comments.
>>>>>
>>>>> Thanks
>>>>> Yumin *
>>>>
>>>
>>
>