RFR: JDK-8221596: test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java failed with FileAlreadyExistsException

[mikhailo.seledtsov at oracle.com]

David, Jie,

   Thank you for providing additional details. This makes sense now, I 
agree with this fix.


Thank you,

Misha


On 3/28/19 9:44 PM, Jie Fu wrote:
> Thanks David.
>
> On 2019/3/29 下午12:24, David Holmes wrote:
>> Hi Jie,
>>
>> On 29/03/2019 12:30 pm, Jie Fu wrote:
>>> Hi Misha,
>>>
>>> Thanks for your deep analysis.
>>>
>>> Actually, this bug was originally detected by a script like this
>>> ----------------------------------------------
>>> rm JTreport JTwork -rf
>>>
>>> i=0
>>> while(true); do
>>>
>>>    let i++
>>>    echo -e "\nRun ${i} ...\n"
>>>
>>>    ${JTREG}/bin/jtreg \
>>>      -jdk:${TESTJDK} \
>>>      ${DOCKER_ARGS} \
>>> test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java
>>>
>>> done
>>> ----------------------------------------------
>>>
>>> This script was used to reproduce random bugs triggered by jtreg 
>>> tests for our mips-port.
>>> Yes, we do rm JT* before the test, but would not like to do that 
>>> during the loop.
>>>
>>> It would take much more time to reproduce and fix bugs like that if 
>>> we had to rm JT* every time.
>>> So I think it's worth fixing it.
>>
>> You're likely to find other tests that also have issues being re-run 
>> this way. For example I just found runtime/appcds/DirClasspathTest.
>>
>> I was somewhat surprised to find that not deleting the work directory 
>> saved about 33% on the runtime (for the appcds tests: 18m vs. 29m).
>>
>> So this fix seems okay to me - and it won't hurt anyone who does 
>> always clear JTwork first.
>>
>> Thanks,
>> David
>>
>>> Thanks a lot.
>>>
>>> Best regards,
>>> Jie
>>>
>>>
>>> On 2019/3/29 上午5:50, Mikhailo Seledtsov wrote:
>>>> Hi Jie, David,
>>>>
>>>> I did a bit of digging:
>>>>
>>>>    - @run driver ClassFileInstaller -jar whitebox.jar 
>>>> sun.hotspot.WhiteBox sun.hotspot.WhiteBox$WhiteBoxPermission
>>>>      is used in TestCPUSets.java, TestMemoryAwareness.java and 
>>>> TestMisc.java
>>>>      This instruction will generate whitebox.jar and place it in 
>>>> "current working jtreg directory", which is
>>>>      ./JTwork/scratch/
>>>>
>>>>    - in order to be able to use the WhiteBox.jar from within the 
>>>> Docker container, it is copied
>>>>      to the "JTReg test classes" directory (defined by 
>>>> jdk.test.lib.utils.TEST_CLASSES)
>>>>      This Utils.TEST_CLASSES directory is mounted via --volume at 
>>>> docker-run-time, so that classes can be
>>>>      visible/usable/loadable from within the container.
>>>>
>>>>    -  This Utils.TEST_CLASSES directory is unique for each test run:
>>>> ./JTwork/classes/runtime/containers/docker/TestMisc.d/whitebox.jar
>>>> ./JTwork/classes/runtime/containers/docker/TestMemoryAwareness.d/whitebox.jar 
>>>>
>>>> ./JTwork/classes/runtime/containers/docker/TestCPUSets.d/whitebox.jar
>>>>       Hence, I do not see a possibility of file access clash among 
>>>> the different tests.
>>>>
>>>>    - if tests are run concurrently (with -conc:X feature), each 
>>>> test gets their own scratch directory
>>>>      classes directories are also separated, as in non-concurrent case
>>>>      ./JTwork/scratch/1/whitebox.jar
>>>>      ./JTwork/scratch/0/whitebox.jar
>>>>      ./JTwork/scratch/2/whitebox.jar
>>>> ./JTwork/classes/3/runtime/containers/docker/TestCPUSets.d/whitebox.jar 
>>>>
>>>> ./JTwork/classes/4/runtime/containers/docker/TestMisc.d/whitebox.jar
>>>> ./JTwork/classes/0/runtime/containers/docker/TestMemoryAwareness.d/whitebox.jar 
>>>>
>>>>      No file access clashes among tests either
>>>>
>>>>    Is it possible that you did not clean out the directory from the 
>>>> previous run? For test execution, we recommend
>>>>    clearning JT* directories from the prior run before a new run 
>>>> (rm -Rf JT* or similar). This recommendation is not
>>>>    specific to Docker testing, it is a good practice in general for 
>>>> executing of jtreg tests.
>>>>
>>>>
>>>>
>>>> Thank you,
>>>> Misha
>>>>
>>>> On 3/28/19, 4:16 AM, David Holmes wrote:
>>>>> Hi Jie,
>>>>>
>>>>> On 28/03/2019 5:48 pm, Jie Fu wrote:
>>>>>> Hi David,
>>>>>>
>>>>>> Thanks for your review.
>>>>>>
>>>>>>>
>>>>>>> Normally to use WhiteBox a test specifies:
>>>>>>>
>>>>>>>  @run driver ClassFileInstaller sun.hotspot.WhiteBox
>>>>>> Yes, we do have it in 
>>>>>> test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java
>>>>>
>>>>> Okay I'm not understanding why Common is copying the WhiteBox 
>>>>> classes or where it is copying them to? If this is something to do 
>>>>> with the image shouldn't it be completely clean when the test is 
>>>>> run - either by deleting existing files or by ensuring a unique 
>>>>> location to begin with?
>>>>>
>>>>> Thanks,
>>>>> David
>>>>>
>>>>>> Best regard,
>>>>>> Jie
>>>>>>>
>>>>>>> Is there a reason this test can't use that?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> David
>>>>>>> -----
>>>>>>
>>>
>