RFR: JDK-8221596: test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java failed with FileAlreadyExistsException

[Jie Fu]

Thanks David.

On 2019/3/29 下午12:24, David Holmes wrote:
> Hi Jie,
>
> On 29/03/2019 12:30 pm, Jie Fu wrote:
>> Hi Misha,
>>
>> Thanks for your deep analysis.
>>
>> Actually, this bug was originally detected by a script like this
>> ----------------------------------------------
>> rm JTreport JTwork -rf
>>
>> i=0
>> while(true); do
>>
>>    let i++
>>    echo -e "\nRun ${i} ...\n"
>>
>>    ${JTREG}/bin/jtreg \
>>      -jdk:${TESTJDK} \
>>      ${DOCKER_ARGS} \
>> test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java
>>
>> done
>> ----------------------------------------------
>>
>> This script was used to reproduce random bugs triggered by jtreg 
>> tests for our mips-port.
>> Yes, we do rm JT* before the test, but would not like to do that 
>> during the loop.
>>
>> It would take much more time to reproduce and fix bugs like that if 
>> we had to rm JT* every time.
>> So I think it's worth fixing it.
>
> You're likely to find other tests that also have issues being re-run 
> this way. For example I just found runtime/appcds/DirClasspathTest.
>
> I was somewhat surprised to find that not deleting the work directory 
> saved about 33% on the runtime (for the appcds tests: 18m vs. 29m).
>
> So this fix seems okay to me - and it won't hurt anyone who does 
> always clear JTwork first.
>
> Thanks,
> David
>
>> Thanks a lot.
>>
>> Best regards,
>> Jie
>>
>>
>> On 2019/3/29 上午5:50, Mikhailo Seledtsov wrote:
>>> Hi Jie, David,
>>>
>>> I did a bit of digging:
>>>
>>>    - @run driver ClassFileInstaller -jar whitebox.jar 
>>> sun.hotspot.WhiteBox sun.hotspot.WhiteBox$WhiteBoxPermission
>>>      is used in TestCPUSets.java, TestMemoryAwareness.java and 
>>> TestMisc.java
>>>      This instruction will generate whitebox.jar and place it in 
>>> "current working jtreg directory", which is
>>>      ./JTwork/scratch/
>>>
>>>    - in order to be able to use the WhiteBox.jar from within the 
>>> Docker container, it is copied
>>>      to the "JTReg test classes" directory (defined by 
>>> jdk.test.lib.utils.TEST_CLASSES)
>>>      This Utils.TEST_CLASSES directory is mounted via --volume at 
>>> docker-run-time, so that classes can be
>>>      visible/usable/loadable from within the container.
>>>
>>>    -  This Utils.TEST_CLASSES directory is unique for each test run:
>>> ./JTwork/classes/runtime/containers/docker/TestMisc.d/whitebox.jar
>>> ./JTwork/classes/runtime/containers/docker/TestMemoryAwareness.d/whitebox.jar 
>>>
>>> ./JTwork/classes/runtime/containers/docker/TestCPUSets.d/whitebox.jar
>>>       Hence, I do not see a possibility of file access clash among 
>>> the different tests.
>>>
>>>    - if tests are run concurrently (with -conc:X feature), each test 
>>> gets their own scratch directory
>>>      classes directories are also separated, as in non-concurrent case
>>>      ./JTwork/scratch/1/whitebox.jar
>>>      ./JTwork/scratch/0/whitebox.jar
>>>      ./JTwork/scratch/2/whitebox.jar
>>> ./JTwork/classes/3/runtime/containers/docker/TestCPUSets.d/whitebox.jar
>>> ./JTwork/classes/4/runtime/containers/docker/TestMisc.d/whitebox.jar
>>> ./JTwork/classes/0/runtime/containers/docker/TestMemoryAwareness.d/whitebox.jar 
>>>
>>>      No file access clashes among tests either
>>>
>>>    Is it possible that you did not clean out the directory from the 
>>> previous run? For test execution, we recommend
>>>    clearning JT* directories from the prior run before a new run (rm 
>>> -Rf JT* or similar). This recommendation is not
>>>    specific to Docker testing, it is a good practice in general for 
>>> executing of jtreg tests.
>>>
>>>
>>>
>>> Thank you,
>>> Misha
>>>
>>> On 3/28/19, 4:16 AM, David Holmes wrote:
>>>> Hi Jie,
>>>>
>>>> On 28/03/2019 5:48 pm, Jie Fu wrote:
>>>>> Hi David,
>>>>>
>>>>> Thanks for your review.
>>>>>
>>>>>>
>>>>>> Normally to use WhiteBox a test specifies:
>>>>>>
>>>>>>  @run driver ClassFileInstaller sun.hotspot.WhiteBox
>>>>> Yes, we do have it in 
>>>>> test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java
>>>>
>>>> Okay I'm not understanding why Common is copying the WhiteBox 
>>>> classes or where it is copying them to? If this is something to do 
>>>> with the image shouldn't it be completely clean when the test is 
>>>> run - either by deleting existing files or by ensuring a unique 
>>>> location to begin with?
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>> Best regard,
>>>>> Jie
>>>>>>
>>>>>> Is there a reason this test can't use that?
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>> -----
>>>>>
>>