RFR: JDK-8221596: test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java failed with FileAlreadyExistsException

[Jie Fu]

Hi Misha,

Thanks for your deep analysis.

Actually, this bug was originally detected by a script like this
----------------------------------------------
rm JTreport JTwork -rf

i=0
while(true); do

   let i++
   echo -e "\nRun ${i} ...\n"

   ${JTREG}/bin/jtreg \
     -jdk:${TESTJDK} \
     ${DOCKER_ARGS} \
     test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java

done
----------------------------------------------

This script was used to reproduce random bugs triggered by jtreg tests 
for our mips-port.
Yes, we do rm JT* before the test, but would not like to do that during 
the loop.

It would take much more time to reproduce and fix bugs like that if we 
had to rm JT* every time.
So I think it's worth fixing it.

Thanks a lot.

Best regards,
Jie


On 2019/3/29 上午5:50, Mikhailo Seledtsov wrote:
> Hi Jie, David,
>
> I did a bit of digging:
>
>    - @run driver ClassFileInstaller -jar whitebox.jar 
> sun.hotspot.WhiteBox sun.hotspot.WhiteBox$WhiteBoxPermission
>      is used in TestCPUSets.java, TestMemoryAwareness.java and 
> TestMisc.java
>      This instruction will generate whitebox.jar and place it in 
> "current working jtreg directory", which is
>      ./JTwork/scratch/
>
>    - in order to be able to use the WhiteBox.jar from within the 
> Docker container, it is copied
>      to the "JTReg test classes" directory (defined by 
> jdk.test.lib.utils.TEST_CLASSES)
>      This Utils.TEST_CLASSES directory is mounted via --volume at 
> docker-run-time, so that classes can be
>      visible/usable/loadable from within the container.
>
>    -  This Utils.TEST_CLASSES directory is unique for each test run:
> ./JTwork/classes/runtime/containers/docker/TestMisc.d/whitebox.jar
> ./JTwork/classes/runtime/containers/docker/TestMemoryAwareness.d/whitebox.jar
> ./JTwork/classes/runtime/containers/docker/TestCPUSets.d/whitebox.jar
>       Hence, I do not see a possibility of file access clash among the 
> different tests.
>
>    - if tests are run concurrently (with -conc:X feature), each test 
> gets their own scratch directory
>      classes directories are also separated, as in non-concurrent case
>      ./JTwork/scratch/1/whitebox.jar
>      ./JTwork/scratch/0/whitebox.jar
>      ./JTwork/scratch/2/whitebox.jar
> ./JTwork/classes/3/runtime/containers/docker/TestCPUSets.d/whitebox.jar
> ./JTwork/classes/4/runtime/containers/docker/TestMisc.d/whitebox.jar
> ./JTwork/classes/0/runtime/containers/docker/TestMemoryAwareness.d/whitebox.jar
>      No file access clashes among tests either
>
>    Is it possible that you did not clean out the directory from the 
> previous run? For test execution, we recommend
>    clearning JT* directories from the prior run before a new run (rm 
> -Rf JT* or similar). This recommendation is not
>    specific to Docker testing, it is a good practice in general for 
> executing of jtreg tests.
>
>
>
> Thank you,
> Misha
>
> On 3/28/19, 4:16 AM, David Holmes wrote:
>> Hi Jie,
>>
>> On 28/03/2019 5:48 pm, Jie Fu wrote:
>>> Hi David,
>>>
>>> Thanks for your review.
>>>
>>>>
>>>> Normally to use WhiteBox a test specifies:
>>>>
>>>>  @run driver ClassFileInstaller sun.hotspot.WhiteBox
>>> Yes, we do have it in 
>>> test/hotspot/jtreg/runtime/containers/docker/TestCPUSets.java
>>
>> Okay I'm not understanding why Common is copying the WhiteBox classes 
>> or where it is copying them to? If this is something to do with the 
>> image shouldn't it be completely clean when the test is run - either 
>> by deleting existing files or by ensuring a unique location to begin 
>> with?
>>
>> Thanks,
>> David
>>
>>> Best regard,
>>> Jie
>>>>
>>>> Is there a reason this test can't use that?
>>>>
>>>> Thanks,
>>>> David
>>>> -----
>>>