RFR: CODETOOLS-7902418: jtreg fails to guarantee a cleaned scratch directory sometimes on windows and CODETOOLS-7901369: jtreg should be able to clean up read-only files under result dir

[Jonathan Gibbons]

Pushed.

-- Jon


On 03/07/2019 07:38 AM, Langer, Christoph wrote:
> Thanks, Jon.
>
> Looking forward to test the final fix in our CI.
>
>> -----Original Message-----
>> From: Jonathan Gibbons <jonathan.gibbons at oracle.com>
>> Sent: Mittwoch, 6. März 2019 17:12
>> To: Langer, Christoph <christoph.langer at sap.com>; code-tools-
>> dev at openjdk.java.net
>> Cc: Zeller, Arno <arno.zeller at sap.com>
>> Subject: Re: RFR: CODETOOLS-7902418: jtreg fails to guarantee a cleaned
>> scratch directory sometimes on windows and CODETOOLS-7901369: jtreg
>> should be able to clean up read-only files under result dir
>>
>> Christoph,
>>
>> Thanks for looking at this. It looks ok, but I'll take a more detailed
>> look later today, and will sponsor the change unless there is reason to
>> get back to you.
>>
>> -- Jon
>>
>> On 3/6/19 7:47 AM, Langer, Christoph wrote:
>>> Hi,
>>>
>>> here’s the fix that I actually wanted to build/test when I saw the Cygwin
>> issue…
>>> In our CI test infrastructure, we use jtreg in agentvm mode to run tests in
>> parallel. On Windows we regularly see an issue that bubbles up because the
>> 2 tests:
>> jdk/modules/scenarios/automaticmodules/RunWithAutomaticModules.java
>> jdk/modules/scenarios/overlappingpackages/OverlappingPackagesTest.java
>>> are run one after another in the same agent vm, hence in the same scratch
>> directory.
>>> The issue boils down to the Windows DeleteFileW API which is used under
>> the covers of java.io.File::delete. It does not always guarantee an
>> instantaneous deletion of a file. When there are handles open to a file, it'll
>> set a delete on close flag and defer deletion to a point when the last open
>> handle is closed.
>>> The problematic behavior in the tests is that the first test
>> (RunWithAutomaticModules.java) would create a jar file to be used on the
>> module path. Due to some compilation operation, I think the process keeps
>> the handle to the jar file open. When the next test
>> (OverlappingPackagesTest.java) is run (in the same VM that still holds the
>> open file handle), jtreg empties the scratch directory and thinks it is
>> successful. But then the test stumbles over the zombie file which is still there
>> and cannot be accessed.
>>> Jtreg can be improved to check if a file was really deleted after the call to
>> File::delete.
>>> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/CODETOOLS-
>> 7902418.0/
>>> Bug: https://bugs.openjdk.java.net/browse/CODETOOLS-7902418
>>>
>>> I verified that this will fix the issue that we observe. I’d think that the
>> performance overhead for an additional call to File::exists is neglectable.
>>> This would also resolve this bug:
>> https://bugs.openjdk.java.net/browse/CODETOOLS-7901369 which is
>> possible now as https://bugs.openjdk.java.net/browse/CODETOOLS-
>> 7901611 has been resolved, updating the jtreg baseline to JDK 1.7.
>>> If you like the fix, I would also need a sponsor ��
>>>
>>> Thanks & Best regards
>>> Christoph
>>>