RFR: 8254042: gtest/GTestWrapper.java failed os.test_random [v2]

[Thomas Stuefe]

On Fri, 27 Nov 2020 06:56:33 GMT, Igor Ignatyev <iignatyev at openjdk.org> wrote:

>> Marked as reviewed by dholmes (Reviewer).
>
>> _Mailing list message from [Thomas St��fe](mailto:thomas.stuefe at gmail.com) on [hotspot-dev](mailto:hotspot-dev at openjdk.java.net):_
>> 
>> On Thu, Nov 26, 2020 at 12:51 AM Kim Barrett <kim.barrett at oracle.com> wrote:
>> 
>> > > On Nov 25, 2020, at 12:25 AM, Thomas Stuefe <stuefe at openjdk.java.net>
>> > > wrote:
>> > > Why is someone concurrently changing the seed? I thought "TEST" tests do
>> > > not start the VM? Or is it that some earlier test already did?
>> > 
>> > 
>> > I was surprised by this too.
>> > "TEST" tests and "TEST_VM" tests are collected in a single list and
>> > run sequentially.  The first "TEST_VM" test that gets run by that will
>> > trigger VM initialization, and all the remaining tests (whether "TEST"
>> > or "TEST_VM") get run in the resulting context.
>> > I don't much like that behavior.
>> 
>> I don't either. One of the problems I keep encountering is that I use TEST
>> only to later see that I accidentally used VM infrastructure; but you only
>> notice when executing the test in a different order.
>> 
>> googletests has all these nice options, eg --gtest_shuffle. Maybe we should
>> randomize the execution order of these tests to shake loose these errors.
>> 
>> Cheers, Thome
> 
> there is an easier way to find all *current* errors, you can just run each test individually, so all `TEST` which should have been `TEST_VM` will fail. there is a script which I used to use to do that, and I actually thought we have integrated it alongside the prev. rounds of such clean.
> 
> as a historical note, one of the first iterations of JEP [281](https://bugs.openjdk.java.net/browse/JDK-8047975) ran `TEST` before all other tests, but at some point, we decided that the complexity this brought just didn't worth it. it can easily be a case that this decision wasn't right (or isn't right anymore), and we should be able to change the behavior to ensure that `TEST` tests are run w/o VM being initialized, the only problem I can foresee is w/ `--gtest_shuffle`.
> 
> Cheers,
> -- Igor

I created https://bugs.openjdk.java.net/browse/JDK-8257226 to track that RFE

-------------

PR: https://git.openjdk.java.net/jdk/pull/1422