RFR: 8254042: gtest/GTestWrapper.java failed os.test_random [v2]

[Igor Ignatyev]

On Fri, 27 Nov 2020 00:59:35 GMT, David Holmes <dholmes at openjdk.org> wrote:

>> Coleen Phillimore has updated the pull request incrementally with two additional commits since the last revision:
>> 
>>  - Restore old copyright
>>  - Refix os.random test using Thomas Stuefe's better suggestion.
>
> Marked as reviewed by dholmes (Reviewer).

> _Mailing list message from [Thomas St��fe](mailto:thomas.stuefe at gmail.com) on [hotspot-dev](mailto:hotspot-dev at openjdk.java.net):_
> 
> On Thu, Nov 26, 2020 at 12:51 AM Kim Barrett <kim.barrett at oracle.com> wrote:
> 
> > > On Nov 25, 2020, at 12:25 AM, Thomas Stuefe <stuefe at openjdk.java.net>
> > > wrote:
> > > Why is someone concurrently changing the seed? I thought "TEST" tests do
> > > not start the VM? Or is it that some earlier test already did?
> > 
> > 
> > I was surprised by this too.
> > "TEST" tests and "TEST_VM" tests are collected in a single list and
> > run sequentially.  The first "TEST_VM" test that gets run by that will
> > trigger VM initialization, and all the remaining tests (whether "TEST"
> > or "TEST_VM") get run in the resulting context.
> > I don't much like that behavior.
> 
> I don't either. One of the problems I keep encountering is that I use TEST
> only to later see that I accidentally used VM infrastructure; but you only
> notice when executing the test in a different order.
> 
> googletests has all these nice options, eg --gtest_shuffle. Maybe we should
> randomize the execution order of these tests to shake loose these errors.
> 
> Cheers, Thome

there is an easier way to find all *current* errors, you can just run each test individually, so all `TEST` which should have been `TEST_VM` will fail. there is a script which I used to use to do that, and I actually thought we have integrated it alongside the prev. rounds of such clean.

as a historical note, one of the first iterations of JEP [281](https://bugs.openjdk.java.net/browse/JDK-8047975) ran `TEST` before all other tests, but at some point, we decided that the complexity this brought just didn't worth it. it can easily be a case that this decision wasn't right (or isn't right anymore), and we should be able to change the behavior to ensure that `TEST` tests are run w/o VM being initialized, the only problem I can foresee is w/ `--gtest_shuffle`.

Cheers,
-- Igor

-------------

PR: https://git.openjdk.java.net/jdk/pull/1422