Testing <was> Re: RFR 8143628: Fork sun.misc.Unsafe and jdk.internal.misc.Unsafe native method tables

[Paul Sandoz]

Hi Coleen,

Thanks for the useful information.

I am surprised that the Java class used by jtreg to execute the test in the generated script gets in the way of debugging the VM since it’s all executed from within the same process.

Within a few minutes I was able to successfully debug in the VM using jtreg + testng on my Mac by prefixing the java command with "lldb --“. [*] The annoying part was actually dealing with the “$” character for the launcher class "com.sun.javatest.regtest.TestNGAction$TestNGRunner” :-)

With a little more time i replaced the jtreg launcher with the explicit testng launcher (just to reduce a layer, if it makes any difference). However, this is harder than it should be because the version of testng bundled with jtreg is missing stuff.

It does involve the jtreg and testing launcher but is that really a major issue? If the test fails when running with the launcher class then presumably debugging wise it’s also useful to run with that too? but i can understand in certain cases it may be preferable not to do that. Overall, with my admittedly more JDK core libraries goggles, on the above does not seems that different to your existing process.

—

Perhaps jtreg could be enhanced to make it easier to execute the test under gdb or lldb?

Paul.

[*] Here is an example requiring two modifications to the script jtreg output:

DISPLAY=/private/tmp/com.apple.launchd.dfh3MfqE9D/org.macosforge.xquartz:0 \
HOME=/Users/sandoz \
JTREG_HOME=/Users/sandoz/Applications/jtreg \
LANG=en_GB.UTF-8 \
PATH=/bin:/usr/bin \
lldb -- /Users/sandoz/Projects/jdk9/hs-comp/build/macosx-x86_64-normal-server-release/jdk/bin/java \
        -Dtest.class.path.prefix=/Users/sandoz/Projects/jdk9/hs-comp/jdk/JTwork/classes/java/util/Arrays:/Users/sandoz/Projects/jdk9/hs-comp/jdk/test/java/util/Arrays \
        -Dtest.src=/Users/sandoz/Projects/jdk9/hs-comp/jdk/test/java/util/Arrays \
        -Dtest.src.path=/Users/sandoz/Projects/jdk9/hs-comp/jdk/test/java/util/Arrays \
        -Dtest.classes=/Users/sandoz/Projects/jdk9/hs-comp/jdk/JTwork/classes/java/util/Arrays \
        -Dtest.class.path=/Users/sandoz/Projects/jdk9/hs-comp/jdk/JTwork/classes/java/util/Arrays \
        -Dtest.vm.opts='-ea -esa' \
        -Dtest.tool.vm.opts='-J-ea -J-esa' \
        -Dtest.compiler.opts= \
        -Dtest.java.opts= \
        -Dtest.jdk=/Users/sandoz/Projects/jdk9/hs-comp/build/macosx-x86_64-normal-server-release/jdk \
        -Dcompile.jdk=/Users/sandoz/Projects/jdk9/hs-comp/build/macosx-x86_64-normal-server-release/jdk \
        -Dtest.timeout.factor=1.0 \
        -classpath /Users/sandoz/Applications/jtreg/lib/javatest.jar:/Users/sandoz/Applications/jtreg/lib/jtreg.jar:/Users/sandoz/Applications/jtreg/lib/testng.jar:/Users/sandoz/Projects/jdk9/hs-comp/jdk/JTwork/classes/java/util/Arrays:/Users/sandoz/Projects/jdk9/hs-comp/jdk/test/java/util/Arrays \
        com.sun.javatest.regtest.TestNGAction\\\$TestNGRunner java/util/Arrays/ArraysEqCmpTest.java ArraysEqCmpTest


> On 3 Dec 2015, at 00:23, Coleen Phillimore <coleen.phillimore at oracle.com> wrote:
> 
> 
> Thank you Stefan for making the distinction between debugging Java inside an IDE vs. the JVM.
> 
> On 12/2/15 6:00 PM, Paul Sandoz wrote:
>> Hi Stefan,
>> 
>>> On 2 Dec 2015, at 17:20, Stefan Särne <stefan.sarne at oracle.com> wrote:
>>> 
>>> 
>>> Hi Paul,
>>> 
>>> The reason we stick on standard jtreg tests is because it is simpler.
>>> For us, a java test is not a unit test, it is an application.  :)
>>> 
>> I tend to think of that as an artificial distinction since such java test classes often contain a logical grouping of tests (and perhaps data over which to test) and make test assertions. Let’s call it duck unit testing, it looks and quacks like a unit test :-)
>> 
>>> I agree with you that when writing and debugging java code, I would choose testng over jtreg and run and debug it inside my java IDE.
>> In the case of the JDK it's not jtreg over testng it is jtreg + testng.
> 
> We already use jtreg and do not welcome another layer on the java class files.
> 
> When debugging a jtreg failure, I go to the "rerun" section in the .jtr file.  Edit it for 10 minutes to remove the extra '\' at the end and remove jtreg so that I get simply:
> 
> export CLASSPATH=...
> java <some options> ClassFile
> 
> and save it to a rerun.sh file.
> 
> I haven't figured out which -D test option I need for the test, but try to remove most of them also.
> 
> Generally, I debug like:
> 
> gdb --args $JAVA_HOME/bin/java <some options> ClassFile
> 
> What we're interested in is generally in the ClassFile and not in the jtreg java code.
> 
> I do not want to spend an additional 10 minutes to hack out testng.
> 
> Also, if the test fails and has a agent.jar file, that's usually found in the directory below, so I have to copy my rerun.sh file there.
>> 
>> 
>>> But debugging the VM is instead done with a native debugger and what the framework gives you for java development, becomes a level of indirection in VM land. Just adding the test class as argument to the java launcher where a main method exists is preferred.
>>> 
>> How do HotSpot engineers debug the VM with a jtreg test that uses @library (or @module once Jigsaw gets integrated), or uses WhiteBox, or uses ProcessTools?
> 
> See above.  Luckily CLASSPATH points to it in the end of the jtr file.
> 
> So no.  We do not want testng harness code added to the the Hotspot runtime tests.
> 
> Coleen
>> 
>> Paul.
>