Backport vmTestbase tests to jdk8u-dev?

Konstantin Shefov kshefov at azul.com
Mon Oct 7 10:10:54 UTC 2019


Hi. Liu Xin

Removing @modules is indeed not mandatory, it just strange to have this tag in JDK 8.

As for G1 GC tests, G1 GC in JDK8 has not been in production state yet, so it is expected that some tests from JDK 14 are failing against JDK 8. We have 4 tests failing, but we have not looked deeply into them yet.

Konstantin


On 07/10/2019, 10:07, "Liu, Xin" <xxinliu at amazon.com> wrote:

    Hi, Konstantin and other reviewers,  
    
    I used hg/mq to reformat those 7 patches.  It's a very useful tool to manage commits in hg. I can jump back and modify what I comitted. 
    
    
    $hg qapplied
    base
    updateAPIs
    classFileInstaller
    gclog
    nativeLibraries
    azul_contrib1
    azul_contrib2
    
    Could you review my changes? I didn't generate webrev because they are huge. 
    https://cr.openjdk.java.net/~xliu/8231089/
    
    @Konstantin,
    
    I move some changes to updateAPIs and gclog from your patches.  It's you who identified and solved that problems. I just try to collect one kind of problem into one patch. It's easier for reviewers. 
    I have some comments for your patches. 
    1) previously, you have deleted all @module.  Eg.
    /*
      * @test
    - * @modules java.base/jdk.internal.misc:+open
      * @key stress gc
      *
      * @summary converted from VM Testbase gc/g1/unloading/tests/unloading_anonclassloader_inMemoryCompilation_keep_class.
      * VM Testbase keywords: [gc, stress, stressopt, nonconcurrent, javac]
      *
    - * @modules java.base/jdk.internal.misc
      * @library /vmTestbase
      *          /testlibrary
      *          /test/lib
    @@ -40,7 +38,6 @@
      * @comment generate HumongousTemplateClass and compile it to test.classes
      * @run driver gc.g1.unloading.bytecode.GenClassesBuilder
      *
    - * @requires vm.gc.G1 
    
    I don't think it's mandatory. Jtreg automatically ignores unknown annotations like @module.  if you don't modify it, your patch will be dramatically smaller. 
    It's easy to review and reduce conflicts for future merges.  I also added property vm.gc.G1, which returns true unconditionally. 
    
    2. has you solved this issue? 
    https://gist.github.com/navyxliu/856ac2feab41591ec9220fb259d83072
    I still can't pass some tests under vmTestbase/gc/g1/unloading. They all failed with the same error code. it looks that G1GC unloaded some classes. 
    
    
    Thanks,
    --lx
    
    On 9/26/19, 4:52 AM, "Konstantin Shefov" <kshefov at azul.com> wrote:
    
        Hi, Liu Xin
        
        I have to change my previous answer about Platform.java.
        We cannot just remove it and change vmProps.java refer to jdk.testlibrary.Platform.
        
        The reasons are:
        1) There is no jdk.testlibrary.Platform class. We have com.oracle.java.testlibrary.Platform instead.
        2) com.oracle.java.testlibrary.Platform is NOT a superset of jdk.test.lib.Platform - it contains less methods.
        3) A lot of tests fail to compile because of absent methods in com.oracle.java.testlibrary.Platform.
        
        So if we want to get rid of "hostspot/test/test/lib" folder, we have to merge new methods into hotspot/test/testlibrary/com/oracle/java/testlibrary folder.
        
        The other solution is to leave everything as is, so use the "azul_contrib2" patch exactly as we provided it.
        
        Regards,
        Konstantin
        
        -----Original Message-----
        From: Konstantin Shefov 
        Sent: Thursday, September 26, 2019 12:52 PM
        To: 'Liu, Xin' <xxinliu at amazon.com>; jdk8u-dev at openjdk.java.net
        Subject: RE: Backport vmTestbase tests to jdk8u-dev?
        
        Hello
        
        I have no objections against removing Platform.java and using the existing one.
        
        Konstantin
        
        -----Original Message-----
        From: Liu, Xin <xxinliu at amazon.com> 
        Sent: Thursday, September 26, 2019 2:24 AM
        To: Konstantin Shefov <kshefov at azul.com>; jdk8u-dev at openjdk.java.net
        Subject: Re: Backport vmTestbase tests to jdk8u-dev?
        
        Hi, Konstantin,  
        
        I think you can also remove Platform.java in test/lib/jdk/test. 
        https://cr.openjdk.java.net/~xliu/8231089/azul_contrib2/webrev/test/test/lib/jdk/test/lib/Platform.java.patch
        and change vmProps.java refer to jdk.testlibrary.Platform. 
        https://cr.openjdk.java.net/~xliu/8231089/azul_contrib2/webrev/test/jtreg-ext/requires/VMProps.java.udiff.html
        
        It's because jdk.testlibrary.Platform is superset of the delete one. 
        Thanks, 
        --lx
        
        On 9/25/19, 3:20 AM, "Konstantin Shefov" <kshefov at azul.com> wrote:
        
            Hi
            
            Here is one more patch from Azul side: http://cr.openjdk.java.net/~kshefov/vmTestbase_8_azul_contrib_2/webrev/
            
            This patch is supposed to be applied just after "azul_contrib" patch sent by Liu Xin.
            
            The list of changes is the following:
            1) Removed unused code from hotspot/test/test/lib folder (we ran all tests after that, nothing breaks);
            2) Fixed some tests so they work with JDK 8 (they used JDK 13 API, so had to refactor);
            3) Removed tests that test "Compact String" feature, which is not available in JDK 8;
            4) Made a small change in test/JtregNativeHotspot.gmk file so that now native test libs are automatically copied to jdk8u-dev/build/<YOUR_BUILD_NAME>/images/test directory exactly as it is done in JDK 11+. In JDK 8 "TEST_IMAGE_DIR" variable is not defined, so it was replaced by "$(IMAGES_OUTPUTDIR)/test".
            
            With all changes provided by Liu Xin and the patch provided here we think it can be pushed.
            
            Thanks,
            --Konstantin
            
            -----Original Message-----
            From: Konstantin Shefov 
            Sent: Monday, September 23, 2019 6:30 PM
            To: 'Liu, Xin' <xxinliu at amazon.com>; 'jdk8u-dev at openjdk.java.net' <jdk8u-dev at openjdk.java.net>
            Subject: RE: Backport vmTestbase tests to jdk8u-dev?
            
            Hi, Liu Xin
            
            After applying all the patches, I have successfully built the native part, but I had to do some more changes.
            After all tests run, I will prepare one more patch "azul_contrib_2" and send it here.
            
            Konstantin
            
            -----Original Message-----
            From: Konstantin Shefov 
            Sent: Monday, September 23, 2019 4:47 PM
            To: 'Liu, Xin' <xxinliu at amazon.com>; jdk8u-dev at openjdk.java.net
            Subject: RE: Backport vmTestbase tests to jdk8u-dev?
            
            Hi, Liu Xin
            
            Thank you for combining our patch with yours, we will test the whole chain of patches to work at our side.
            
            As an answer to your question about modification from 0x21 to 0x2e at earlyretint.c.
            Java class file format has changed between JDK 8 and JDK 11+.
            In earlyretint.c from JDK 11+ we have the following line:
            
             "jlocation loc_exp = (i == 0) ? 0x21 : 0xd; "
            
            This "loc_exp" is an expected location of instruction inside a java method (see and [1] and [2]).
            By the test scenario we breakpoint inside countDownBoolean(I)V method after checkPoint()V method.
            In Java 8 the number of instruction before which we breakpoint is 0x2e (hex) and 46 (decimal).
            In Java 11+ the number of instruction before which we breakpoint is 0x21 (hex) and 33 (decimal).
            That is why we have to change the test here.
            
            [1] https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#SingleStep
            [2] https://docs.oracle.com/javase/8/docs/platform/jvmti/jvmti.html#jlocation
            
            Regards,
            Konstantin
            
            -----Original Message-----
            From: Liu, Xin <xxinliu at amazon.com> 
            Sent: Monday, September 23, 2019 10:13 AM
            To: Konstantin Shefov <kshefov at azul.com>; jdk8u-dev at openjdk.java.net
            Subject: Re: Backport vmTestbase tests to jdk8u-dev?
            
            hi, Konstantin, 
            
            I also apply your patch to the last webrev.
             
            eventually, JDK-8231089 consists of 6 patches.  now I think it's easier to review it. https://cr.openjdk.java.net/~xliu/8231089/
            
            We should apply them in order. 
            1. base
            2. apiChanges
            3. classFileInstaller
            4. gclog
            5. native_libraries
            6. azul_contrib
            
            for your patch, i keep it everything except for -Xloggc.
            Could you tell me why you modify from 0x21 to 0x2e? 
            diff -r 95eb898ada95 -r 6c35ac679aa8 test/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretint/earlyretint.c
            --- a/test/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretint/earlyretint.c	Thu Sep 19 17:30:08 2019 +0300
            +++ b/test/vmTestbase/nsk/jvmti/unit/ForceEarlyReturn/earlyretint/earlyretint.c	Thu Sep 19 17:35:58 2019 +0300
            @@ -89,7 +89,7 @@
                        jlocation loc, jint i) {
                 jvmtiError err;
                 jclass cls;
            -    jlocation loc_exp = (i == 0) ? 0x21 : 0xd;
            +    jlocation loc_exp = (i == 0) ? 0x2e : 0xd;
                 char *sigClass, *name, *sig, *generic;
                 jvmtiLocalVariableEntry *table = NULL;
                 jint entryCount = 0;
            
            ________________________________________
            From: jdk8u-dev <jdk8u-dev-bounces at openjdk.java.net> on behalf of Liu, Xin <xxinliu at amazon.com>
            Sent: Saturday, September 21, 2019 4:48 AM
            To: Konstantin Shefov; jdk8u-dev at openjdk.java.net
            Subject: Re: Backport vmTestbase tests to jdk8u-dev?
            
            Hi, Konstantin,
            
            As I promised, here is the refactored webrev for native compilation
            
            Hotspot:
            https://cr.openjdk.java.net/~xliu/8231089/native_libraries/hotspot/webrev/
            root:
            https://cr.openjdk.java.net/~xliu/8231089/native_libraries/root/webrev/
            so the new target 'test-image' is added.
            
            to fix https://gist.github.com/navyxliu/bacd47891b17ac8d33d04708b648b531,
            I added "LDFLAGS_SUFFIX := $$(LIBCXX), " to TestFilesCompilation.gmk.
            
            Thanks,
            --lx
            
            
            
            
            
            
            From: Konstantin Shefov <kshefov at azul.com>
            Date: Friday, September 20, 2019 at 11:01 AM
            To: "Liu, Xin" <xxinliu at amazon.com>, "jdk8u-dev at openjdk.java.net" <jdk8u-dev at openjdk.java.net>
            Subject: RE: Backport vmTestbase tests to jdk8u-dev?
            
            Hi, Liu Xin
            
            > I am refactoring makefiles. I plan to upload this webrev today.
            Fine! We will try to compile with our toolchains.
            
            >> BTW, some tests are extensions to JCK and check JVM Specification. The Specification in Java 14 has changed from that in Java 8.
            > Could you give me an example tests for this?
            
            One example is this
            https://github.com/navyxliu/corretto-8/blob/vmTestbase/src/hotspot/test/vmTestbase/vm/runtime/defmeth/ConflictingDefaultsTest.java#L329
            
            And most of vmTestbase/vm/runtime/defmeth tests seem to test JVM spec in some way.
            See here https://github.com/navyxliu/corretto-8/blob/vmTestbase/src/hotspot/test/vmTestbase/vm/runtime/defmeth/BasicTest.java#L50
            and here https://github.com/navyxliu/corretto-8/blob/vmTestbase/src/hotspot/test/vmTestbase/vm/runtime/defmeth/BasicTest.java#L134
            
            Konstantin
            
            From: Liu, Xin <xxinliu at amazon.com>
            Sent: Friday, September 20, 2019 8:35 PM
            To: Konstantin Shefov <kshefov at azul.com>; jdk8u-dev at openjdk.java.net
            Subject: Re: Backport vmTestbase tests to jdk8u-dev?
            
            Konstantin,
            
            I am refactoring makefiles. I plan to upload this webrev today.
            
            I found that you have solved almost all problems. I can help to troubleshoot the remaining tests.
            >BTW, some tests are extensions to JCK and check JVM Specification. The Specification in Java 14 has changed from that in Java 8.
            Could you give me an example tests for this?
            
            Thanks,
            --lx
            
            From: Konstantin Shefov <kshefov at azul.com<mailto:kshefov at azul.com>>
            Date: Friday, September 20, 2019 at 2:26 AM
            To: "Liu, Xin" <xxinliu at amazon.com<mailto:xxinliu at amazon.com>>, "jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>" <jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>>
            Subject: Re: Backport vmTestbase tests to jdk8u-dev?
            
            Hi,
            
            We will use your new patches to run tests and merge my changes with yours for those tests that will not work.
            
            BTW, some tests are extensions to JCK and check JVM Specification. The Specification in Java 14 has changed from that in Java 8.
            
            I see your webrevs do not contain native part build scripts for now. We have to address this problem so the scripts should work for older GCC versions also.
            
            Regards
            Konstantin
            
            From: "Liu, Xin" <xxinliu at amazon.com<mailto:xxinliu at amazon.com>>
            Date: Wednesday, 18 September 2019 at 20:59
            To: Konstantin Shefov <kshefov at azul.com<mailto:kshefov at azul.com>>, "jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>" <jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>>
            Subject: Re: Backport vmTestbase tests to jdk8u-dev?
            
            
            Hi, Konstantin,
            
            
            
            thank you for updating.
            
            
            
            We care about x86_64 and aarch64 as well. You ran more tests than us. we never run all vmTestbase.
            
            Previously, I used a ProblemList.txt file to screen Problem and move on. Do you plan to fix all problems before we check in?
            
            
            
            Back to refactor, I have filed a JBS issues.
            
            https://bugs.openjdk.java.net/browse/JDK-8231089
            
            In the meanwhile, I start splitting the commits to individual hg changesets.
            https://cr.openjdk.java.net/~xliu/8231089/
            
            1. base -> copy code from tag jdk-14+14
            intact commit for vmTestbase and testlib 2. make apiChanges for ProcessUtils and Unsafe Could you review it?
            
            3. ClassFileInstaller needs /testlibrary it's because ClassFileInstaller is in /test/testlibrary. I use this script to install @library /testlibrary for all tests.
            
            out=$(grep "ClassFileInstaller" -R vmTestbase | awk '{print $1}') for i in $out; do
                f=${i%\:}
                gsed -i 's/@library \/vmTestbase/&\n *          \/testlibrary/' $f
            done
            
            
            What do you think that this approach?
            If I got something wrong, we can update the webtrev with yours.
            
            thanks,
            --lx
            
            
            ________________________________
            From: Konstantin Shefov <kshefov at azul.com<mailto:kshefov at azul.com>>
            Sent: Wednesday, September 18, 2019 4:56 PM
            To: Liu, Xin; jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>
            Subject: RE: Backport vmTestbase tests to jdk8u-dev?
            
            Hi
            
            We ran the tests against x86_64 Ubuntu Linux and there are just the same failed tests as it was with Arm64 Ubuntu Linux plus 2 more tests:
            vmTestbase/gc/lock/jni/jnilock002/TestDescription.java
            vmTestbase/nsk/jdb/set/set001/set001.java
            
            Konstantin
            
            
            From: Konstantin Shefov
            Sent: Wednesday, September 11, 2019 10:53 AM
            To: 'Liu, Xin' <xxinliu at amazon.com<mailto:xxinliu at amazon.com>>; jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>
            Subject: RE: Backport vmTestbase tests to jdk8u-dev?
            
            Hi, LX
            
            Answering to your e-mail, as for what we modified in [3]:
            
              1.  API changes (Process, Unsafe), Xloggc option, that were not in your patches;
              2.  In a file hotspot/test/JtregNativeHotspot.gmk we changed "-lpthread" option to "-pthread" to prevent compilation error.
            
            I agree that there should be a series of patches to track file changes.
            
            As for investigation of failures: do you have the same? We ran against Arm64 platform, but we can try against x86-64 too.
            
            Konstantin
            
            From: Liu, Xin <xxinliu at amazon.com<mailto:xxinliu at amazon.com>>
            Sent: Monday, September 9, 2019 11:04 PM
            To: Konstantin Shefov <kshefov at azul.com<mailto:kshefov at azul.com>>; jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>
            Subject: Re: Backport vmTestbase tests to jdk8u-dev?
            
            Hi, Konstantin,
            
            We are glad to work with you to get vmTestbase landed into jdk8u.
            Our original intent is get design reviewed by the  jdk8u maintainer.  After then, we can proceed to refactor it.  We can work together to refactor it. we plan to do it anyway.
            
            Back to the webrev you send.  What did you modify in [3]?
            Ideally, we can should have a series of patches.  The first one should be intact vmTestbase code from the tip of JDK. All we changes should come after it and  be reviewed by this mail-list.
            In addition to refactoring, we can also offer help to figure out those failed cases. What do you think?
            
            Thanks,
            --lx
            
            
            From: Konstantin Shefov <kshefov at azul.com<mailto:kshefov at azul.com>>
            Date: Friday, September 6, 2019 at 12:55 PM
            To: "jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>" <jdk8u-dev at openjdk.java.net<mailto:jdk8u-dev at openjdk.java.net>>
            Cc: "Liu, Xin" <xxinliu at amazon.com<mailto:xxinliu at amazon.com>>
            Subject: Re: Backport vmTestbase tests to jdk8u-dev?
            
            Hi,
            
            We at Azul think it is a good idea to backport vmTestbase tests to OpenJDK 8.
            
            We have used patches you suggested, done some refactoring to enable more tests to work with JDK 8.
            
            We have run all 3550 tests from "vmTestbase" folder against Zulu 8 at ARM64 platform:
            
              1.  3371 tests PASS (see [1]);
              2.  179 tests FAIL (see [2]): reasons are being investigated.
            
            Patches we have used (contain our refactoring):
            
            1.       Hotspot folder [3]
            
            2.       Main folder [4]
            
            We plan to investigate causes of those 177 failures, purify unused code from hotspot/test/test/lib/jdk, and run the tests against different platforms.
            
            We also compared vmTestbase from JDK 11 with that from JDK 13, we found out:
            
              1.  332 test have been removed in JDK 13;
              2.  Only 2 test have been added in JDK 13;
              3.  Tests that remained were modified, but these modifications mostly are copyright years, or output format, or API changes.
            From the above results we can tell that vmTestbase does not grow. It looks like that tests are been removed from there or transferred somewhere else.
            
            [1] http://cr.openjdk.java.net/~kshefov/vmTestbase_8_backport/passed_tests.html
            [2] http://cr.openjdk.java.net/~kshefov/vmTestbase_8_backport/failed_tests.html
            [3] http://cr.openjdk.java.net/~kshefov/vmTestbase_8_backport/vmTestbase_hotspot.patch
            [4] http://cr.openjdk.java.net/~kshefov/vmTestbase_8_backport/vmTestbase_root.patch
            
            Regards,
            --Konstantin
            
            
            
        
        
    
    



More information about the jdk8u-dev mailing list