[aarch64-port-dev ] JTREG, JCStress, SPECjbb2015 and Hadoop/Terasort results for OpenJDK 9 on AArch64

Thu Jan 26 15:41:21 UTC 2017

Hi,
   I've been making some changes to how we're running JTRegs to reduce
the number of errors. I'm compiling JTreg with asmtools and building
and packaging the native libraries for the jdk and langtools tests
that require them. Increasing the timeout factor by 8x has removed
most of the rest of the errors, however, all of the remaining errors
are timeouts. These are all runs with the default options, except for
when the unit tests set options themselves, or when we are selecting
between running with C1 or C2.

Regards,
    Stuart

On 25 January 2017 at 22:36,  <ci_notify at linaro.org> wrote:
> This is a summary of the JTREG test results
> ===========================================
>
> The build and test results are cycled every 15 days.
>
> For detailed information on the test output please refer to:
>
>   http://openjdk.linaro.org/jdk9/openjdk-jtreg-nightly-tests/summary/2017/024/summary.html
>
> -------------------------------------------------------------------------------
> client-release/hotspot
> -------------------------------------------------------------------------------
> Build 0: aarch64/2016/dec/27 pass: 1,273; fail: 25; error: 77
> Build 1: aarch64/2016/dec/28 pass: 1,275; fail: 24; error: 78
> Build 2: aarch64/2016/dec/30 pass: 1,272; fail: 25; error: 80
> Build 3: aarch64/2017/jan/03 pass: 1,273; fail: 24; error: 80
> Build 4: aarch64/2017/jan/04 pass: 1,271; fail: 25; error: 81
> Build 5: aarch64/2017/jan/06 pass: 1,270; fail: 24; error: 83
> Build 6: aarch64/2017/jan/07 pass: 1,269; fail: 24; error: 84
> Build 7: aarch64/2017/jan/09 pass: 1,270; fail: 24; error: 83
> Build 8: aarch64/2017/jan/10 pass: 1,267; fail: 25; error: 85
> Build 9: aarch64/2017/jan/11 pass: 1,271; fail: 24; error: 82
> Build 10: aarch64/2017/jan/12 pass: 1,270; fail: 24; error: 83
> Build 11: aarch64/2017/jan/13 pass: 1,266; fail: 25; error: 86
> Build 12: aarch64/2017/jan/14 pass: 1,296; fail: 24; error: 57
> Build 13: aarch64/2017/jan/23 pass: 1,368; fail: 4; error: 5
> Build 14: aarch64/2017/jan/24 pass: 1,367; fail: 5; error: 5
>
> -------------------------------------------------------------------------------
> client-release/jdk
> -------------------------------------------------------------------------------
> Build 0: aarch64/2016/dec/28 pass: 7,209; fail: 636; error: 65
> Build 1: aarch64/2016/dec/30 pass: 7,234; fail: 619; error: 57
> Build 2: aarch64/2017/jan/03 pass: 7,206; fail: 634; error: 70
> Build 3: aarch64/2017/jan/04 pass: 7,227; fail: 622; error: 64
> Build 4: aarch64/2017/jan/06 pass: 7,217; fail: 638; error: 59
> Build 5: aarch64/2017/jan/07 pass: 7,207; fail: 637; error: 70
> Build 6: aarch64/2017/jan/09 pass: 7,205; fail: 640; error: 69
> Build 7: aarch64/2017/jan/10 pass: 7,208; fail: 648; error: 58
> Build 8: aarch64/2017/jan/11 pass: 7,200; fail: 651; error: 64
> Build 9: aarch64/2017/jan/12 pass: 7,209; fail: 650; error: 61
> Build 10: aarch64/2017/jan/13 pass: 7,210; fail: 654; error: 61
> Build 11: aarch64/2017/jan/14 pass: 7,234; fail: 630; error: 62
> Build 12: aarch64/2017/jan/22 pass: 7,234; fail: 630; error: 62
> Build 13: aarch64/2017/jan/23 pass: 7,271; fail: 628; error: 24
> Build 14: aarch64/2017/jan/24 pass: 7,269; fail: 641; error: 25
>
> -------------------------------------------------------------------------------
> client-release/langtools
> -------------------------------------------------------------------------------
> Build 0: aarch64/2016/dec/28 pass: 3,782; fail: 2; error: 29
> Build 1: aarch64/2016/dec/30 pass: 3,786; fail: 1; error: 26
> Build 2: aarch64/2017/jan/03 pass: 3,789; fail: 3; error: 23
> Build 3: aarch64/2017/jan/04 pass: 3,795; fail: 2; error: 20
> Build 4: aarch64/2017/jan/06 pass: 3,795; fail: 1; error: 22
> Build 5: aarch64/2017/jan/07 pass: 3,795; fail: 1; error: 22
> Build 6: aarch64/2017/jan/09 pass: 3,788; fail: 1; error: 29
> Build 7: aarch64/2017/jan/10 pass: 3,790; fail: 1; error: 27
> Build 8: aarch64/2017/jan/11 pass: 3,789; fail: 2; error: 29
> Build 9: aarch64/2017/jan/12 pass: 3,791; fail: 2; error: 28
> Build 10: aarch64/2017/jan/13 pass: 3,796; fail: 1; error: 25
> Build 11: aarch64/2017/jan/14 pass: 3,806; error: 17
> Build 12: aarch64/2017/jan/22 pass: 3,806; error: 17
> Build 13: aarch64/2017/jan/23 pass: 3,820; fail: 1; error: 11
> Build 14: aarch64/2017/jan/24 pass: 3,825; fail: 1; error: 6
>
> -------------------------------------------------------------------------------
> server-release/hotspot
> -------------------------------------------------------------------------------
> Build 0: aarch64/2016/dec/28 pass: 1,274; fail: 27; error: 79
> Build 1: aarch64/2016/dec/30 pass: 1,273; fail: 27; error: 80
> Build 2: aarch64/2017/jan/03 pass: 1,270; fail: 28; error: 82
> Build 3: aarch64/2017/jan/04 pass: 1,270; fail: 27; error: 83
> Build 4: aarch64/2017/jan/06 pass: 1,270; fail: 27; error: 83
> Build 5: aarch64/2017/jan/07 pass: 1,271; fail: 27; error: 82
> Build 6: aarch64/2017/jan/09 pass: 1,271; fail: 27; error: 82
> Build 7: aarch64/2017/jan/10 pass: 1,268; fail: 28; error: 84
> Build 8: aarch64/2017/jan/11 pass: 1,269; fail: 28; error: 83
> Build 9: aarch64/2017/jan/12 pass: 1,267; fail: 27; error: 86
> Build 10: aarch64/2017/jan/13 pass: 1,291; fail: 28; error: 61
> Build 11: aarch64/2017/jan/14 pass: 1,294; fail: 27; error: 59
> Build 12: aarch64/2017/jan/22 pass: 1,294; fail: 27; error: 59
> Build 13: aarch64/2017/jan/23 pass: 1,367; fail: 8; error: 5
> Build 14: aarch64/2017/jan/24 pass: 1,367; fail: 7; error: 6
>
> -------------------------------------------------------------------------------
> server-release/jdk
> -------------------------------------------------------------------------------
> Build 0: aarch64/2016/dec/28 pass: 7,221; fail: 646; error: 43
> Build 1: aarch64/2016/dec/30 pass: 7,228; fail: 629; error: 53
> Build 2: aarch64/2017/jan/03 pass: 7,223; fail: 633; error: 54
> Build 3: aarch64/2017/jan/04 pass: 7,228; fail: 627; error: 58
> Build 4: aarch64/2017/jan/06 pass: 7,229; fail: 643; error: 42
> Build 5: aarch64/2017/jan/07 pass: 7,223; fail: 639; error: 52
> Build 6: aarch64/2017/jan/09 pass: 7,226; fail: 639; error: 49
> Build 7: aarch64/2017/jan/10 pass: 7,234; fail: 627; error: 53
> Build 8: aarch64/2017/jan/11 pass: 7,238; fail: 623; error: 54
> Build 9: aarch64/2017/jan/12 pass: 7,240; fail: 631; error: 49
> Build 10: aarch64/2017/jan/13 pass: 7,251; fail: 617; error: 57
> Build 11: aarch64/2017/jan/14 pass: 7,240; fail: 624; error: 62
> Build 12: aarch64/2017/jan/22 pass: 7,240; fail: 624; error: 62
> Build 13: aarch64/2017/jan/23 pass: 7,278; fail: 622; error: 23
> Build 14: aarch64/2017/jan/24 pass: 7,253; fail: 642; error: 40
>
> -------------------------------------------------------------------------------
> server-release/langtools
> -------------------------------------------------------------------------------
> Build 0: aarch64/2016/dec/28 pass: 3,784; fail: 1; error: 28
> Build 1: aarch64/2016/dec/30 pass: 3,783; fail: 1; error: 29
> Build 2: aarch64/2017/jan/03 pass: 3,785; fail: 1; error: 29
> Build 3: aarch64/2017/jan/04 pass: 3,786; fail: 1; error: 30
> Build 4: aarch64/2017/jan/06 pass: 3,785; fail: 1; error: 32
> Build 5: aarch64/2017/jan/07 pass: 3,786; fail: 1; error: 31
> Build 6: aarch64/2017/jan/09 pass: 3,785; fail: 1; error: 32
> Build 7: aarch64/2017/jan/10 pass: 3,787; fail: 1; error: 30
> Build 8: aarch64/2017/jan/11 pass: 3,791; fail: 1; error: 28
> Build 9: aarch64/2017/jan/12 pass: 3,788; fail: 1; error: 32
> Build 10: aarch64/2017/jan/13 pass: 3,792; fail: 1; error: 29
> Build 11: aarch64/2017/jan/14 pass: 3,800; error: 23
> Build 12: aarch64/2017/jan/22 pass: 3,800; error: 23
> Build 13: aarch64/2017/jan/23 pass: 3,825; error: 7
> Build 14: aarch64/2017/jan/24 pass: 3,824; error: 8
>
> Previous results can be found here:
>
>   http://openjdk.linaro.org/jdk9/openjdk-jtreg-nightly-tests/index.html
>
>
> SPECjbb2015 composite regression test completed
> ===============================================
>
> This test measures the relative performance of the server
> compiler running the SPECjbb2015 composite tests and compares
> the performance against the baseline performance of the server
> compiler taken on 2016-11-21.
>
> In accordance with [1], the SPECjbb2015 tests are run on a system
> which is not production ready and does not meet all the
> requirements for publishing compliant results. The numbers below
> shall be treated as non-compliant (nc) and are for experimental
> purposes only.
>
> Relative performance: Server max-jOPS (nc): 0.86x
> Relative performance: Server critical-jOPS (nc): 0.75x
>
> Details of the test setup and historical results may be found here:
>
>     http://openjdk.linaro.org/jdk9/SPECjbb2015-results/
>
> [1] http://www.spec.org/fairuse.html#Academic
>
> Regression test Hadoop-Terasort completed
> =========================================
>
> This test measures the performance of the server and client compilers
> running Hadoop sorting a 1GB file using Terasort and compares
> the performance against the baseline performance of the Zero interpreter
> and against the baseline performance of the client and server compilers
> on 2014-04-01.
>
> Relative performance: Zero: 1.0, Client: 69.88, Server: 106.13
>
> Client 69.88 / Client 2014-04-01 (43.00): 1.63x
> Server 106.13 / Server 2014-04-01 (71.00): 1.49x
>
> Details of the test setup and historical results may be found here:
>
>     http://openjdk.linaro.org/jdk9/hadoop-terasort-benchmark-results/
>
> This is a summary of the jcstress test results
> ==============================================
>
> The build and test results are cycled every 15 days.
>
> 2016-12-28 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2016/362/results/
> 2016-12-29 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2016/363/results/
> 2016-12-31 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2016/365/results/
> 2017-01-04 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/003/results/
> 2017-01-05 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/004/results/
> 2017-01-07 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/006/results/
> 2017-01-08 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/007/results/
> 2017-01-10 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/009/results/
> 2017-01-11 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/010/results/
> 2017-01-12 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/011/results/
> 2017-01-13 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/012/results/
> 2017-01-14 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/013/results/
> 2017-01-15 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/014/results/
> 2017-01-24 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/023/results/
> 2017-01-25 pass rate: 6050/6050, results: http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/2017/024/results/
>
> For detailed information on the test output please refer to:
>
>   http://openjdk.linaro.org/jdk9/jcstress-nightly-runs/
>
>