April 2020 - core-libs-dev

RFR: 8242842: Avoid reallocating name when checking for trailing slash in ZipFile.getEntryPos
by Claes Redestad 15 Apr '20

15 Apr '20

Hi, a trivial optimization to ZipFile.getEntryPos extracted from some experiments conducted mostly by Eirik Bjørsnøs. Webrev: http://cr.openjdk.java.net/~redestad/8242842/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8242842 This removes an extra array copy on every miss, and the patch means a small speed-up on a few startup applications I've tested (Spring Petclinic: ~50ms faster). This is an optimization which was lost in translation when porting from native to Java[1] in JDK 9. The native implementation allocates an array that can fit the extra '/' if needed, and updates the array in place for the follow-up check. Thanks! /Claes [1] https://bugs.openjdk.java.net/browse/JDK-8145260

3 3

Improving ZipFile.getEntry performance using Bloom filters
by Eirik Bjørsnøs 15 Apr '20

15 Apr '20

Hi, While working on improvements to JarFile.getVersionedEntry, it occurred to me that the missing lookups we are removing there may be just one example of a more general issue. To get a better understanding of how ZipFile.getEntry is used, I added some PerfCounters for the number of hits and misses in calls to ZipFile.Source.getEntryPos() and the elapsed run times for these hits and misses. To test this I used Spring PetClinic. I only care about startup time here, since that's when most of the Zip lookups happens. Here's what I found: Hits 10776, misses: 744642 => 1.4 percent of lookups are hits Hits runtime: 7995508 ns, miss runtime: 328360900 ns => 2.4 percent of run time is spent on hits. The startup time reported by Spring Petclinic was 6395 ms. 5 percent of this startup time was spent on missed getEntry lookups. If we can improve the performance of lookup misses, I think there is a good potential for overall performance wins. One idea I had about how this could be done was to use Bloom filters. Bloom filters provide a fast, space-efficient, probabilistic data structure which may be used to determine that an element is definitely not a member of a set. I made a sloppy Bloom filter implementation and updated ZipFile.Source.getEntryPos to check this filter first and return -1 if the name is definitely not in the jar. This gave the following results for Spring Petclinic startup: Hits run time: 10964825 ns, miss runtime: 233868876 ns We see that while hits are now 1.4x slower, the total time spent on lookups is only 73 percent compared to that of the baseline. This translates to a Petclinic startup improvement of 91 ms or 1.4 percent (assuming single-threaded class loading here). I expect that this can be improved further with a more clever use of hash functions. Specifically it would be great to skip earlier in case of a miss, before the String to byte[] encoding happens. Also, I haven't analysed the false positive rate in the bloom filter, so that's probably open for some tuning. I also expect lookup misses to be even more common on applications with longer class paths and more complex class loader delegation. Cheers, Eirik.

4 10

promoting/moving @jdk.internal.PreviewFeature to be a standard JavaSE API
by Dmitry Bessonov 14 Apr '20

14 Apr '20

Hi, Noticed in the OpenJDK sources that String::stripIndent, String::translateEscapes, String::formatted were annotated with @jdk.internal.PreviewFeature until the very recent time [1]. Are there any plans to move this annotation out of “jdk” namespace to “java” or “javax” namespace? This movement looks logical as the concept of “Preview Feature” [2] is not JDK-specific - JEP12 has "Scope: SE” and “Exposure: Open". kind regards, dmitry [1] http://hg.openjdk.java.net/jdk/jdk/rev/60ec850952da#l1.7 [2] https://bugs.openjdk.java.net/browse/JDK-8195734

1 0

Reasoning behind exposing getGenericSignature in RecordComponent
by Rafael Winterhalter 14 Apr '20

14 Apr '20

Hello, I was wondering what the motivation was behind exposing the getGenericSignature method in RecordComponent. This method also exists in the Method and Field classes but is private there. If there is an argument to expose the method in RecordComponent, wouldn't it make sense to also expose it there? I am a bit surprised to find the method public in the first place, few people have a relation to class format concepts at runtime. Thanks for the clarification. Best regards, Rafael

1 0

RFR: 8242283: Can't start JVM when java home path includes non-ASCII character
by Yasumasa Suenaga 11 Apr '20

11 Apr '20

Hi all, Please review this change: JBS: https://bugs.openjdk.java.net/browse/JDK-8242283 webrev: http://cr.openjdk.java.net/~ysuenaga/JDK-8242283/webrev.00/ After JDK-8240197 and JDK-8240725, java cannot start when java home path includes non-ASCII character e.g. Japanese Kanji. We've discussed about this issue on JBS, and we decide to replace CP_THREAD_ACP to CP_ACP in MultiByteToWideChar() because CP_ACP works nicely rather than CP_THREAD_ACP. But this change would not resolve in all cases. For example, Japanese encoded pathname cannot be recognized under the English system locale. It is (implicitly) described in the spec for JNI_CreateJavaVM(). https://docs.oracle.com/en/java/javase/14/docs/specs/jni/invocation.html#jn…: ``` char *optionString; /* the option as a string in the default platform encoding */ ``` Thanks, Yasumasa

2 2

RFR 15 8242462: Residual Cleanup of rmic removal
by Roger Riggs 10 Apr '20

10 Apr '20

Please review a few cleanups I missed on the removal of rmic. I didn't get a reply on the changes to the hotspot (cds) tests. (I did get an ok on the javadoc changes) Webrev: http://cr.openjdk.java.net/~rriggs/webrev-remove-rmic-8225319-misc-1/ Issue: https://bugs.openjdk.java.net/browse/JDK-8242462 Thanks, Roger

4 3

JarFile.getVersionedEntry scalability with new release cadence
by Eirik Bjørsnøs 10 Apr '20

10 Apr '20

I recently needed to re-implement multi-release lookup logic for a ModuleReader capable of reading modules from unpacked (exploded) jar files [1] It occurred to me that JarFile.getVersionedEntry checks _every_ version between 8 and the runtime version when looking up paths. Since META-INF/versions will probably be sparsely populated, I'm wondering if something could be done to avoid checking 20 different paths in OpenJDK 28. Perhaps scanning META-INF/versions once when opening the file could work, then only check existing versions in getVersionedEntry? Maybe a premature optimization today, but with the new release cadence, this problem is going to surface at some point in the future, right? [1] https://mail.openjdk.java.net/pipermail/jigsaw-dev/2020-April/014414.html Eirik.

1 0

Re: os::javaTimeSystemUTC to call nanosecond precision OS API, so Clock.systemUTC() can give nanosecond precision UTC
by Roger Riggs 10 Apr '20

10 Apr '20

Hi, Created an entry to track the enhancement: https://bugs.openjdk.java.net/browse/JDK-8242504 Changes affecting the Core Libraries APIs also need be reviewed by core-libs-dev. Regards, Roger On 4/10/20 1:03 PM, Mark Kralj-Taylor wrote: > I'd like to help Java Clock.systemUTC() expose nanosecond precision > UTC realtime (wall-time) clock, where OS and CPU allow. > > Please let me know if this would be acceptable for OpenJDK, and what > next steps I can take to progress this towards submitting a patch. > - I have some time at the moment. I've signed the OCA, but don't have > login to the OpenJDK bugtracker, if you want me to submit a proper hg > patch there. > - This mail includes the patch as `hg diff` outputs together with JMH > and test output from it. > > The patch here improves the existing API java.time.Instant.now() i.e. > Clock.systemUTC().instant() on Linux to return Linux > clock_gettime(CLOCK_REALTIME) which can be nanosecond precision (and > very low cost, se JMH results) when Linux has a suitable clocksource > (tsc). > > The nice thing about this is that it doesn't add a new JDK API, nor > change semantics of the existing Clock.systemUTC().instant() API which > already supports nanosecond granularity. > > Clock.systemUTC().instant() is backed by Hotspot > os::javaTimeSystemUTC, which can be updated to call higher precision > OS real-time clock API when available (and has similar performance), > rather than the current code that multiplies microseconds from > gettimeofday() by 1000 to get nanoseconds. > > In Linux this means calling the Posix clock_gettime(CLOCK_REALTIME) > API instead of the older gettimeofday() whose output is limited to > microsecond precision. On a suitable CPU Linux can back > clock_gettime(CLOCK_REALTIME) with a nanosecond precision CPU TSC > clocksource. Whereas the output of gettimeofday() is limited to > microsecond precision (even when Linux is using a higher precision > clocksource). > > A similar change could be made for other OS that offer a performant > API to get hi-precision wall-time. > - OSX: Unfortunately on my Mac I found that > clock_gettime(CLOCK_REALTIME) is limited to microseconds precision > (source shows it calling gettimeofday() and multiplying by 1000 - see > note on recent Linux change to discourage that). An alternate approach > that did get nanosecond precision wall-time was too slow to be of > interest (>900ns!): Calling OSX host_get_clock_service CALENDAR_CLOCK > in JVM init, then clock_get_time(clockPortFromPrev,...). > - Windows: GetSystemTimePreciseAsFileTime looks promising. See > https://docs.microsoft.com/en-gb/windows/win32/api/sysinfoapi/nf-sysinfoapi… > . Unfortunately I don't have access to a Windows machine to try this > on, but if you are interested in the Linux patch I could buy windows > for a laptop we have to see how fares in JMH. > > Details, patch and JMH / test outputs follow below. > Thanks, > Mark > > ----------------- > > # RATIONAL: Why does Java need nanosecond precision real-time clock? > > Like many others working on lower-latency Java systems running on > Linux I have been using JNI to call clock_gettime(CLOCK_REALTIME, ...) > to do this for years. Hi-precision (nanosecond) UTC timestamps have > been essential in understanding and improving end-to-end latency and > performance of multi-process systems. Including nanosecond wall-time > after-receive and before-send timestamps in messages sent over the > wire (or shared memory, or in stored events) gives tremendous > transparency on latency in multi-process and multi-host event > processing and workflows. > > Java has evolved to remove more and more common cases for JNI. Getting > a nanosecond real-time clock timestamp feels a good candidate for that > process, especially because it looks like hi-precision real-time UTC > clock APIs are common in the minimum OS versions Java is built for > (even if some OS still give microsecond precision wall-time behind an > API that returns nanoseconds - as OSX 10.15 did for me - built with > XCode 10.1 as per JDK docs. > > Timestamps from a monotonic clock can only be compared within the same > host, or for Java's System.nanoTime() within the same Java process. > This is very useful, but doesn't help with multi-process, multi-host, > multi-language use-cases. > > Realtime (wall-time) UTC clock timestamps are useful because they can > be compared between different processes, possibly running on a > different host, as well as within a process. Within the same > data-centre latency between processes is typically well under a > millisecond for UDP / multicast between hosts. Latency can be > sub-microsecond for shared-memory between processes on the same host. > > A monotonic clock is often favoured for timing latency, because > real-time clocks on different hosts are subject to slewing and > stepping. Clock-sync technologies drastically reduce impact of these > on a host and also time offsets between different hosts (especially in > the same data centre). For high event rate systems you can look at > latency as percentiles over time-windows. With many time-windows you > get an idea of clock sync quality because you see the distribution > move when you look at a time-series of percentiles for time-windows if > different hosts have different clock stepping time synchronisation. > > ----------------- > > # IS THIS A SAFE CHANGE for the JDK? > > It makes no change to semantics of Clock.systemUTC(), whose JavaDoc > says: "This clock is based on the best available system clock. This > may use System.currentTimeMillis(), or a higher resolution clock if > one is available.". > > Because there is no new JDK API, nor change in API semantics. This > enhancement can be made on an OS by OS basis. > > ----------------- > > # IS THIS A SAFE CHANGE for Linux? > > The "clock_gettime(3) - Linux man page" says: "All implementations > support the system-wide realtime clock, which is identified by > CLOCK_REALTIME. Its time represents seconds and nanoseconds since the > Epoch.". See: https://linux.die.net/man/3/clock_gettime > > FYI Some older Linux ports used to implement > clock_gettime(CLOCK_REALTIME,) by calling gettimeofday() and > multiplying by 1000. This would be fine with the patch proposed, but > means that resolution would be limited to microseconds by the OS. A > Linux change 6 months ago has discouraged that in ports: "Use > clock_gettime to implement gettimeofday" > https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=5e46749c64d51f50f851… > . > > ----------------- > > # Related OpenJDK BugTrackers > > Comments on these enhancements requests suggest some interest in > calling Linux clock_gettime(REALTIME) intsead of gettimeofday(). > Although the enhancement requests themselves are not the same as this > mail, which asks to enhance an existing API, within its existing > semantics. > - JDK-6709908 > - JDK-8185891 > > Since Java 9 Clock.systemUTC() has up to microsecond resolution on Linux > - JDK-8068730 Increase the precision of the implementation of > java.time.Clock.systemUTC() > > ----------------- > > # POSSIBLE FOLLOW ON (that is NOT part of the patch or proposal discussed here) > > A possible follow-on would be to add a JDK API to give more direct > (lower cost) access to a hi-precision timestamp as a nanoseconds since > UTC epoc in a long. For example System.curretTimeNanos(). This would > be different to the proposal above in that its range would be limited > to year 2262 for a unsigned long, which is more than 200 years in the > future, but less than Instant.MAX. A JMH benchmarks with > systemTimeNanos() as an intrinsic (following the style of > currentTimeMillis) demonstrated that this is attractive, with similar > cost to System.currentTimeMillis()/nanoTime(), so better than > Instant.now() or JNI that I used. But initially I'd rather focus on > improving Java without adding any new APIs, or discussing if ~200 > years is enough of a range. > > ----------------- > > # PATCHES to Hotspot, tests and output of those tests > > ## PATCH to hotspot/os/linux/os_linux.cpp > > This patch changes both os::javaTimeMillis and os::javaTimeSystemUTC > to call clock_gettime(CLOCK_REALTIME, ..) instead of gettimeofday(). > This keeps the 2 methods obviously consistent, and avoids someone > reading the code having questions on the consistency of different > Linux APIs. Results form JMH benchmark (see below) show this is ok. > > This seams consistent with direction being taken by Linux, based on > change: "Use clock_gettime to implement gettimeofday" at > https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=5e46749c64d51f50f851… > > ``` > diff --git a/src/hotspot/os/linux/os_linux.cpp > b/src/hotspot/os/linux/os_linux.cpp > --- a/src/hotspot/os/linux/os_linux.cpp > +++ b/src/hotspot/os/linux/os_linux.cpp > @@ -88,6 +88,7 @@ > # include <errno.h> > # include <dlfcn.h> > # include <stdio.h> > +# include <time.h> > # include <unistd.h> > # include <sys/resource.h> > # include <pthread.h> > @@ -1374,18 +1375,18 @@ > } > jlong os::javaTimeMillis() { > - timeval time; > - int status = gettimeofday(&time, NULL); > + timespec time; > + int status = clock_gettime(CLOCK_REALTIME, &time); > assert(status != -1, "linux error"); > - return jlong(time.tv_sec) * 1000 + jlong(time.tv_usec / 1000); > + return jlong(time.tv_sec) * 1000 + jlong(time.tv_nsec / 1000000); > } > void os::javaTimeSystemUTC(jlong &seconds, jlong &nanos) { > - timeval time; > - int status = gettimeofday(&time, NULL); > + timespec time; > + int status = clock_gettime(CLOCK_REALTIME, &time); > assert(status != -1, "linux error"); > seconds = jlong(time.tv_sec); > - nanos = jlong(time.tv_usec) * 1000; > + nanos = jlong(time.tv_nsec); > } > void os::Linux::fast_thread_clock_init() { > ``` > > ## PATCH to test/micro JMH benchmark to assess impact of change on > Instant.now(), and show its performance relative to > System.currentTimeMillis()/nanoTime(). > > Where would the best place be to add an Instant.now() JMH bennchmark? > - While I'd guess the convention is to follow the package of the API > being benchmarked. I like that by putting benchmarks of system > timestamping APIs in one class its easy and natural to spot them all > and compare them. Could the benchmark be added here, then the class > renamed to SystemTime.java to focus on benchmarking those features? > > ``` > diff --git a/test/micro/org/openjdk/bench/java/lang/Systems.java > b/test/micro/org/openjdk/bench/java/lang/Systems.java > --- a/test/micro/org/openjdk/bench/java/lang/Systems.java > +++ b/test/micro/org/openjdk/bench/java/lang/Systems.java > @@ -28,6 +28,7 @@ > import org.openjdk.jmh.annotations.OutputTimeUnit; > import java.util.concurrent.TimeUnit; > +import java.time.Instant; > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @@ -43,4 +44,9 @@ > return System.nanoTime(); > } > + @Benchmark > + public long instant_now_asEpocNanos() { > + Instant now = Instant.now(); > + return now.getEpochSecond() * 1_000_000_000L + now.getNano(); > + } > ``` > > ### RESULTS from JDK Micro benchmark: > Run on Linux with clocksource=tsc on a 3GHz Intel i5 CPU (see details below). > > `make test TEST="micro:java.lang.Systems"` > > WITH change to os_linux.cpp: > ``` > Benchmark Mode Cnt Score Error Units > Systems.currentTimeMillis avgt 25 19.190 ? 0.166 ns/op > Systems.instant_now_asEpocNanos avgt 25 29.809 ? 0.191 ns/op > Systems.nanoTime avgt 25 18.534 ? 0.024 ns/op > ``` > > WITHOUT change to os_linux.cpp: (but with the added JMH benchmark for > purpose of comparison) > ``` > Benchmark Mode Cnt Score Error Units > Systems.currentTimeMillis avgt 25 19.013 ? 0.033 ns/op > Systems.instant_now_asEpocNanos avgt 25 30.459 ? 0.017 ns/op > Systems.nanoTime avgt 25 18.971 ? 0.053 ns/op > ``` > > ### Platform details: > System: Linux (Ubuntu) running on Mac Mini 2018 3 GHz Intel i5 > ``` > $ uname -a > Linux MacMiniLinux 5.3.0-42-generic #34-Ubuntu SMP Fri Feb 28 05:49:40 > UTC 2020 x86_64 x86_64 x86_64 GNU/Linux > > $ cat /sys/devices/system/clocksource/clocksource0/current_clocksource > tsc > > $ ldd --version > ldd (Ubuntu GLIBC 2.30-0ubuntu2.1) 2.30 > > $ lscpu | egrep '(Model name|CPU.s.:)' > CPU(s): 6 > Model name: Intel(R) Core(TM) i5-8500B CPU @ 3.00GHz > NUMA node0 CPU(s): 0-5 > > $ lscpu | grep Flags | sed 's@ @\n@g' | grep -i tsc > tsc > rdtscp > constant_tsc > nonstop_tsc > tsc_deadline_timer > tsc_adjust > ``` > > ## PATCH to JDK test code - that logs realtime clock precision: > > ``` > diff --git a/test/jdk/java/time/test/java/time/TestClock_System.java > b/test/jdk/java/time/test/java/time/TestClock_System.java > --- a/test/jdk/java/time/test/java/time/TestClock_System.java > +++ b/test/jdk/java/time/test/java/time/TestClock_System.java > @@ -177,7 +177,8 @@ > + formatTime("\n\thighest1", highest1)); > } > - int count=0; > + int count_betterThanMillisPrecision=0; > + int count_betterThanMicrosPrecision=0; > // let's preheat the system a bit: > int lastNanos = 0; > for (int i = 0; i < 1000 ; i++) { > @@ -191,7 +192,10 @@ > lastNanos = nanos; > if ((nanos % 1000000) > 0) { > - count++; // we have micro seconds > + count_betterThanMillisPrecision++; // we have microseconds > + } > + if ((nanos % 1000) > 0) { > + count_betterThanMicrosPrecision++; // we have nanoseconds > } > if ((sysnan % 1000000) > 0) { > throw new RuntimeException("Expected only millisecconds " > @@ -200,10 +204,12 @@ > } > } > System.out.println("\nNumber of time stamps which had better than" > - + " millisecond precision: "+count+"/"+1000); > + + " millisecond precision: "+count_betterThanMillisPrecision+"/"+1000); > + System.out.println("\nNumber of time stamps which had better than" > + + " microsecond precision: "+count_betterThanMicrosPrecision+"/"+1000); > System.out.println(formatTime("\nsystemUTC ", system1)); > System.out.println(formatTime("highestResolutionUTC ", highest1)); > - if (count == 0) { > + if (count_betterThanMillisPrecision == 0) { > System.err.println("Something is strange: no microsecond " > + "precision with highestResolutionUTC?"); > throw new RuntimeException("Micro second preccision not reached"); > ``` > > ### OUTPUT of JDK Test logs observed clock precision > > Extract of test log when test patch run on same host as JMH below. > Shows nanosecond precision (`999/1000 better than microsecond`). > > `make test TEST="jtreg:test/jdk/java/time/test/java/time/TestClock_System.java" > JTREG="VERBOSE=all"` > > ``` > Number of time stamps which had better than millisecond precision: 1000/1000 > > Number of time stamps which had better than microsecond precision: 999/1000 > > systemUTC : 2020-04-03T23:14:12.276Z - seconds: 1585955652, nanos: 276000000 > highestResolutionUTC : 2020-04-03T23:14:12.276472680Z - seconds: > 1585955652, nanos: 276472680 > ```

1 0

RFR 15 8242327: List spec should state that unmodifiable lists implement RandomAccess
by Stuart Marks 10 Apr '20

10 Apr '20

Hi all, Please review this small change to the List interface that specifies that unmodifiable lists (returned by List.of et al) implement the RandomAccess marker interface. In fact, they already do; the intent of this section was to specify the user-visible characteristics of such lists, but this particular characteristic was accidentally omitted. https://bugs.openjdk.java.net/browse/JDK-8242327 Patch below. I'll follow up with a CSR request. Thanks, s'marks # HG changeset patch # User smarks # Date 1586379408 25200 # Wed Apr 08 13:56:48 2020 -0700 # Node ID f0b78a923c2b2eadfc79cfec7f65377588f10574 # Parent 46108b5b69d92dd424b73b828454849c397509cb 8242327: List spec should state that unmodifiable lists implement RandomAccess Reviewed-by: XXX diff -r 46108b5b69d9 -r f0b78a923c2b src/java.base/share/classes/java/util/List.java --- a/src/java.base/share/classes/java/util/List.java Tue Apr 07 16:31:46 2020 -0700 +++ b/src/java.base/share/classes/java/util/List.java Wed Apr 08 13:56:48 2020 -0700 @@ -104,6 +104,7 @@ * <li>They are serializable if all elements are serializable. * <li>The order of elements in the list is the same as the order of the * provided arguments, or of the elements in the provided array. + * <li>The lists and their {@link subList subList} views implement the {@link RandomAccess} interface. * <li>They are <a href="../lang/doc-files/ValueBased.html">value-based</a>. * Callers should make no assumptions about the identity of the returned instances. * Factories are free to create new instances or reuse existing ones. Therefore,

2 3

RFR [15/java.xml] 8242470 : Upgrade Xerces to Version 2.12.1
by Joe Wang 09 Apr '20

09 Apr '20

Please review an update to Xerces 2.12.1. Xerces 2.12.1 was a bug fix release. Bugs in the release were either already in the JDK or not applicable. This patch therefore includes only an update to the md file. --- a/src/java.xml/share/legal/xerces.md +++ b/src/java.xml/share/legal/xerces.md @@ -1,4 +1,4 @@ -## Apache Xerces v2.12.0 +## Apache Xerces v2.12.1 ### Apache Xerces Notice <pre> @@ -8,9 +8,11 @@ ========================================================================= Apache Xerces Java - Copyright 1999-2018 The Apache Software Foundation + Copyright 1999-2020 The Apache Software Foundation + This product includes software developed at The Apache Software Foundation (http://www.apache.org/) + Portions of this software were originally based on the following: - software copyright (c) 1999, IBM Corporation., http://www.ibm.com. - software copyright (c) 1999, Sun Microsystems., http://www.sun.com. Thanks, Joe

3 2