July 2015 - core-libs-dev

RFR 9: 8078099 : (process) ProcessHandle should uniquely identify processes
by Roger Riggs 07 Jul '15

07 Jul '15

Please review changes to ProcessHandle implementation to uniquely identify processes based on the start time provided by the OS. It addresses the issue of PID reuse. This is the implementation of the ProcessHandle.equals() spec change in 8129344 : (process) ProcessHandle instances should define equals and be value-based Webrev: http://cr.openjdk.java.net/~rriggs/webrev-starttime-8078099/ Issue: https://bugs.openjdk.java.net/browse/JDK-8078099 Thanks, Roger

3 6

Re: RFR: JDK-8080679: Include jline in JDK for Java and JavaScript REPLs
by Ståle W Pedersen 07 Jul '15

07 Jul '15

hi, sorry, im a bit late to this discussion, i wasnt aware of any interest for adding jline2 to the jdk until late this week. im working on a project called æsh/aesh: https://github.com/aeshell/aesh its another java console project derived from jline/jline2. it started as an extension/fork of jline2 and evolved into a separate project. im not quite sure what the reason behind choosing jline2 for the jdk, but i just wanted to give some info on æsh and what it does compared to jline2. æsh has all the same functionalities as jline2 and has a number of additional features: - history (search, persistence) - undo/redo - paste buffer - redirect/pipeline - alias +++ jline2 has several bugs regarding input that is resolved in æsh. one of the bigger differences between jline2 and æsh is that æsh isnt locking in the same way as jline2, but rather has a callback when input is given. this give better flexibility for the user when creating commands. perhaps the biggest difference is that æsh provides two apis for the developer. one is more low-level and similar to jline2: https://gist.github.com/stalep/cea6fbfee1de2b1ac6e4 plus a high level api that æsh provides for the developer to create commands. https://gist.github.com/stalep/98103347df1dd9b67fd1 in this example we create a ls command that will automatically complete filenames/directories and inject the values into the list field. this it just a very simple example, the api supports converters/completers/activators/renders/+++ we added this api because we noticed that the developers needed to write a lot of code to parse user input. with this api they can focus on creating commands and not parse for input data. if there is any interest in using æsh in the jdk we would do whatever we can to help you integrate it. æsh has evolved fairly quickly and tries to resolve any bugs/issues as quickly as possible. it is also supported by red hat so we can promise that this project will not be abandoned any time soon. regards, ståle

2 1

RFR JDK-8075526: Need a way to read and write time in UTC
by Xueming Shen 06 Jul '15

06 Jul '15

Hi, Please help review and comment on this rfe. Issue: https://bugs.openjdk.java.net/browse/JDK-8075526 webrev: http://cr.openjdk.java.net/~sherman/8075526 Background info: The title of the RFE is a little "mis-leading". All the existing set/get date-time methods actually work with "UTC" time. Both the old pair public void setTime(long time); public long getTime(); and the newly introduced pair pubic ZipEntry set/getLastModifiedTime(FileTime time); public FileTime getLastModifiedTime(); are to set/get the last modified time in UTC time. However the ZIP specification clearly specifies that the "normal" date and time fields in the zip file entry (loc and cen) are defined as the date/time in dos format, which unfortunately is a "local" date-time. Therefor timezone conversion must be applied before/after the utc time can be set into/got from those fields (the UTC timestamps set/get by the new pair are therefor being set into/got from the "extended timestamp fields" of the optional extra data of each entry, those fields are specified as unix/utc timestamp) We did not have an "local-date-time" abstract before the java.time.LocalDateTime was introduced in jdk8, the epoc date/time is the only date/time abstract in java vm. The proposed change here is to add yet another pair of set/get modified time methods public void setTimeLocale(LocalDateTime time); public LocalDateTime getTimeLocal(); to use the java.time.LocalDateTime type to set/get the modified time into zip entry's dos timestamp fields directly WITHOUT involving the timezone conversion (implied, with default TimeZone). Other than solving the pack/unpack problem raised in this RFE, it should also help improve the performance when local-date-time is actually desired when interfacing with the ZipEntry by eliminating the un-necessary/implied timezone conversion. For example, in our jar tool, currently we are "printing" the timestamp for zip entry "e" as new Date(e.getTime()).toString(); in which we are converting the local-date-time (ms-dos-formatted in zip entry) to utc time by using the default timezone (in ZipEntry), and then converting the utc time (in Date) back to the printable "local date time" again. It might be desired to format the "local-date-time" directly without involving the timezone conversion now via the proposed method java.time.format.DateTimeFormatter.ofPattern("EEE MMM dd HH:mm:ss zzz yyyy") .withZone(java.time.ZoneId.systemDefault()); .format(e.getTimeLocal() In above example, we still use the "system default timezone", however, it is used purely to output the zone name for the "zzz" (which the Date.toString() does), not for conversion. if the "zzz" is not required/needed, it can just be java.time.format.DateTimeFormatter.ofPattern("EEE MMM dd HH:mm:ss yyyy").format(e.getTimeLocal()); Comment/Opinion please. If we agree the approach/webrev, I will submit the CCC before integration. Thanks, -Sherman

3 8

Bad interaction between wildcard and functional interface conversion
by Remi Forax 05 Jul '15

05 Jul '15

Hi all, The way the conversion between a lambda (or a method reference) and a functional interface is specified doesn't take wildcard (exactly ? super) into account making the concept of contravariance of functional interface less intuitive that it should be. The following code compiles: private static void create(Consumer<Consumer<String>> consumer) { consumer.accept(s -> System.out.println(s)); } This one doesn't compile because "? super Consumer<? super String>" is not a functional interface: private static void create2(Consumer<? super Consumer<? super String>> consumer) { consumer.accept(s -> System.out.println(s)); } The workaround is to introduce a cast :( private static void create3(Consumer<? super Consumer<? super String>> consumer) { consumer.accept((Consumer<String>)s -> System.out.println(s)); } which is stupid in this case because there is no ambiguity. This cast is just here because the JLS doesn't consider that ? super Consumer<...> is a valid target type IMO, this bug is very similar to JDK-6964923 and i think the spec should be changed to allow ? super Foo to be a valid target type for a lambda conversion (obviously if Foo is a functional interface). regards, Rémi

2 3

Exception handling in Files.walk and Files.find
by Peter Levart 03 Jul '15

03 Jul '15

Hi, Recently I wanted to use Files.walk method returning a Stream<Path> to scan a directory of files. I quickly learned that it is of limited use. For example, the following code: long pngFilesCount = Files.walk(Paths.get("/usr")) .filter(path -> path.toString().endsWith(".png")) .count(); Throws exception on my computer: Exception in thread "main" java.io.UncheckedIOException: java.nio.file.AccessDeniedException: /usr/share/polkit-1/rules.d This is a consequence of the fact that some files and/or directories in the tree being scanned are not accessible to my user account on the computer. If I wanted to skip those files/directories I would be forced to use different API (Files.walkFileTree using FileVisitor which can handle exceptions too). I propose the addition of overloads to Files.walk and Files.find that take an additional parameter of type Consumer<IOException> to be responsible for handling IOExceptions that are thrown while walking the file tree: http://cr.openjdk.java.net/~plevart/jdk9-dev/Files.walkIOException/webrev.0… Ignoring AccessDeniedException(s) in above code can therefore be coded as: long pngFilesCount = Files.walk(Paths.get("/usr"), Integer.MAX_VALUE, ioe -> { if (!(ioe instanceof AccessDeniedException)) { throw new UncheckedIOException(ioe); } }) .filter(path -> path.toString().endsWith(".png")) .count(); What do you think? Regards, Peter

2 2

JEP 132: More-prompt finalization
by Peter Levart 03 Jul '15

03 Jul '15

Hi, Did you know that the following simple loop: public class FinalizableBottleneck { static boolean no; @Override protected void finalize() throws Throwable { // empty finalize() method does not make the object finalizable // (it is not even registered on the finalizer's list) if (no) { throw new AssertionError(); } } public static void main(String[] args) { while (true) { new FinalizableBottleneck(); } } } ...quickly fills the entire heap with FinalizableBottleneck and internal Finalizer objects and brings the JVM to a halt? After a few seconds of running the above program, jmap -histo:live reports: num #instances #bytes class name ---------------------------------------------- 1: 50048325 2001933000 java.lang.ref.Finalizer 2: 50048278 800772448 FinalizableBottleneck There are a couple of bottlenecks that make this happen: - ReferenceHandler thread synchronizes with VM to unhook Reference(s) from the pending chain one be one and dispatches them to their respected ReferenceQueue(s) which also use synchronization for equeueing each Reference. - Enqueueing synchronizes with the finalization thread which removes the Finalizer(s) (FinalReferences) from the finalization queue and executes them. - Executing the Finalizer(s) removes them from the doubly-linked list of all Finalizer(s) which is used to retain them until they are needed and this synchronizes with the threads that link new Finalizer(s) into the doubly-linked list as new finalizable objects get registered. We see that the creation of a finalizable object only takes one synchronization (registering into the doubly-linked list) and is performed synchronously, while finalization takes 4 synchronizations among 4 different threads (in pairs) and happens when the Finalizer instance "travels" over from VM thread to ReferenceHandler thread and then to finalization thread. No wonder that finalization can not keep up with allocation in a single thread. The situation is even worse when finalize() methods do some actual work. I have experimented with various approaches to widen these bottlenecks and found out that I can not beat the ForkJoinPool when combined with some improvements to internal data structures used in reference processing. Here's a prototype I came up with: http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.01/ And this is the benchmark I use for measuring the throughput: http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/Finalizer… The benchmark shows (results inline in source) that using unpatched JDK, on my PC (i7-2700K, Linux, JDK8) I can not construct more than 1500 finalizable objects per ms in a single thread and that while doing so, finalization only manages to process approx. 100 - 120 objects at the same time. Objects "in-flight" quickly accumulate and bring the VM to a halt, where it is not doing anything but full GC cycles. When constructing in 4 threads, there's not much difference. Construction of finalizable objects simply doesn't scale. Patched JDK shows something completely different. Single thread construction achieves a rate of 3600 objects / ms. Number of "in-flight" objects is kept constant at about 5-6M instances which amounts to approx 1.5 s of allocation. I think this is about the rate of GC cycles during which VM also processes the references. The benchmark also shows the ForkJoinPool statistics which shows that the number of queued tasks is also kept low. Increasing the allocation threads to 4 increases allocation rate to about 4300 objects / ms and finalization keeps up. Increasing allocation threads to 8, further increases allocation rate to about 4600 objects / ms and finalization still keeps up. The increase in rate is not linear, but keep in mind that i7 is a 4-core CPU. About the implementation... 1st improvement I did was for the doubly-linked list of Finalizer instances that is used to keep them alive until they are needed. I ripped-off the wonderful ConcurrentLinkedDeque by Doug Lea and Martin Buchholz and just kept the internal link/unlink methods while specializing them to Finalizer entries (very straight-forward). I experimented with throughput and got some improvement, but throughput has increased much more when I used several instances of independent lists and distributed registrations among them randomly (unlinking consequently is also distributed randomly). I found out that no matter how hard I try to optimize ReferenceQueue while keeping the API unchanged, I can only do so much and that was not enough. I have been surprised by how well ForkJoinPool distributes tasks among threads, so I concluded that leveraging it is the best choice. I re-designed the pending-list unhooking loop to unhook pending references in chunks which greatly improves the throughput. Since unhooking can be performed by a single thread while holding a lock which is mandated by interface between VM and Java, I didn't employ multiple threads, but a single eternal ForkJoinTask that unhooks in chunks and forks-off other processing tasks that process chunks. When there are just a couple of References pending at one time and a not-full chunk is unhooked, then the processing is performed by the same thread that unhooked the refrences, but when there are more, worker tasks are forked off and the unhooking thread continues with full peace. This processing includes execution of Cleaners, forking the finalizer tasks and enqueue-ing other references. Finalizer(s) are always executed as separate ForkJoinTask(s). It's interesting how Runtime.runFinalizers() is implemented in this patch - it basically amounts to ForkJoinPool.awaitQuiescence() ... I also tweaked the ReferenceQueue implementation a bit (it is still used for other kinds of references) so that it avoids synchronization with a monitor lock when there are no blocking waiters and uses CAS to enqueue/dequeue. This improves throughput when the queue is not empty. Since in the prototype multiple threads can enqueue into the same queue, I thought this would improve throughput in such situations. Comments, suggestions, criticism are welcome. Regards, Peter

7 25

[9] RFR(M): 8076112: Add @HotSpotIntrinsicCandidate annotation to indicate methods for which Java Runtime has intrinsics
by Zoltán Majó 02 Jul '15

02 Jul '15

Hi, please review the patch for JDK-8076112. Bug: https://bugs.openjdk.java.net/browse/JDK-8076112 Problem: There is need to indicate Java methods that are potentially intrinsified by JVM. Solution: Mark intrinsified methods with the jdk.internal.HotSpotIntrinsicCandidate annotation. Add checks that are omitted by VM-level intrinsics to the library code. Add a new diagnostic flag, CheckIntrinsics. If CheckIntrinsics is enabled, the VM performs the following checks when a class C is loaded: - all intrinsics defined by the VM for class C are present in the loaded class file and are marked; - an intrinsic is defined by the VM for all marked methods of C. If a mismatch is detected, the following is done: - a fastdebug VM prints a warning and then exits; - a product VM prints a warning and unmarked are not intrinsified. Webrev: - top: http://cr.openjdk.java.net/~zmajo/8076112/top/webrev.05/ - jdk: http://cr.openjdk.java.net/~zmajo/8076112/jdk/webrev.05/ - hotspot: http://cr.openjdk.java.net/~zmajo/8076112/hotspot/webrev.05/ Testing: - JPRT run with the 'hotspot' testset, all tests pass; - all JTREG hotspot tests, all tests pass that pass with a VM version that does not include the patch; all tests were run with -XX:+CheckIntrinsics. Thank you and best regards, Zoltan

7 20

RFR: JDK-8080679: Include jline in JDK for Java and JavaScript REPLs
by Jan Lahoda 02 Jul '15

02 Jul '15

Hello, I am proposing to add JLine 2.12.1 into the jdk repository for use by the Java and Nashorn REPLs. Full patch is available here: http://cr.openjdk.java.net/~jlahoda/8080679/webrev.00/full/ To aid the review, I've split this patch into to smaller patches: -a patch that only adds unmodified jline sources at appropriate places in the jdk repository: http://cr.openjdk.java.net/~jlahoda/8080679/webrev.00/clean-jline/ -a patch that shows the additional changes I've done: http://cr.openjdk.java.net/~jlahoda/8080679/webrev.00/additional/ This split is intended solely to simplify reviewing, my plan is to integrate this as a single patch. The main additional changes are: -plugging the new module, jdk.jline, into the JDK build. Currently, the JLine packages are exported only to jdk.scripting.nashorn (the plan is to also export them to the future jdk.jshell module). (The patch is not adding the dependency from jdk.scripting.nashorn to jdk.jline, though - I expect that to be added when needed.) -the sources are re-packaged from package "jline" to "jdk.internal.jline" -removing trailing whitespace, adding newlines at the end of the files, encoding characters that are not ASCII -avoiding the dependency on another library, jansi, by reimplementing two elements that were used from the other library. These are mainly the changes in WindowsTerminal and ConsoleReader.java. This also includes the WindowsTerminal.cpp native library. The native part is heavily inspired by: http://cr.openjdk.java.net/~sherman/rl/src/java.base/windows/native/libjava… As I am not experienced in native programming, comments to the native part would be particularly useful. -changes to resolve javac warnings in JLine. -tests for some of the added functionality. Any comments are welcome! Thanks, Jan

7 28

Re: RFR(XXS): 8130315: Fix wrong prototype of GrowKnownVMs() in java.c
by Alan Bateman 02 Jul '15

02 Jul '15

On 02/07/2015 13:43, Volker Simonis wrote: > Hi, > > could someone please review this tiny fix: > > http://cr.openjdk.java.net/~simonis/webrevs/2015/8130315/ > https://bugs.openjdk.java.net/browse/JDK-8130315 > > Because of this bug jdk9 doesn't build without > '--disable-warnings-as-errors' on SLES 12 with gcc 4.8.3 > (moving this to core-libs-dev). This looks right. -Alan.

3 3

RFR [9] 8130247: fix some new tidy warnings from jaxws and CORBA
by alexander stepanov 02 Jul '15

02 Jul '15

Hello, Could you please review the following fix http://cr.openjdk.java.net/~avstepan/8130247/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8130247 Just a minor correction for the docs. Some "empty trimming <p>" tidy warnings fixed; some deprecated "<tt>" tags were replaced with "{@code }" as well. Thanks, Alexander

2 2