June 2015 - core-libs-dev

RFR 8073056: Repeating annotations throws java.security.AccessControlException with a SecurityManager
by Joel Borggrén-Franck 24 Aug '15

24 Aug '15

Hi, Here is a fix for an issue with repeating annotations when a security manager is set. Fix is to use the Proxy invocation handler to unwrap the array containing the repeating annotations. In theory it might be possible to have instances of Annotations that are not implemented using Proxies, so the old code which is independent of implementation is kept as a fall-back, but the user or these theoretical annotations will have to configure the security policy accordingly. http://cr.openjdk.java.net/~jfranck/8073056/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8073056 cheers /Joel

5 25

SelectableChannels and Process API
by Peter Levart 28 Jul '15

28 Jul '15

Hi Roger, So I started new thread... On 04/14/2015 11:33 PM, Roger Riggs wrote: > > On 4/14/2015 11:47 AM, Peter Levart wrote: >> I have been thinking of another small Process API update. Some people >> find it odd how redirected in/out/err streams are exposed: >> >> http://blog.headius.com/2013/06/the-pain-of-broken-subprocess.html > yep, I've read that several times. To be fair, it's mostly, but not entirely correct. The part that says: " So when the child process exits, the any data waiting to be read from its output stream is drained into a buffer. All of it. In memory. Did you launch a process that writes a gigabyte of data to its output stream and then terminates? Well, friend, I sure hope you have a gigabyte of memory, because the JDK is going to read that sucker in and there's nothing you can do about it. And let's hope there's not more than 2GB of data, since this code basically just grows a byte[], which in Java can only grow to 2GB. If there's more than 2GB of data on that stream, this logic errors out and the data is lost forever." ...is exaggeration. This does not happen as the pipe has a bounded buffer. When subprocess exits, there is at most that much data left in the buffer (64k typically) and only that much is sucked into the Java process and the underlying handle closed. >> >> They basically don't like: >> >> - that exposed Input/Output streams are buffered >> - that underlying streams are File(Input/Output)Streams which, >> although the backing OS implementation are not files but pipes, don't >> expose selectable channels so that non-blocking event-based IO could >> be performed on them. >> - that exposed IO streams are automatically "managed" in UNIX >> variants of ProcessImpl which needs subtle "hacks" to do it in a >> perceptively transparent way (delayed close, draining input on exit >> and making it available after the underlying handle is already >> closed, ...) >> >> So I've been playing with the idea of exposing the "real" pipe >> channels in last couple of days. Here's the prototype I came up with: >> >> http://cr.openjdk.java.net/~plevart/jdk9-sandbox/JDK-8046092-branch/Process… >> >> >> This adds new Redirect type to the API and 3 new methods to Process >> that return Pipe channels when this new Redirect type is used. It's >> interesting that no native code changes were necessary. The behavior >> of pipes on Windows is a little different (perhaps because the Pipe >> NIO API uses sockets under the hood on Windows - why is that? Windows >> does have a pipe equivalent). What bothers me is that file handles >> opened on files (when redirecting to/from File) can be closed as soon >> as the subprocess is started and the subprocess is still able to >> read/write from the files (like with UNIX). It's not the same with >> pipe (i.e. socket) handles on Windows. They must be closed only after >> subprocess exits. >> >> If this subtle difference between file handles and socket handles on >> Windows could be dealt with (perhaps some options exist that affect >> subprocess spawning), then the extra waiting thread would not be >> needed on Windows. >> >> So what do you think of this API update? > Definitely worthy of a separate thread. It looks promising and > addresses some of the issues > raised, while moving other problems from the implementation to the > application. > Such as closing of the channels and cleanup. I worry about how the > resources are freed > if the code spawning the app doesn't do the cleanup. Will it require > hooks (like a finalizer) > to do the cleanup? > Also, it doesn't help with Martin's goal of being able to implement > emacs in Java since it doesn't provide pty control. > As you are aware the complexity in Process is to ensure a timely > cleanup and > allowing the Process to terminate and release the process resources > when it was done and not having to wait for the stdout/stderr consumer. I wonder how this automatic stream cleanup really helps in real-world programs. It doesn't help the Process to terminate and release the process resources any sooner as the process terminates on it's own (unless killed) and OS releases it's resources without the outside help anyway. Draining and closing the stream after the process has already exited just releases one file handle (the consuming side of the pipe) in a promptly manner. This could be left to the user and/or finalizer. Draining after the process has already exited does not help the process to exit any sooner as it happens after the fact. A program that doesn't consume the stream can cause the process to hang forever as the pipe's buffer is bounded (64k typically). So draining and closing after the process has exited only potentially helps for the last 64k of the stream and only to release one file handle in a potentially more timely manner. OTOH now that ProcessImpl for UNIX does that (and why does Windows implementation not do that?) sloppy programs might exist that would potentially break if the status quo is not maintained. But new functionality need not be so permissive. I'll take a look at how and if Channel(s) do any kind of automatic cleanup based on reachability and whether this can be bolted on for Process use. I doubt it is possible to drain and close a Channel without disturbing the ongoing Selector IO processing... Regards, Peter > > Thanks, Roger > >

3 6

RFR: 6260652: (coll) Arrays.asList(x).toArray().getClass() should be Object[].class
by Martin Buchholz 09 Jul '15

09 Jul '15

10 years later ... still asking for approval to commit. https://bugs.openjdk.java.net/browse/JDK-6260652 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Arrays.asList.toArray/

4 10

RFR 9: 8078099 : (process) ProcessHandle should uniquely identify processes
by Roger Riggs 07 Jul '15

07 Jul '15

Please review changes to ProcessHandle implementation to uniquely identify processes based on the start time provided by the OS. It addresses the issue of PID reuse. This is the implementation of the ProcessHandle.equals() spec change in 8129344 : (process) ProcessHandle instances should define equals and be value-based Webrev: http://cr.openjdk.java.net/~rriggs/webrev-starttime-8078099/ Issue: https://bugs.openjdk.java.net/browse/JDK-8078099 Thanks, Roger

3 6

RFR JDK-8075526: Need a way to read and write time in UTC
by Xueming Shen 06 Jul '15

06 Jul '15

Hi, Please help review and comment on this rfe. Issue: https://bugs.openjdk.java.net/browse/JDK-8075526 webrev: http://cr.openjdk.java.net/~sherman/8075526 Background info: The title of the RFE is a little "mis-leading". All the existing set/get date-time methods actually work with "UTC" time. Both the old pair public void setTime(long time); public long getTime(); and the newly introduced pair pubic ZipEntry set/getLastModifiedTime(FileTime time); public FileTime getLastModifiedTime(); are to set/get the last modified time in UTC time. However the ZIP specification clearly specifies that the "normal" date and time fields in the zip file entry (loc and cen) are defined as the date/time in dos format, which unfortunately is a "local" date-time. Therefor timezone conversion must be applied before/after the utc time can be set into/got from those fields (the UTC timestamps set/get by the new pair are therefor being set into/got from the "extended timestamp fields" of the optional extra data of each entry, those fields are specified as unix/utc timestamp) We did not have an "local-date-time" abstract before the java.time.LocalDateTime was introduced in jdk8, the epoc date/time is the only date/time abstract in java vm. The proposed change here is to add yet another pair of set/get modified time methods public void setTimeLocale(LocalDateTime time); public LocalDateTime getTimeLocal(); to use the java.time.LocalDateTime type to set/get the modified time into zip entry's dos timestamp fields directly WITHOUT involving the timezone conversion (implied, with default TimeZone). Other than solving the pack/unpack problem raised in this RFE, it should also help improve the performance when local-date-time is actually desired when interfacing with the ZipEntry by eliminating the un-necessary/implied timezone conversion. For example, in our jar tool, currently we are "printing" the timestamp for zip entry "e" as new Date(e.getTime()).toString(); in which we are converting the local-date-time (ms-dos-formatted in zip entry) to utc time by using the default timezone (in ZipEntry), and then converting the utc time (in Date) back to the printable "local date time" again. It might be desired to format the "local-date-time" directly without involving the timezone conversion now via the proposed method java.time.format.DateTimeFormatter.ofPattern("EEE MMM dd HH:mm:ss zzz yyyy") .withZone(java.time.ZoneId.systemDefault()); .format(e.getTimeLocal() In above example, we still use the "system default timezone", however, it is used purely to output the zone name for the "zzz" (which the Date.toString() does), not for conversion. if the "zzz" is not required/needed, it can just be java.time.format.DateTimeFormatter.ofPattern("EEE MMM dd HH:mm:ss yyyy").format(e.getTimeLocal()); Comment/Opinion please. If we agree the approach/webrev, I will submit the CCC before integration. Thanks, -Sherman

3 8

Bad interaction between wildcard and functional interface conversion
by Remi Forax 05 Jul '15

05 Jul '15

Hi all, The way the conversion between a lambda (or a method reference) and a functional interface is specified doesn't take wildcard (exactly ? super) into account making the concept of contravariance of functional interface less intuitive that it should be. The following code compiles: private static void create(Consumer<Consumer<String>> consumer) { consumer.accept(s -> System.out.println(s)); } This one doesn't compile because "? super Consumer<? super String>" is not a functional interface: private static void create2(Consumer<? super Consumer<? super String>> consumer) { consumer.accept(s -> System.out.println(s)); } The workaround is to introduce a cast :( private static void create3(Consumer<? super Consumer<? super String>> consumer) { consumer.accept((Consumer<String>)s -> System.out.println(s)); } which is stupid in this case because there is no ambiguity. This cast is just here because the JLS doesn't consider that ? super Consumer<...> is a valid target type IMO, this bug is very similar to JDK-6964923 and i think the spec should be changed to allow ? super Foo to be a valid target type for a lambda conversion (obviously if Foo is a functional interface). regards, Rémi

2 3

JEP 132: More-prompt finalization
by Peter Levart 03 Jul '15

03 Jul '15

Hi, Did you know that the following simple loop: public class FinalizableBottleneck { static boolean no; @Override protected void finalize() throws Throwable { // empty finalize() method does not make the object finalizable // (it is not even registered on the finalizer's list) if (no) { throw new AssertionError(); } } public static void main(String[] args) { while (true) { new FinalizableBottleneck(); } } } ...quickly fills the entire heap with FinalizableBottleneck and internal Finalizer objects and brings the JVM to a halt? After a few seconds of running the above program, jmap -histo:live reports: num #instances #bytes class name ---------------------------------------------- 1: 50048325 2001933000 java.lang.ref.Finalizer 2: 50048278 800772448 FinalizableBottleneck There are a couple of bottlenecks that make this happen: - ReferenceHandler thread synchronizes with VM to unhook Reference(s) from the pending chain one be one and dispatches them to their respected ReferenceQueue(s) which also use synchronization for equeueing each Reference. - Enqueueing synchronizes with the finalization thread which removes the Finalizer(s) (FinalReferences) from the finalization queue and executes them. - Executing the Finalizer(s) removes them from the doubly-linked list of all Finalizer(s) which is used to retain them until they are needed and this synchronizes with the threads that link new Finalizer(s) into the doubly-linked list as new finalizable objects get registered. We see that the creation of a finalizable object only takes one synchronization (registering into the doubly-linked list) and is performed synchronously, while finalization takes 4 synchronizations among 4 different threads (in pairs) and happens when the Finalizer instance "travels" over from VM thread to ReferenceHandler thread and then to finalization thread. No wonder that finalization can not keep up with allocation in a single thread. The situation is even worse when finalize() methods do some actual work. I have experimented with various approaches to widen these bottlenecks and found out that I can not beat the ForkJoinPool when combined with some improvements to internal data structures used in reference processing. Here's a prototype I came up with: http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/webrev.01/ And this is the benchmark I use for measuring the throughput: http://cr.openjdk.java.net/~plevart/misc/JEP132/ReferenceHandling/Finalizer… The benchmark shows (results inline in source) that using unpatched JDK, on my PC (i7-2700K, Linux, JDK8) I can not construct more than 1500 finalizable objects per ms in a single thread and that while doing so, finalization only manages to process approx. 100 - 120 objects at the same time. Objects "in-flight" quickly accumulate and bring the VM to a halt, where it is not doing anything but full GC cycles. When constructing in 4 threads, there's not much difference. Construction of finalizable objects simply doesn't scale. Patched JDK shows something completely different. Single thread construction achieves a rate of 3600 objects / ms. Number of "in-flight" objects is kept constant at about 5-6M instances which amounts to approx 1.5 s of allocation. I think this is about the rate of GC cycles during which VM also processes the references. The benchmark also shows the ForkJoinPool statistics which shows that the number of queued tasks is also kept low. Increasing the allocation threads to 4 increases allocation rate to about 4300 objects / ms and finalization keeps up. Increasing allocation threads to 8, further increases allocation rate to about 4600 objects / ms and finalization still keeps up. The increase in rate is not linear, but keep in mind that i7 is a 4-core CPU. About the implementation... 1st improvement I did was for the doubly-linked list of Finalizer instances that is used to keep them alive until they are needed. I ripped-off the wonderful ConcurrentLinkedDeque by Doug Lea and Martin Buchholz and just kept the internal link/unlink methods while specializing them to Finalizer entries (very straight-forward). I experimented with throughput and got some improvement, but throughput has increased much more when I used several instances of independent lists and distributed registrations among them randomly (unlinking consequently is also distributed randomly). I found out that no matter how hard I try to optimize ReferenceQueue while keeping the API unchanged, I can only do so much and that was not enough. I have been surprised by how well ForkJoinPool distributes tasks among threads, so I concluded that leveraging it is the best choice. I re-designed the pending-list unhooking loop to unhook pending references in chunks which greatly improves the throughput. Since unhooking can be performed by a single thread while holding a lock which is mandated by interface between VM and Java, I didn't employ multiple threads, but a single eternal ForkJoinTask that unhooks in chunks and forks-off other processing tasks that process chunks. When there are just a couple of References pending at one time and a not-full chunk is unhooked, then the processing is performed by the same thread that unhooked the refrences, but when there are more, worker tasks are forked off and the unhooking thread continues with full peace. This processing includes execution of Cleaners, forking the finalizer tasks and enqueue-ing other references. Finalizer(s) are always executed as separate ForkJoinTask(s). It's interesting how Runtime.runFinalizers() is implemented in this patch - it basically amounts to ForkJoinPool.awaitQuiescence() ... I also tweaked the ReferenceQueue implementation a bit (it is still used for other kinds of references) so that it avoids synchronization with a monitor lock when there are no blocking waiters and uses CAS to enqueue/dequeue. This improves throughput when the queue is not empty. Since in the prototype multiple threads can enqueue into the same queue, I thought this would improve throughput in such situations. Comments, suggestions, criticism are welcome. Regards, Peter

7 25

[9] RFR(M): 8076112: Add @HotSpotIntrinsicCandidate annotation to indicate methods for which Java Runtime has intrinsics
by Zoltán Majó 02 Jul '15

02 Jul '15

Hi, please review the patch for JDK-8076112. Bug: https://bugs.openjdk.java.net/browse/JDK-8076112 Problem: There is need to indicate Java methods that are potentially intrinsified by JVM. Solution: Mark intrinsified methods with the jdk.internal.HotSpotIntrinsicCandidate annotation. Add checks that are omitted by VM-level intrinsics to the library code. Add a new diagnostic flag, CheckIntrinsics. If CheckIntrinsics is enabled, the VM performs the following checks when a class C is loaded: - all intrinsics defined by the VM for class C are present in the loaded class file and are marked; - an intrinsic is defined by the VM for all marked methods of C. If a mismatch is detected, the following is done: - a fastdebug VM prints a warning and then exits; - a product VM prints a warning and unmarked are not intrinsified. Webrev: - top: http://cr.openjdk.java.net/~zmajo/8076112/top/webrev.05/ - jdk: http://cr.openjdk.java.net/~zmajo/8076112/jdk/webrev.05/ - hotspot: http://cr.openjdk.java.net/~zmajo/8076112/hotspot/webrev.05/ Testing: - JPRT run with the 'hotspot' testset, all tests pass; - all JTREG hotspot tests, all tests pass that pass with a VM version that does not include the patch; all tests were run with -XX:+CheckIntrinsics. Thank you and best regards, Zoltan

7 20

RFR: JDK-8080679: Include jline in JDK for Java and JavaScript REPLs
by Jan Lahoda 02 Jul '15

02 Jul '15

Hello, I am proposing to add JLine 2.12.1 into the jdk repository for use by the Java and Nashorn REPLs. Full patch is available here: http://cr.openjdk.java.net/~jlahoda/8080679/webrev.00/full/ To aid the review, I've split this patch into to smaller patches: -a patch that only adds unmodified jline sources at appropriate places in the jdk repository: http://cr.openjdk.java.net/~jlahoda/8080679/webrev.00/clean-jline/ -a patch that shows the additional changes I've done: http://cr.openjdk.java.net/~jlahoda/8080679/webrev.00/additional/ This split is intended solely to simplify reviewing, my plan is to integrate this as a single patch. The main additional changes are: -plugging the new module, jdk.jline, into the JDK build. Currently, the JLine packages are exported only to jdk.scripting.nashorn (the plan is to also export them to the future jdk.jshell module). (The patch is not adding the dependency from jdk.scripting.nashorn to jdk.jline, though - I expect that to be added when needed.) -the sources are re-packaged from package "jline" to "jdk.internal.jline" -removing trailing whitespace, adding newlines at the end of the files, encoding characters that are not ASCII -avoiding the dependency on another library, jansi, by reimplementing two elements that were used from the other library. These are mainly the changes in WindowsTerminal and ConsoleReader.java. This also includes the WindowsTerminal.cpp native library. The native part is heavily inspired by: http://cr.openjdk.java.net/~sherman/rl/src/java.base/windows/native/libjava… As I am not experienced in native programming, comments to the native part would be particularly useful. -changes to resolve javac warnings in JLine. -tests for some of the added functionality. Any comments are welcome! Thanks, Jan

7 28

RFR [S] 8077242: (str) Optimize AbstractStringBuilder.append(CharSequence, int, int) for String argument
by Ivan Gerasimov 01 Jul '15

01 Jul '15

Hello! AbstractStringBuilder, which is base for StringBuilder and StringBuffer has a method for appending a sub-sequence of a CharSequence. Internally, it copies one char at a time in a loop, picking them up with CharSequence.charAt() method. For the case when the argument is a String, it can be done more efficiently, by a call to String.getChars(). Since String is a final class, it should be safe to pass 'value' to its method. I've also found a few places in JDK where code can be done more efficient, using this optimization. Would you please help review this fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8077242 WEBREV: http://cr.openjdk.java.net/~igerasim/8077242/02/webrev/ Sincerely yours, Ivan

2 2