December 2017 - core-libs-dev

[10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations
by Paul Sandoz 12 Jan '18

12 Jan '18

Hi, Please review the following webrev that makes flatMap non-aggressive when pushing elements downstream if any downstream operation is short-circuiting. http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8075939-flatMap-aggressive-pu… <http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8075939-flatMap-aggressive-pu…> This enables support for flat mapping to an infinite stream (assuming there is a downstream short-circuiting operation to terminate the stream computation). Thanks, Paul.

3 5

Android and Log4j
by Ralph Goers 10 Jan '18

10 Jan '18

Log4j added support for Java 9 by: Converting the Log4j-API jar to a multi-release jar that includes support for StackWalker and the new Process Id support. Adding a module-info.jar to the Log4j API jar. We are now getting complaints from Android users (as well as a few others) that their tools no longer work with log4j. During development I ran into problems with OSGi. The problems seem to mostly revolve around the fact that they can’t deal with the classes for Java 9. I was surprised that Android is failing on the classes in META-INF/versions/9 as I had assumed that would be an invalid location for a class file prior to Java 9, but that seems not to be the case. The fact that module-info.java turns into a class file also seems to be a problem since the various tools are seeing it and having problems with it. We have been discussing various ways to handle this in https://issues.apache.org/jira/browse/LOG4J2-2133 <https://issues.apache.org/jira/browse/LOG4J2-2133>. There seems to be a strong push to just remove the support for Java 9 since it is breaking so many things. It seems impossible to have a module-info.java file in a jar that is going to be included in Android. If this had been a json file that was interpreted by the classloader or had a different file extension other than .class we wouldn’t be in this mess. We also need another mechanism to bring in our code that uses StackWalker as calling it via Reflection and emulating lambda expressions seems like it would be painful and slow. Do you have any recommendations on how we can resolve this impasse? Ralph

6 12

RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method
by Brian Burkhalter 05 Jan '18

05 Jan '18

https://bugs.openjdk.java.net/browse/JDK-8194133 http://cr.openjdk.java.net/~bpb/8194133/webrev.00/ Add jdk.internal.io.IOSupport with copy() methods for InputStream-to-OutputStream copying and modify some classes to use these new methods. One thing that I noticed when looking at this is that in the fix for https://bugs.openjdk.java.net/browse/JDK-8193842, the Files.copy() method had a loop like while ((n = read(…)) > 0) whereas InputStream.transferTo() had while((n = read(…)) >= 0) which is to say that Files.copy() would terminate if there were an empty read() but transferTo() would not. The patch for 8193842 therefore possibly introduced a subtle behavioral change which no one noticed. Thanks, Brian

3 9

Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad
by Volker Simonis 03 Jan '18

03 Jan '18

Hi Martin, I've just pushed the fix to jdk8u/jdk8u-dev [1] so it should be in the next regular 8u update (probably 8u172 ?). You can of course test it any time by building the tip of http://hg.openjdk.java.net/jdk8u/jdk8u-dev yourself :) Merry Christmas and a happy new year, Volker [1] http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/def07b5ce3be On Wed, Dec 20, 2017 at 11:18 AM, Evans, Martin <Martin.Evans(a)kingfisher.com> wrote: > Many thanks guys, I look forward to testing. > > Regards, > > Martin > > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis@gmail.com] > Sent: 20 December 2017 09:33 > To: Baesken, Matthias <matthias.baesken(a)sap.com> > Cc: jdk8u-dev-request(a)openjdk.java.net; core-libs-dev(a)openjdk.java.net; ppc-aix-port-dev(a)openjdk.java.net; Simonis, Volker <volker.simonis(a)sap.com>; Evans, Martin <Martin.Evans(a)kingfisher.com>; build-dev <build-dev(a)openjdk.java.net> > Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad > > Hi Matthias, > > the change looks good! > I can sponsor it once we get the approval. > > Also forwarded to buil-dev for the minimal build change. > > Thank you and best regards, > Volker > > > On Wed, Dec 20, 2017 at 9:50 AM, Baesken, Matthias <matthias.baesken(a)sap.com> wrote: >> Hello , Mark reported this issue on AIX with OpenJDK8 : >> >>> >>>I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: >>> >>> java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D >>> getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 >>> <init> at org/logstash/instrument/monitors/ProcessMonitor.java:40 >>> detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 >>> generate at >>> org/logstash/instrument/reports/ProcessReport.java:15 >>> >>> this is the line in logstash: >>> >>> this.cpuProcessPercent = >>> scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); >>> >>> https://github.com/elastic/logstash/blob/master/logstash-core/src/mai >>> n/java/org/logstash/instrument/monitors/ProcessMonitor.java >>> >>> >>> Could anybody help steer me in the right direction? >>> >> >> This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . >> JDK8 is only affected , JDK9 and higher is not affected . >> >> Could I get a review for this change ? >> >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8193807 >> >> >> Thanks, Matthias > ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation.

2 3

Adding SocketChannel toString to connection exception messages
by Steven Schlansker 03 Jan '18

03 Jan '18

Hi core-libs-dev, While tracking down a connectivity issue, we identified that two of our hosts are unable to talk to each other due to a misconfiguration of the network. This manifested as: 2017-12-21T11:00:34.840Z DEBUG <> [default-pool-34] o.e.j.client.AbstractConnectionPool - Connection 1/32 creation failed java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:716) at org.eclipse.jetty.io.SelectorManager.doFinishConnect(SelectorManager.java:353) at org.eclipse.jetty.io.ManagedSelector.processConnect(ManagedSelector.java:157) ... The message, 'Connection refused', does tell you something -- but it doesn't emit really any information about what actually happened (i.e. host / port of refused connection) that would allow you to track down exactly where it failed. While the application can generally figure this out (it probably knows where it tried to connect) the SocketChannelImpl could helpfully attach this information to the message, rather than relying on the application to correctly figure out the information at every connect point. What if ConnectException included the attempted hostname / IP / port SocketAddress? java.net.ConnectException: Connection to 'foo.mycorp.com[10.x.x.x]:12345' refused Much more useful! This could also be extended to various other socket exceptions. Would such a contribution be accepted? Thanks for considering!

9 18

FileOutputStream.getFD() vs finalization
by Hans Boehm 03 Jan '18

03 Jan '18

The design of the getFD() calls in some Java *Stream classes seems problematic, and doesn't seem cleanly fixable without a spec change. I first noticed this in JDK 8 code, but Roger Riggs recent JDK 10 changes also don't seem to fully address this specific problem, particularly since the code paths remain more-or-less similar when a FOS subclass has a close method. It probably remains easier to describe this in a JDK 8/9 setting, so I'll do so initially. Close() on the *Stream closes the FD. The finalize() method on the Stream (not the FD!) calls close(). Assuming JLS 12.6.2 finalization semantics, if my program consists of: { FileOutputStream f1 = new FileOutputStream("foo"); FileDescriptor fd = f1.getFD(); A: FileOutputStream f2 = new FileOutputStream(fd); } f2 may be associated with a closed fd, since f1 could have been finalized at any pointer after its construction, in particular at point A. This is slightly aggravated by the fact that close() is implemented in a way that doesn't guarantee reachability of the FileOutputStream while, or before, it runs. Close() could easily be inlined, and the *Stream fields promoted to registers. Thus even if you always call close() explicitly, I think you are still affected by this problem. Fundamentally the result of getFD() is only valid as long as the underlying *Stream is reachable, and that is not defined in a way that's understandable by most programmers. That also makes this API un-Java-like, in that the user has to worry about very subtle object lifetime rules. The only semi-plausible solution seems to involve judicious use of ReachabilityFence() on FileOutputStreams after the getFD() result is no longer needed. I personally don't think that's really a very plausible request of the programmer. I think Roger's JDK 10 changes may help in the case in which there is not an overridden close() method, since the cleanup action is registered on the underlying FileDescriptor. But if a subclass overrides close(), I think we're basically still in the same boat. For the reasons he already gave, that seems hard to avoid. And again I think the problem applies even if close() is explicitly called, since that doesn't really prevent finalization of the Stream object. Has this been discussed anywhere? Opinions about how to deal with it? (This arose while going through the core libraries, looking for premature finalization issues. So far, this one seems special, in that It really seems to require an API change.) Hans

2 2

RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass
by Chris Hegarty 02 Jan '18

02 Jan '18

In JDK 9 sun.reflect.Reflection.getCallerClass, and its jdk.internal counterpart, where both deprecated-for-removal ( to give prior warning of the unsupported private API's intended removal ). The supported replacement, since Java SE 9, is java.lang.StackWalker. http://cr.openjdk.java.net/~chegar/8179424/webrev/ -Chris.

7 20

RFR 8193832: Performance of InputStream.readAllBytes() could be improved
by Brian Burkhalter 22 Dec '17

22 Dec '17

https://bugs.openjdk.java.net/browse/JDK-8193832 http://cr.openjdk.java.net/~bpb/8193832/webrev.00/ The implementation of InputStream.readAllBytes() can be modified to be in general faster and to require less memory. The current implementation starts with an internal buffer of size 8192 bytes and doubles the size of the buffer each time more capacity is required resulting in a geometric progression in the buffer size. At the end if the buffer size is not exactly equal to the total number of bytes read, then another allocation equal to the total number read is performed. The amount of memory N required can be calculated for initial buffer size B and total bytes read L as M for L == B*(2^n) N = M + L for L != B*(2^n) where M = B*(2^(n + 1) - 1) and n = ceil(log2(L/B)). An alternative implementation is not to increase the size of the internal buffer each time more capacity is needed, but to instead maintain a list of buffers of up to B bytes each and gather those into an output buffer at the end. If there is only a single buffer in the list then gathering is not needed and it can be returned directly. The amount of memory N required in this case is B + L for L <= B N = B + 2*L for L > B A comparison of memory usage for a number of lengths L with a buffer size B of 8192 is: L N_old N_new 8191 16383 16383 8192 8192 16384 8193 32769 24578 16383 40959 40958 16384 24576 40960 16385 73729 40962 32767 90111 73726 32768 57344 73728 32769 155649 73730 65535 188415 139262 65536 122880 139264 65537 319489 139266 131071 385023 270334 131072 253952 270336 131073 647169 270338 262143 778239 532478 262144 516096 532480 262145 1302529 532482 In general if the size of the last intermediate buffer in the old version does not equal the total number of bytes read, then the old version will require more memory thaw the new. The performance for the same set of lengths was measured using JMH: Benchmark (base) (offset) Mode Cnt Score Error Units BenchReadAllBytesParam.readAllBytes 8192 -1 thrpt 5 578064.253 ± 18955.667 ops/s BenchReadAllBytesParam.readAllBytesArrayList 8192 -1 thrpt 5 530963.634 ± 4799.923 ops/s BenchReadAllBytesParam.readAllBytes 8192 0 thrpt 5 1414243.593 ± 68520.245 ops/s BenchReadAllBytesParam.readAllBytesArrayList 8192 0 thrpt 5 532019.149 ± 7051.738 ops/s BenchReadAllBytesParam.readAllBytes 8192 1 thrpt 5 293586.365 ± 8196.939 ops/s BenchReadAllBytesParam.readAllBytesArrayList 8192 1 thrpt 5 361682.265 ± 27081.111 ops/s BenchReadAllBytesParam.readAllBytes 16384 -1 thrpt 5 216809.517 ± 4853.852 ops/s BenchReadAllBytesParam.readAllBytesArrayList 16384 -1 thrpt 5 212346.244 ± 2980.916 ops/s BenchReadAllBytesParam.readAllBytes 16384 0 thrpt 5 379757.470 ± 14524.249 ops/s BenchReadAllBytesParam.readAllBytesArrayList 16384 0 thrpt 5 218802.256 ± 4361.080 ops/s BenchReadAllBytesParam.readAllBytes 16384 1 thrpt 5 123570.712 ± 1665.661 ops/s BenchReadAllBytesParam.readAllBytesArrayList 16384 1 thrpt 5 208801.577 ± 8005.196 ops/s BenchReadAllBytesParam.readAllBytes 32768 -1 thrpt 5 93083.146 ± 1024.309 ops/s BenchReadAllBytesParam.readAllBytesArrayList 32768 -1 thrpt 5 104398.304 ± 1310.509 ops/s BenchReadAllBytesParam.readAllBytes 32768 0 thrpt 5 151104.878 ± 3461.359 ops/s BenchReadAllBytesParam.readAllBytesArrayList 32768 0 thrpt 5 104849.088 ± 3841.224 ops/s BenchReadAllBytesParam.readAllBytes 32768 1 thrpt 5 58039.827 ± 398.908 ops/s BenchReadAllBytesParam.readAllBytesArrayList 32768 1 thrpt 5 104489.685 ± 2118.496 ops/s BenchReadAllBytesParam.readAllBytes 65536 -1 thrpt 5 43144.695 ± 440.349 ops/s BenchReadAllBytesParam.readAllBytesArrayList 65536 -1 thrpt 5 55583.338 ± 2115.162 ops/s BenchReadAllBytesParam.readAllBytes 65536 0 thrpt 5 67216.536 ± 2055.057 ops/s BenchReadAllBytesParam.readAllBytesArrayList 65536 0 thrpt 5 55680.238 ± 1235.571 ops/s BenchReadAllBytesParam.readAllBytes 65536 1 thrpt 5 27908.000 ± 568.473 ops/s BenchReadAllBytesParam.readAllBytesArrayList 65536 1 thrpt 5 55938.779 ± 813.991 ops/s BenchReadAllBytesParam.readAllBytes 131072 -1 thrpt 5 20299.014 ± 568.616 ops/s BenchReadAllBytesParam.readAllBytesArrayList 131072 -1 thrpt 5 28115.036 ± 842.392 ops/s BenchReadAllBytesParam.readAllBytes 131072 0 thrpt 5 31617.475 ± 467.378 ops/s BenchReadAllBytesParam.readAllBytesArrayList 131072 0 thrpt 5 28274.289 ± 259.699 ops/s BenchReadAllBytesParam.readAllBytes 131072 1 thrpt 5 13640.406 ± 303.125 ops/s BenchReadAllBytesParam.readAllBytesArrayList 131072 1 thrpt 5 28221.298 ± 515.030 ops/s BenchReadAllBytesParam.readAllBytes 262144 -1 thrpt 5 9995.183 ± 249.368 ops/s BenchReadAllBytesParam.readAllBytesArrayList 262144 -1 thrpt 5 14043.194 ± 138.026 ops/s BenchReadAllBytesParam.readAllBytes 262144 0 thrpt 5 15309.238 ± 353.752 ops/s BenchReadAllBytesParam.readAllBytesArrayList 262144 0 thrpt 5 14048.569 ± 348.699 ops/s BenchReadAllBytesParam.readAllBytes 262144 1 thrpt 5 5940.442 ± 206.855 ops/s BenchReadAllBytesParam.readAllBytesArrayList 262144 1 thrpt 5 14055.357 ± 412.470 ops/s In each case the number of bytes read is the sum of the base and offset parameters. Similar behavior with respect to data length is observed for performance as for memory usage: if the data length is not equal to the size of the last intermediate buffer used, then the performance of the old version is in general worse than that of the new. Thanks, Brian

7 31

[RFR] 8160768: Add capability to custom resolve host/domain names within the default JDNI LDAP provider
by Rob McKenna 22 Dec '17

22 Dec '17

As this enhancement is limited to JDK10 this frees us up to explore a different approach. http://cr.openjdk.java.net/~robm/8160768/webrev.06/ In this webrev we're using the Service Provider Interface to load an implementation of the new LdapDnsProvider class. Implementations of this class are responsible for resolving DNS endpoints for a given ldap url should the default implementation be insufficient in a particular environment. Note: if a security manager is installed the ldapDnsProvider RuntimePermission must be granted. -Rob

4 5

RFR (JDK10/JAXP Doc-only) 8184431: References to @sun.com
by Joe Wang 22 Dec '17

22 Dec '17

Hi, This change uses a process to strip off the @sun.com email addresses and links, and keep just the name of the authors. Please review: JBS: https://bugs.openjdk.java.net/browse/JDK-8184431 webrev: http://cr.openjdk.java.net/~joehw/jdk10/8184431/webrev/index.html Thanks, Joe

3 4