March 2016 - core-libs-dev

(Round 2) RFR(s): 8150460: (linux|bsd|aix)_close.c: file descriptor table may become large or may not work at all
by Thomas Stüfe 14 Apr '16

14 Apr '16

Hi all, https://bugs.openjdk.java.net/browse/JDK-8150460 thanks to all who took the time to review the first version of this fix! This is the new version: http://cr.openjdk.java.net/~stuefe/webrevs/8150460-linux_close-fdTable/webr… I reworked the fix, trying to add in all the input I got: This fix uses a simple one-dimensional array, preallocated at startup, for low-value file descriptors. Like the code did before. Only for large values of file descriptors it switches to an overflow table, organized as two dimensional sparse array of fixed sized slabs, which are allocated on demand. Only the overflow table is protected by a lock. For 99% of all cases we will be using the plain simple fdTable structure as before. Only for unusually large file descriptor values we will be using this overflow table. Memory footprint is kept low: for small values of RLIMIT_NOFILE, we will only allocate as much space as we need. Only if file descriptor values get large, memory is allocated in the overflow table. Note that I avoided the proposed double-checked locking solution: I find it too risky in this place and also unnecessary. When calling getFdEntry(), we will be executing a blocking IO operation afterwards, flanked by two mutex locks (in startOp and endOp). So, I do not think the third mutex lock in getFdEntry will add much, especially since it is only used in case of larger file descriptor values. I also added the fix to bsd_close.c and aix_close.c. I do not like this code triplication. I briefly played around with unifying this code, but this is more difficult than it seems: implementations subtly differ between the three platforms, and solaris implementation is completely different. It may be a worthwhile cleanup, but that would be a separate issue. I did some artificial tests to check how the code does with many and large file descriptor values, all seemed to work well. I also ran java/net jtreg tests on Linux and AIX. Kind Regards, Thomas

3 9

JDK 9 Build 111 seems to miss some locale data, Lucene tests fail with Farsi and Thai language
by Uwe Schindler 11 Apr '16

11 Apr '16

Hi, after also testing the separate "Jigsaw" build on jdk9.java.net I see the same problems. So both builds 111 are wrong. To me it looks like the Unicode data files are missing some information - which could again be a packaging bug. As said before, build 110 does not have this problem, so it seems to be a side-effect of Jigsaw merging. The following stuff does not work: (1) Thai's locale does not have working dictionary-based BreakIterator available. The following "check" in Lucene for this fails, because it cannot detect a boundary correctly: /** * True if the JRE supports a working dictionary-based breakiterator for Thai. * If this is false, this tokenizer will not work at all! */ public static final boolean DBBI_AVAILABLE; private static final BreakIterator proto = BreakIterator.getWordInstance(new Locale("th")); static { // check that we have a working dictionary-based break iterator for thai proto.setText("ภาษาไทย"); DBBI_AVAILABLE = proto.isBoundary(4); } After this static initializer, DBBI_AVAILABLE is false. This makes some tests to be ignored, but 2 fail because of this (which might be an oversight on our side). But nevertheless, this is a bug in build 111. (2) The collator for Arabic (Farsi) language fails to work correctly. This also looks like missing data. Collator collator = Collator.getInstance(new Locale("ar")); This one fails some tests that have worked before (in any previous Java 9 build and any previous Java version 1.4+). I have to revert to build 110, because we cannot make our tests pass on the Jenkins server. I hope this can be fixed with the next build! Uwe ----- Uwe Schindler uschindler(a)apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces@openjdk.java.net] On > Behalf Of Uwe Schindler > Sent: Saturday, March 26, 2016 11:49 AM > To: 'Alan Bateman' <Alan.Bateman(a)oracle.com> > Cc: 'Rory O'Donnell' <rory.odonnell(a)oracle.com>; 'Core-Libs-Dev' <core-libs- > dev(a)openjdk.java.net> > Subject: RE: JDK 9 Build 111 still has ManagementFactory problems, but only > on Linux - not on Windows > > Hi Alan, > > It is ok from the management factory perspective - sorry for trouble. But the > first test showed that during language analysis something seems missing, > which did not happen with the b110 one. > > Currently we see the following tests fail: Everything Thai and Farsi language > related. > > It looks like the BreakIterators (at least the Thai one) are missing. I have to > dig into this. I will also compare with the Jigsaw-build 111, if this is different > (what is the difference between both, I only see size is largely different and > Windows has a ZIP file?). Those test failures did not happen with neither > build 110 of Jigsaw and without. > > Here are 2 failures: > http://jenkins.thetaphi.de/job/Lucene-Solr-6.x-Linux/253/consoleFull > http://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/16339/consoleFull > > Uwe > > ----- > Uwe Schindler > uschindler(a)apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > > > > -----Original Message----- > > From: Alan Bateman [mailto:Alan.Bateman@oracle.com] > > Sent: Saturday, March 26, 2016 11:27 AM > > To: Uwe Schindler <uschindler(a)apache.org> > > Cc: 'Rory O'Donnell' <rory.odonnell(a)oracle.com>; 'Core-Libs-Dev' <core- > libs- > > dev(a)openjdk.java.net> > > Subject: Re: JDK 9 Build 111 still has ManagementFactory problems, but > only > > on Linux - not on Windows > > > > > > > > On 26/03/2016 09:59, Uwe Schindler wrote: > > > Sorry, > > > > > > false alarm! I missed to change the build number when changing EA build > > paths on the Jenkins server. There was still "b110" in it. :( > > > Sorry for noise, all fine. > > > > > > > > No problem and thanks for confirming that jdk-9+111 is okay. > > > > -Alan

3 8

RFR: 8150469: unpack200 fails to compare crc correctly.
by Kumar Srinivasan 06 Apr '16

06 Apr '16

Hello John, Alex, Please review fix for https://bugs.openjdk.java.net/browse/JDK-8150469 The previous fix keeps the storage in the unpacker object which is reset, for a multi-segmented pack archive, thus the computed crc32 is lost. Now the storage is moved into the gzip container, which persists, across multiple segments, I also added the length check as specified by RFC 1952, as implemented by JDK. http://hg.openjdk.java.net/jdk9/dev/jdk/file/e4af8119eba4/src/java.base/sha… The review is at: http://cr.openjdk.java.net/~ksrini/8150469/webrev.00/ Thanks Kumar

3 2

RFR:JDK-8148849:Truncating Duration
by nadeesh tv 06 Apr '16

06 Apr '16

Hi all, Bug Id : https://bugs.openjdk.java.net/browse/JDK-8148849 Enhanced Duration by adding public Duration truncatedTo(TemporalUnit unit) Please http://cr.openjdk.java.net/~ntv/8148849/webrev.00/ -- Thanks and Regards, Nadeesh TV

3 5

RFR 8150829: Enhanced drop-args, identity and default constant, varargs adjustment
by shilpi.rastogi＠oracle.com 06 Apr '16

06 Apr '16

Hi All, Please review the following- https://bugs.openjdk.java.net/browse/JDK-8150829 http://cr.openjdk.java.net/~srastogi/8150829/webrev.04 Thanks, Shilpi

2 2

Bug id : 8140747 - Data corruption when parsing XML using StAX/Xerces
by Vlad-Lucian Gînju 05 Apr '16

05 Apr '16

Hello, Encountered this bug on our server and after some investigations I was able to find the bug reported here https://bugs.openjdk.java.net/browse/JDK-8140747. It seems that you were not able to reproduce on the Java versions mentioned in the bug's description. I am able to reproduce it on JDK 7u75 constantly. Installed this Java version on several linux machines and I am able to reproduce it on every one of them using the code provided in bug's description.I hope this info can help you find the problem mentioned in the bug's description. [vlg@sec vladtmp]$ for i in $(seq 1 10); do java TestXmlReader; done; bugs bugs bugs bugs bugs bugs bugs bugs bugs bugs [vlg@sec vladtmp]$ java -version java version "1.7.0_75" Java(TM) SE Runtime Environment (build 1.7.0_75-b13) Java HotSpot(TM) 64-Bit Server VM (build 24.75-b04, mixed mode) Regards,Vlad-Lucian Gînju

2 1

toRealPath throws InternalError: Should not get here on Windows (paths with junctions)
by Dawid Weiss 05 Apr '16

05 Apr '16

I couldn't find an appropriate bug in bugzilla, but this fails reliably for me with Java 8 and 9 on Windows 10, 64-bit: mkdir foo mklink /J bar foo cd bar java -cp Test . where Test.java is as simple as: import java.io.*; import java.nio.file.*; public class Test { public static void main(String[] args) throws Exception { System.out.println("p.toRealPath(): " + Paths.get(args[0]).toRealPath()); } } The thrown error shows: Exception in thread "main" java.lang.InternalError: Should not get here at sun.nio.fs.WindowsNativeDispatcher.GetFinalPathNameByHandle(Native Method) at sun.nio.fs.WindowsLinkSupport.getFinalPath(Unknown Source) at sun.nio.fs.WindowsLinkSupport.getRealPath(Unknown Source) at sun.nio.fs.WindowsPath.toRealPath(Unknown Source) at sun.nio.fs.WindowsPath.toRealPath(Unknown Source) Is this just me? :) Dawid

3 7

RFR [9] 8153181: Examine sun.misc.VMSupport
by Chris Hegarty 04 Apr '16

04 Apr '16

As part of JEP 260, all non-Critical APIs in sun.misc are being examined. sun.misc.VMSupport is a utility class supporting two functions: 1) the initialization of management Agent properties, and 2) the retrieval of the VM temporary directory used by the attach and perf data files. The initialization of Agent properties is used statically by the java.managment module, or can be provoked by a serviceability tool on a VM that has had an agent loaded in it. The location of VM temporary files is a low-level interface between the libraries and VM. For these reasons, VMSupport should remain in the base module, but be moved to an internal package. http://cr.openjdk.java.net/~chegar/8153181/00/ https://bugs.openjdk.java.net/browse/JDK-8153181 -Chris.

3 6

JDK 9 proposal: allocating ByteBuffers on heterogeneous memory
by Dohrmann, Steve 01 Apr '16

01 Apr '16

This is a JDK 9 proposal to support allocation of direct java.nio.ByteBuffer instances backed by memory other than off-heap RAM. The current allocateDirect() static method in java.nio.ByteBuffer is being used by some applications to meet special memory needs -- in particular, allocating large amounts of memory without contributing significantly to GC pause times. Other kinds of memory could offer advantages to such applications. For example, Intel's 3D XPoint memory offers large memory capacities that are more affordable than RAM and speeds that are much faster than NAND storage. Currently, there is no well-defined interface that can be used to offer a Java developer access to these other kinds of memory. Specifically, we propose adding a common memory interface that can be implemented by an open-ended set of memory classes. The interface would provide one method that allocates a java.nio.ByteBuffer on the memory associated with the specific memory instance. package java.nio; interface Memory { public ByteBuffer allocateByteBuffer(int size); } These ByteBuffers will have the same behavior as those allocated by the allocateDirect method in java.nio.ByteBuffer: 1) the ByteBuffer instance is managed on the Java heap, 2) the buffer's backing memory resides on whatever space the memory instance represents, and 3) the backing memory is automatically freed when the ByteBuffer object is garbage collected. Developers would obtain instances of these Memory objects by calling public constructors on specific memory classes. We propose having developers call constructors directly because it is a simple way to accommodate varying initialization requirements (e.g. partitioning) that specific memory instances may have. For uniformity, we propose mirroring the existing off-heap java.nio.ByteBuffer allocation through an off-heap RAM class that implements the Memory interface: package java.nio; public class OffHeapRAM implements Memory { @Override public ByteBuffer allocateByteBuffer(int size) { return ByteBuffer.allocateDirect(size); } } Uniform access could be extended to on-heap ByteBuffers with a class that wraps the non-direct allocation method in ByteBuffer: package java.nio; public class HeapRAM implements Memory { @Override public ByteBuffer allocateByteBuffer(int size) { return ByteBuffer.allocate(size); } } The 3 additions above are the only changes proposed. Links to a bug report and to a JDK 9 patch containing these additions are shown below. For sample code, we are also creating a class that implements the Memory interface for java.nio.ByteBuffers backed by Intel's 3D XPoint memory. bug: https://bugs.openjdk.java.net/browse/JDK-8153111 patch: http://cr.openjdk.java.net/~vdeshpande/8153111/webrev.00/ While other useful capabilities in this space (e.g. persistent memory, process-shared memory) are being explored, they are specifically not addressed or proposed here. We believe that supporting java.nio.ByteBuffer allocation on other memory spaces is sufficiently useful to propose it now for JDK 9. Please let us know if there is interest in this proposal and if you would like to sponsor a patch. Best regards, Steve Dohrmann

10 12

RFR: 8152641: Plugin to generate BMH$Species classes ahead-of-time
by Claes Redestad 01 Apr '16

01 Apr '16

Hi, please review this patch which add an enabled-by-default plugin to generate a configurable list of BoundMethodHandle$Species_*-classes using jlink. Bug: https://bugs.openjdk.java.net/browse/JDK-8152641 Webrev: http://cr.openjdk.java.net/~redestad/8152641/webrev.01/ This plugin adds the --generate-bmh flag to jlink, which takes a comma-separated list of shortened species type strings. As an example: --generate-bmh LL,L3,L4 will generate BoundMethodHandle$Species_LL, ...$Species_L3 and ...$Species_L4 and add them to the java.base module. The Species class lookup in BoundMethodHandle will first check if there is a generated class, otherwise generate it as previously. Adding an exceptional path might seem problematic, but as the code generation is magnitudes more expensive than the exception it doesn't seem fruitful to go to lengths to avoid the CNFE. More notes along with some results can be found here: http://cr.openjdk.java.net/~redestad/scratch/bmh_species_gen.txt Thanks! /Claes

7 25