From chris.hegarty at oracle.com Mon Jan 4 14:02:34 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 4 Jan 2016 14:02:34 +0000 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc Message-ID: <568A7B7A.5060001@oracle.com> sun.misc.VM provides a low-level interface for a small number of specific operations with the VM. In preparation for JEP 260, this class should be moved out of sun.misc and located in a non-exported package [. http://cr.openjdk.java.net/~chegar/8145544/00/ Note: as in other areas some tests that require access to internal APIs have been updated to use types from a more stable package, sun.security.x509. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8145544 From sean.mullan at oracle.com Mon Jan 4 14:12:16 2016 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 4 Jan 2016 09:12:16 -0500 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568A7B7A.5060001@oracle.com> References: <568A7B7A.5060001@oracle.com> Message-ID: <568A7DC0.6090604@oracle.com> On 01/04/2016 09:02 AM, Chris Hegarty wrote: > sun.misc.VM provides a low-level interface for a small number > of specific operations with the VM. In preparation for JEP 260, > this class should be moved out of sun.misc and located in a > non-exported package [. > > http://cr.openjdk.java.net/~chegar/8145544/00/ The security-libs changes look fine to me. > > Note: as in other areas some tests that require access to > internal APIs have been updated to use types from a more > stable package, sun.security.x509. Not sure what you mean - which tests do you mean and why is sun.security.x509 related to this issue? --Sean > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8145544 From weijun.wang at oracle.com Mon Jan 4 14:13:30 2016 From: weijun.wang at oracle.com (Wang Weijun) Date: Mon, 4 Jan 2016 22:13:30 +0800 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568A7B7A.5060001@oracle.com> References: <568A7B7A.5060001@oracle.com> Message-ID: I am OK with the change for krb5 in both src and test. Thanks Max > On Jan 4, 2016, at 10:02 PM, Chris Hegarty wrote: > > sun.misc.VM provides a low-level interface for a small number > of specific operations with the VM. In preparation for JEP 260, > this class should be moved out of sun.misc and located in a > non-exported package [. > > http://cr.openjdk.java.net/~chegar/8145544/00/ > > Note: as in other areas some tests that require access to > internal APIs have been updated to use types from a more > stable package, sun.security.x509. > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8145544 From chris.hegarty at oracle.com Mon Jan 4 14:14:12 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 4 Jan 2016 14:14:12 +0000 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568A7DC0.6090604@oracle.com> References: <568A7B7A.5060001@oracle.com> <568A7DC0.6090604@oracle.com> Message-ID: <568A7E34.8060703@oracle.com> On 04/01/16 14:12, Sean Mullan wrote: > On 01/04/2016 09:02 AM, Chris Hegarty wrote: >> sun.misc.VM provides a low-level interface for a small number >> of specific operations with the VM. In preparation for JEP 260, >> this class should be moved out of sun.misc and located in a >> non-exported package [. >> >> http://cr.openjdk.java.net/~chegar/8145544/00/ > > The security-libs changes look fine to me. Thanks Sean. >> Note: as in other areas some tests that require access to >> internal APIs have been updated to use types from a more >> stable package, sun.security.x509. > > Not sure what you mean - which tests do you mean and why is > sun.security.x509 related to this issue? Not an issue for security, there are a couple of langtools tests that verify that the compiler issues an appropriate warning when internal types are used. These tests used to use sun.misc.VM, I changed them to use types from sun.security.x509. We have done this in a few other placed too. -Chris. > --Sean > >> >> -Chris. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8145544 From Alan.Bateman at oracle.com Mon Jan 4 14:16:20 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 4 Jan 2016 14:16:20 +0000 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568A7B7A.5060001@oracle.com> References: <568A7B7A.5060001@oracle.com> Message-ID: <568A7EB4.4080308@oracle.com> On 04/01/2016 14:02, Chris Hegarty wrote: > sun.misc.VM provides a low-level interface for a small number > of specific operations with the VM. In preparation for JEP 260, > this class should be moved out of sun.misc and located in a > non-exported package [. > > http://cr.openjdk.java.net/~chegar/8145544/00/ > > Note: as in other areas some tests that require access to > internal APIs have been updated to use types from a more > stable package, sun.security.x509. This looks okay to me. There's a few places where importing jdk.internal.misc.VM might be nicer to avoid repeating the fully qualified name - java.lang.Class and java.lang.System are two examples. In passing then I wonder if it's time to drop OSEnvironment, I don't think it has been needed for a long time (the win32 error mode stuff is legacy). -Alan. From sean.mullan at oracle.com Mon Jan 4 14:19:48 2016 From: sean.mullan at oracle.com (Sean Mullan) Date: Mon, 4 Jan 2016 09:19:48 -0500 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568A7E34.8060703@oracle.com> References: <568A7B7A.5060001@oracle.com> <568A7DC0.6090604@oracle.com> <568A7E34.8060703@oracle.com> Message-ID: <568A7F84.4060707@oracle.com> On 01/04/2016 09:14 AM, Chris Hegarty wrote: >>> Note: as in other areas some tests that require access to >>> internal APIs have been updated to use types from a more >>> stable package, sun.security.x509. >> >> Not sure what you mean - which tests do you mean and why is >> sun.security.x509 related to this issue? > > > Not an issue for security, there are a couple of langtools tests > that verify that the compiler issues an appropriate warning when > internal types are used. These tests used to use sun.misc.VM, I > changed them to use types from sun.security.x509. We have done > this in a few other placed too. Ah, ok. --Sean From kubota.yuji at gmail.com Mon Jan 4 14:26:52 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Mon, 4 Jan 2016 23:26:52 +0900 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> Message-ID: Hi Sherman and all, Happy new year to everyone! Please let know your feedback about this proposal. :-) Thanks, Yuji 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji : > Hi Sherman, > > 2015-12-20 16:35 GMT+09:00 Xueming Shen : >> It is no longer necessary to touch the native code (zip_util.c/h) after the >> native ZipFile implementation has been moved up to the java level. Those >> native code are for vm access only now, which I dont think care about the >> password support at all. > > Thanks for your information. We do not take care the native. > > I discussed with Yasumasa, and our thought is as below. > >> (1) what's the benefit of exposing the public interface ZipCryption? the real >> question is whether or not this interface is good enough for other encryption >> implementation to plugin their implementation to support the ZipFile/Input/ >> OutputStream to their encryption spec. > > We aimed that the public interface ZipCryption supports the > extensibillity for other encrypt engine. The JDK core libs developers > have to implementation ZipyCryption only. If not provide, the JDK > developers must implement ZipStream/Entry by JDK API to design the > data structure of entry. > If you want to use binary key data such as PKI, you can implement new > encrypt/decrypt engine by ZipCryption interface. > So we think we should provide this interface to be clearly how to > implement a new engine, e.g., cipher algorithm, cipher strength and > converting the header, etc. > >> (2) it seems like it might be possible to hide most of the implementation >> and only expose the "String password" (instead of the ZipCryption) as the >> public interface to support the "traditional" encryption. This depends on the >> result of (1) though. > > Thanks for your clues. We think the string password at first. However, > we should also create a new binary interface given we support PKI in > the future. > >> (3) I'm concerned of pushing ZipCryption into InflaterInputStream/DeflaterOutputStream. >> It might be worth considering to replace the ZipCryption implementation with >> a pair of FilterOutput/InputStream. It would be easy and reasonable to use >> the FilterOutputStream for the ZipOutputStream and the FilterInputStream for the >> ZipFile. The PushbackInputStream in ZipInputStream might be an issue ... > > Thanks for your clues, too. Honestly speaking, we think the current > zip implementation may break the data when used PushbackInputStream > for the following reasons. > > * PushbackInputStream uses an unique internal buffer for re-read operation. > * But, InflaterInputStream provide date to Inflater per reads and > buffer by JNI (zlib). > * So we think PushbackInputStream is poor compatibility with > InflaterInputStream. > > We generally use InputStream through ZipEntry#getInputStream(). We do > not touch FileInputStream for reading ZIP data. If we call unread() > when we use PushbackInputStream as reading ZIP archive, we guess that > it will break the reading data. > So, our approach do not affect the PushbackInputStream. > What do you think about this? > >> (4) It seems the ZipOutputStream only supports the "stream based" password, while >> the ZipInputStream supports the "entry based" password. Do we really need >> "entry based" support here? > > As your suggestion, we should support "entry based". We will start to > implement "entry based" after this discussion is closed. > > Thanks, > Yuji > >> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>> >>> Hi Jason, >>> >>> Thank you for your comment. >>> I've fixed it in new webrev: >>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>> >>> >>> Thanks, >>> >>> Yasumasa >>> >>> >>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>> >>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is in >>>> conflict with line 350 and 348. >>>> >>>> ________________________________________ >>>> From: core-libs-dev on behalf of >>>> Yasumasa Suenaga >>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>> To: Sergey Bylokhov; Xueming Shen >>>> Cc: core-libs-dev at openjdk.java.net >>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>> protection to Zip entries >>>> >>>> Hi Sergey, >>>> >>>> Thank you for your comment. >>>> >>>> I added that description in new webrev: >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>> >>>>> Should the new methods describe how they will work in case of null >>>>> params? >>>>> >>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>> >>>>>> I adapted this enhancement after JDK-8145260: >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>> >>>>>> Could you review it? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> Hi Sherman, >>>>>>> >>>>>>> Our proposal is affected by JDK-8142508. >>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>> Thus we will create a new webrev for current (after 8142508) jdk9/dev >>>>>>> repos. >>>>>>> >>>>>>> Do you have any comments about current webrev? >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>> >>>>>>> If you have comments, we will fix them in new webrev. >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>> >>>>>>>> Hi Sherman, >>>>>>>> >>>>>>>> Thanks for your quick response :) >>>>>>>> >>>>>>>> I aimed to implement the "traditional" at this proposal by the below >>>>>>>> reasons. >>>>>>>> >>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>> * Many people use the "traditional" in problem-free scope like a >>>>>>>> temporary file. >>>>>>>> * We do not know which implementation of the "stronger" is best >>>>>>>> for >>>>>>>> openjdk. >>>>>>>> * PKWare claims that they have patents about the "stronger" on >>>>>>>> Zip[1]. >>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>> "stronger" [2][3]. >>>>>>>> * Instead, we prepared the extensibility by ZipCryption interface >>>>>>>> to >>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>> >>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>> In the future, anyone who want to implement the "stronger" can easily >>>>>>>> add their code by virtue of this proposal. >>>>>>>> >>>>>>>> [1] https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>> (1.4 Permitted Use & 7.0 Strong Encryption Specification) >>>>>>>> [2] >>>>>>>> >>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>> >>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Yuji >>>>>>>> >>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming Shen : >>>>>>>>> >>>>>>>>> >>>>>>>>> Hi Yuji, >>>>>>>>> >>>>>>>>> I will take a look at your PoC. Might need some time and even bring >>>>>>>>> in the >>>>>>>>> security guy >>>>>>>>> to evaluate the proposal. It seems like you are only interested in >>>>>>>>> the >>>>>>>>> "traditional PKWare >>>>>>>>> decryption", which is, based on the wiki, "known to be seriously >>>>>>>>> flawed, and >>>>>>>>> in particular >>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to support >>>>>>>>> "stronger" encryption >>>>>>>>> mechanism, such as the AES based? >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Sherman >>>>>>>>> >>>>>>>>> >>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Hi all, >>>>>>>>>> >>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Yuji >>>>>>>>>> >>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji : >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent before >>>>>>>>>>> get >>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>> >>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>> However, >>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we must use >>>>>>>>>>> third party library so far. >>>>>>>>>>> >>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>> >>>>>>>>>>> The current zlib in JDK is completely unaffected by this proposal. >>>>>>>>>>> The >>>>>>>>>>> traditional zip encryption encrypts a data after it is has been >>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing zlib >>>>>>>>>>> implementation. >>>>>>>>>>> >>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> Test code is as below. This code will let you know how this >>>>>>>>>>> PoC >>>>>>>>>>> works. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>> >>>>>>>>>>> In NTT, a Japanese telecommunications company. We are providing >>>>>>>>>>> many >>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>> implement to >>>>>>>>>>> handle password-protected zip file. I guess that this proposal is >>>>>>>>>>> desired for many developers and users. >>>>>>>>>>> >>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>> (ysuenaga). >>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>> >>>>>>>>>>> [1]: https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Yuji >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>> >>>>> >> From chris.hegarty at oracle.com Mon Jan 4 14:37:54 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 4 Jan 2016 14:37:54 +0000 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568A7EB4.4080308@oracle.com> References: <568A7B7A.5060001@oracle.com> <568A7EB4.4080308@oracle.com> Message-ID: <568A83C2.2090807@oracle.com> On 04/01/16 14:16, Alan Bateman wrote: > On 04/01/2016 14:02, Chris Hegarty wrote: >> sun.misc.VM provides a low-level interface for a small number >> of specific operations with the VM. In preparation for JEP 260, >> this class should be moved out of sun.misc and located in a >> non-exported package [. >> >> http://cr.openjdk.java.net/~chegar/8145544/00/ >> >> Note: as in other areas some tests that require access to >> internal APIs have been updated to use types from a more >> stable package, sun.security.x509. > This looks okay to me. Thanks Alan. > There's a few places where importing > jdk.internal.misc.VM might be nicer to avoid repeating the fully > qualified name - java.lang.Class and java.lang.System are two examples. I see these. Integer and few others too. I fixed them and updated the webrev in-place. > In passing then I wonder if it's time to drop OSEnvironment, I don't > think it has been needed for a long time (the win32 error mode stuff is > legacy). Right. If it is ok, I'll file another bug and do the cleanup as a follow up. -Chris. From jaroslav.bachorik at oracle.com Mon Jan 4 14:51:11 2016 From: jaroslav.bachorik at oracle.com (Jaroslav Bachorik) Date: Mon, 4 Jan 2016 15:51:11 +0100 Subject: [PING] Potential infinite waiting at JMXConnection#createConnection In-Reply-To: References: Message-ID: <568A86DF.2030609@oracle.com> Hi Yuji, On 4.1.2016 15:14, KUBOTA Yuji wrote: > Hi all, > > Could you please review this patch? Sorry for the long delay. Shanliang has not been present for some time and probably this slipped the attention of the others. However, core-libs mailing list might be more appropriate place to review this change since you are dealing with s.r.t.t.TCPChannel (http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch) Regards, -JB- > > Thanks, > Yuji > > 2015-12-17 2:05 GMT+09:00 KUBOTA Yuji : >> Hi all, >> >> If Shanliang cannot review, someone could review it? >> >> I reported this issue first at the below. >> http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002152.html >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-May/017241.html >> >> Hope this patch helps to community. >> >> Thanks, >> Yuji >> >> 2015-12-10 22:40 GMT+09:00 KUBOTA Yuji : >>> Hi Shanliang and all, >>> >>> Sorry my reply is too late. But, finally, I reproduced this issue by >>> following test program! :) >>> Could you please review test program and my patch ? >>> >>> The test program includes some files, but I do not have an account of >>> openjdk, so I push it on icedtea server as below. >>> >>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f >>> >>> The test program starts a simple sleep server program (JMXSSLServer) >>> on external jdb process with a breakpoint at >>> sun.security.ssl.ServerHandshaker.clientHello set. It then starts a >>> client process (JMXSSLCient) which tries to connect the sleep/jdb >>> process. >>> ServerHandshaker.clientHello responds to the client hello message and >>> sends SSL record back. By setting breakpont in that function, we can >>> emulate this issue in which client keeps waiting SSL record from >>> server. >>> Now, JMXConnectorFactory.connect() ignores >>> sun.rmi.transport.tcp.responseTimeout, so wait the response (SSL >>> record) from server INFINITELY. Once my patch (jdk9.patch) was added, >>> then the client will return 0 when the connection timeout happen by >>> sun.rmi.transport.tcp.responseTimeout. >>> And test program returns the client's return code. >>> >>> You can reproduce infinite waiting by the below. >>> * hg clone http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/ >>> * cd fixLoopAtJMXConnectorFactory/testProgram >>> * set JAVA_HOME >>> * mvn package >>> * setting debugcontrol.properties if you need. >>> * *NOTE* This test program will stop for "debugcontroltest.stop_time" ms. >>> * ${JAVA_HOME}/bin/java -cp .:target/debugcontrol-1.0-SNAPSHOT.jar >>> debugcontrol.DebugController >>> >>> * The result by JDK without my patch. >>> The client throws java.net.ConnectException after sending quit to jdb. >>> ------ >>> [INFO] Server process args >>> args[0] /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.65-3.b17.fc23.x86_64/bin/jdb >>> : >>> : >>> cli-out: [INFO] Service URL: service:jmx:rmi:///jndi/rmi://localhost:9876/jmxrmi >>> ser-err: Set deferred breakpoint sun.security.ssl.ServerHandshaker.clientHello >>> ser-out: >>> ser-err: Breakpoint hit: "thread=RMI TCP Connection(2)-127.0.0.1", >>> sun.security.ssl.ServerHandshaker.clientHello(), line=339 bci=0 >>> ser-out: >>> [INFO] sending quit to jdb >>> ser-err: RMI TCP Connection(2)-127.0.0.1[1] >>> cli-err: java.rmi.ConnectException: Connection refused to host: >>> 127.0.0.1; nested exception is: >>> cli-err: java.net.ConnectException: Connection refused >>> cli-err: at >>> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619) >>> cli-err: at >>> sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216) >>> cli-err: at >>> sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) >>> cli-err: at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:130) >>> cli-err: at >>> javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown >>> Source) >>> cli-err: at >>> javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2432) >>> cli-err: at >>> javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:308) >>> cli-err: at >>> javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) >>> cli-err: at >>> debugcontrol.client.JMXSSLClient.execute(JMXSSLClient.java:51) >>> cli-err: at debugcontrol.client.JMXSSLClient.main(JMXSSLClient.java:34) >>> cli-err: Caused by: java.net.ConnectException: Connection refused >>> cli-err: at java.net.PlainSocketImpl.socketConnect(Native Method) >>> cli-err: at >>> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) >>> cli-err: at >>> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) >>> cli-err: at >>> java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) >>> cli-err: at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) >>> cli-err: at java.net.Socket.connect(Socket.java:589) >>> cli-err: at >>> sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668) >>> cli-err: at sun.security.ssl.SSLSocketImpl.(SSLSocketImpl.java:427) >>> cli-err: at >>> sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:88) >>> cli-err: at >>> javax.rmi.ssl.SslRMIClientSocketFactory.createSocket(SslRMIClientSocketFactory.java:121) >>> cli-err: at >>> sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613) >>> cli-err: ... 9 more >>> [INFO] Thu Dec 10 16:16:50 JST 2015 Client done. Result code: 2 >>> [INFO] Client took 114462 msec. >>> >>> * By JDK 9 with my patch. >>> The client got java.net.SocketTimeoutException after the connection >>> timeout happen, then return 0. >>> ------ >>> [INFO] Server process args >>> args[0] /workspace/jdk9-netdev-patch/build/linux-x86_64-normal-server-release/jdk/bin/jdb >>> args[1] -classpath >>> args[2] target/classes >>> args[3] -J-Duser.language=en >>> args[4] -Dcom.sun.management.jmxremote.port=9876 >>> args[5] -Dcom.sun.management.jmxremote.password.file=jmxremote.password >>> args[6] -Djavax.net.ssl.keyStore=jmx-test-cert.pkcs12 >>> args[7] -Djavax.net.ssl.keyStoreType=pkcs12 >>> args[8] -Djavax.net.ssl.keyStorePassword=changeit >>> args[9] debugcontrol.server.JMXSSLServer >>> ser-out: Initializing jdb ... >>> ser-err: > Deferring breakpoint sun.security.ssl.ServerHandshaker.clientHello. >>> ser-out: It will be set after the class is loaded. >>> ser-err: > run debugcontrol.server.JMXSSLServer >>> ser-err: Set uncaught java.lang.Throwable >>> ser-out: Set deferred uncaught java.lang.Throwable >>> ser-err: > >>> ser-out: VM Started: [INFO] Server launched then sleep... >>> [INFO] Client process args: >>> args[0] /workspace/jdk9-netdev-patch/build/linux-x86_64-normal-server-release/jdk/bin/java >>> args[1] -classpath >>> args[2] target/classes >>> args[3] -Duser.language=en >>> args[4] -Djavax.net.ssl.trustStore=jmx-test-cert.pkcs12 >>> args[5] -Djavax.net.ssl.trustStoreType=pkcs12 >>> args[6] -Djavax.net.ssl.trustStorePassword=changeit >>> args[7] -Dsun.rmi.transport.tcp.responseTimeout=1000 >>> args[8] -Dsun.rmi.transport.tcp.handshakeTimeout=1000 >>> args[9] debugcontrol.client.JMXSSLClient >>> args[10] localhost >>> args[11] 9876 >>> cli-out: [INFO] Service URL: service:jmx:rmi:///jndi/rmi://localhost:9876/jmxrmi >>> ser-err: Set deferred breakpoint sun.security.ssl.ServerHandshaker.clientHello >>> ser-out: >>> ser-err: Breakpoint hit: "thread=RMI TCP Connection(2)-127.0.0.1", >>> sun.security.ssl.ServerHandshaker.clientHello(), line=356 bci=0 >>> ser-out: >>> cli-out: [INFO] Conglaturation. We got a timeout. >>> [INFO] Thu Dec 10 15:54:39 JST 2015 Client done. Result code: 0 >>> >>> >>> * My patch (no change from the reported) >>> * http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch >>> ------ >>> diff --git a/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java >>> b/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java >>> --- a/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java >>> +++ b/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java >>> @@ -224,6 +224,22 @@ >>> out.writeByte(TransportConstants.SingleOpProtocol); >>> } else { >>> out.writeByte(TransportConstants.StreamProtocol); >>> + >>> + int usableSoTimeout = 0; >>> + try { >>> + /* >>> + * If socket factory had set a non-zero timeout on its >>> + * own, then restore it instead of using the property- >>> + * configured value. >>> + */ >>> + usableSoTimeout = sock.getSoTimeout(); >>> + if (usableSoTimeout == 0) { >>> + usableSoTimeout = responseTimeout; >>> + } >>> + sock.setSoTimeout(usableSoTimeout); >>> + } catch (Exception e) { >>> + // if we fail to set this, ignore and proceed anyway >>> + } >>> out.flush(); >>> >>> /* >>> @@ -231,9 +247,7 @@ >>> * connection handshake; this also serves to guard against >>> * non-JRMP servers that do not respond (see 4322806). >>> */ >>> - int originalSoTimeout = 0; >>> try { >>> - originalSoTimeout = sock.getSoTimeout(); >>> sock.setSoTimeout(handshakeTimeout); >>> } catch (Exception e) { >>> // if we fail to set this, ignore and proceed anyway >>> @@ -281,14 +295,7 @@ >>> * that a remote method call is permitted to execute. >>> */ >>> try { >>> - /* >>> - * If socket factory had set a non-zero timeout on its >>> - * own, then restore it instead of using the property- >>> - * configured value. >>> - */ >>> - sock.setSoTimeout((originalSoTimeout != 0 ? >>> - originalSoTimeout : >>> - responseTimeout)); >>> + sock.setSoTimeout(usableSoTimeout); >>> } catch (Exception e) { >>> // if we fail to set this, ignore and proceed anyway >>> } >>> >>> Thanks, >>> Yuji From xueming.shen at oracle.com Mon Jan 4 21:02:45 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 04 Jan 2016 13:02:45 -0800 Subject: RFR JDK-8146431:j.u.z.ZipFile.getEntry("") throws AIOOBE Message-ID: <568ADDF5.307@oracle.com> Hi, please help review the change for JDK-8146431 Issue: https://bugs.openjdk.java.net/browse/JDK-8146431 webrev: http://cr.openjdk.java.net/~sherman/8146431 Cause: The parameter "ulen" in native version zip_util.c.ZIP_GetEntry() serves two purposes, (1) the length of the name (2) whether or not to try adding a slash if first try failed. It has an explicit check as /* If no real length was passed in, we are done */ if (ulen == 0) { break; } The new java implementation uses and checks the flag "addSlash", but it still needs to check the "name.len" before adding any slash for next try. Thanks, Sherman From lance.andersen at oracle.com Mon Jan 4 21:01:53 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 4 Jan 2016 16:01:53 -0500 Subject: RFR JDK-8146431:j.u.z.ZipFile.getEntry("") throws AIOOBE In-Reply-To: <568ADDF5.307@oracle.com> References: <568ADDF5.307@oracle.com> Message-ID: <038D9127-95A3-410B-B63F-BEC826EEB87B@oracle.com> Looks OK Sherman? Best Lance On Jan 4, 2016, at 4:02 PM, Xueming Shen wrote: > Hi, please help review the change for JDK-8146431 > > Issue: https://bugs.openjdk.java.net/browse/JDK-8146431 > webrev: http://cr.openjdk.java.net/~sherman/8146431 > > Cause: > > The parameter "ulen" in native version zip_util.c.ZIP_GetEntry() serves two > purposes, (1) the length of the name (2) whether or not to try adding a > slash if first try failed. It has an explicit check as > > /* If no real length was passed in, we are done */ > if (ulen == 0) { > break; > } > > The new java implementation uses and checks the flag "addSlash", but it > still needs to check the "name.len" before adding any slash for next try. > > Thanks, > Sherman > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From claes.redestad at oracle.com Mon Jan 4 21:05:22 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 04 Jan 2016 22:05:22 +0100 Subject: RFR JDK-8146431:j.u.z.ZipFile.getEntry("") throws AIOOBE In-Reply-To: <568ADDF5.307@oracle.com> References: <568ADDF5.307@oracle.com> Message-ID: <568ADE92.6070706@oracle.com> Thanks for fixing this! Can getEntryPos(new byte[0], true/false); ever return anything but -1? Otherwise it seems the check could be moved to the start of the method. /Claes On 2016-01-04 22:02, Xueming Shen wrote: > Hi, please help review the change for JDK-8146431 > > Issue: https://bugs.openjdk.java.net/browse/JDK-8146431 > webrev: http://cr.openjdk.java.net/~sherman/8146431 > > Cause: > > The parameter "ulen" in native version zip_util.c.ZIP_GetEntry() > serves two > purposes, (1) the length of the name (2) whether or not to try adding a > slash if first try failed. It has an explicit check as > > /* If no real length was passed in, we are done */ > if (ulen == 0) { > break; > } > > The new java implementation uses and checks the flag "addSlash", but it > still needs to check the "name.len" before adding any slash for next try. > > Thanks, > Sherman > From xueming.shen at oracle.com Mon Jan 4 21:30:22 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 04 Jan 2016 13:30:22 -0800 Subject: RFR JDK-8146431:j.u.z.ZipFile.getEntry("") throws AIOOBE In-Reply-To: <568ADE92.6070706@oracle.com> References: <568ADDF5.307@oracle.com> <568ADE92.6070706@oracle.com> Message-ID: <568AE46E.8050206@oracle.com> Claes, I think the spec does not say the jar/zip file can't have an empty name entry. So it'd be better to check first. Sherman On 01/04/2016 01:05 PM, Claes Redestad wrote: > Thanks for fixing this! > > Can getEntryPos(new byte[0], true/false); ever return anything but -1? Otherwise it seems the check could be moved to the start of the method. > > /Claes > > On 2016-01-04 22:02, Xueming Shen wrote: >> Hi, please help review the change for JDK-8146431 >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8146431 >> webrev: http://cr.openjdk.java.net/~sherman/8146431 >> >> Cause: >> >> The parameter "ulen" in native version zip_util.c.ZIP_GetEntry() serves two >> purposes, (1) the length of the name (2) whether or not to try adding a >> slash if first try failed. It has an explicit check as >> >> /* If no real length was passed in, we are done */ >> if (ulen == 0) { >> break; >> } >> >> The new java implementation uses and checks the flag "addSlash", but it >> still needs to check the "name.len" before adding any slash for next try. >> >> Thanks, >> Sherman >> > From martinrb at google.com Mon Jan 4 22:54:01 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 4 Jan 2016 14:54:01 -0800 Subject: RFR JDK-8146431:j.u.z.ZipFile.getEntry("") throws AIOOBE In-Reply-To: <568ADDF5.307@oracle.com> References: <568ADDF5.307@oracle.com> Message-ID: ZipFile/Assortment already tests empty entry names. Why didn't that test catch any error? IOW, ;how can ZipFile/Assortment be improved? On Mon, Jan 4, 2016 at 1:02 PM, Xueming Shen wrote: > Hi, please help review the change for JDK-8146431 > > Issue: https://bugs.openjdk.java.net/browse/JDK-8146431 > webrev: http://cr.openjdk.java.net/~sherman/8146431 > > Cause: > > The parameter "ulen" in native version zip_util.c.ZIP_GetEntry() serves two > purposes, (1) the length of the name (2) whether or not to try adding a > slash if first try failed. It has an explicit check as > > /* If no real length was passed in, we are done */ > if (ulen == 0) { > break; > } > > The new java implementation uses and checks the flag "addSlash", but it > still needs to check the "name.len" before adding any slash for next try. > > Thanks, > Sherman > From xueming.shen at oracle.com Tue Jan 5 00:29:22 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 04 Jan 2016 16:29:22 -0800 Subject: RFR JDK-8146431:j.u.z.ZipFile.getEntry("") throws AIOOBE In-Reply-To: References: <568ADDF5.307@oracle.com> Message-ID: <568B0E62.8060107@oracle.com> On 01/04/2016 02:54 PM, Martin Buchholz wrote: > ZipFile/Assortment already tests empty entry names. Why didn't that > test catch any error? IOW, ;how can ZipFile/Assortment be improved? The bug only occurs if there is no such empty name entry inside the target zip file. The ZipFile/Assortment probably would be the better place to put test. Did not pay attention to the old tests. -Sherman > On Mon, Jan 4, 2016 at 1:02 PM, Xueming Shen wrote: >> Hi, please help review the change for JDK-8146431 >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8146431 >> webrev: http://cr.openjdk.java.net/~sherman/8146431 >> >> Cause: >> >> The parameter "ulen" in native version zip_util.c.ZIP_GetEntry() serves two >> purposes, (1) the length of the name (2) whether or not to try adding a >> slash if first try failed. It has an explicit check as >> >> /* If no real length was passed in, we are done */ >> if (ulen == 0) { >> break; >> } >> >> The new java implementation uses and checks the flag "addSlash", but it >> still needs to check the "name.len" before adding any slash for next try. >> >> Thanks, >> Sherman >> From kim.barrett at oracle.com Tue Jan 5 02:25:28 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 4 Jan 2016 21:25:28 -0500 Subject: RFR: JDK-8145549 Add support for Visual Studio 2015 Community edition In-Reply-To: References: <7B657FB8-B581-4AB2-81C1-877B4B1BD658@oracle.com> Message-ID: <98B07F13-1D90-4DEC-AAED-D791F4666BF7@oracle.com> On Dec 18, 2015, at 7:41 PM, Kim Barrett wrote: > > On Dec 16, 2015, at 8:50 AM, Magnus Ihse Bursie wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8145549 >> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8145549-vs2015-community-edition/webrev.01 >> >> /Magnus > > I only looked at hotspot files. > > [?] > There are a couple of "TBD"s below that I need to spend more time on. > Back from holiday, and here?s my comments on those two TBDs ------------------------------------------------------------------------------ hotspot/agent/src/share/native/sadis.c 96 return (int)strlen(buf); The only call to the (file-scoped) getLastErrorString function is Java_sun_jvm_hotspot_asm_Disassembler_load_1library, which ignores the result. It would be better to change the return type of getLastErrorString to void and update the return statements accordingly. ------------------------------------------------------------------------------ hotspot/src/share/vm/adlc/arena.hpp 74 // Usual (non-placement) deallocation function to allow placement delete use size_t 75 // See 3.7.4.2 [basic.stc.dynamic.deallocation] paragraph 2. 76 void operator delete(void* p); and associated code in the .cpp file. I think this is another C++11 change: http://open-std.org/JTC1/SC22/WG21/docs/cwg_defects.html#429 I think the proposed code change is OK, although the comment is problematic: [basic.stc.dynamic.deallocation] is C++03 3.7.3.2 and C++11 3.7.4.2. Also, the standard change that leads to this code change is in C++11 5.3.4 [expr.new] paragraph 20 (C++02 p 19). Also, I *think* with the addition of the one-arg operator delete we don't actually use the two-arg form. If so, then I suggest removing it and the proposed new comment for the one-arg form. ------------------------------------------------------------------------------ From mandy.chung at oracle.com Tue Jan 5 02:50:43 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 4 Jan 2016 18:50:43 -0800 Subject: RFR (JAXP): 8144966: Catalog API: Null handling and reference to Reader In-Reply-To: <567B264E.1020502@oracle.com> References: <567B264E.1020502@oracle.com> Message-ID: <27D13FC1-CF10-4DDD-8F63-1D70FE8DBAE4@oracle.com> > On Dec 23, 2015, at 2:55 PM, huizhe wang wrote: > > Hi, > > This is an improvement to the new Catalog API on null handling. At the package level, a null argument shall result in NPE. However, an exception was made to the methods in CatalogManager so that they can used conveniently. With this request, we're in favor of consistency over the minor convenience, which means a resolver can no longer be created by: > CatalogManager.catalogResolver(null); > > Instead, the following may be used to achieve the same result: > CatalogManager.catalogResolver(CatalogFeatures.defaults()); > > While path can be empty, it can no longer be null. The following then would also get a NPE: > CatalogManager.catalogResolver(CatalogFeatures.defaults(), null); > This change is reasonable and it?s good to pass CatalogFeatures.defaults() explicitly rather than special casing null to represent that. > In consistence with ignoring invalid catalogs as required by the Catalog standard, the statements that required CatalogException when no catalog is found are removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8144966 > webrev: http://cr.openjdk.java.net/~joehw/jdk9/8144966/webrev/ > 41 * Creates a {@code Catalog} object using the specified feature settings and path to 42 * a catalog file. If the path is empty, Nit: should it say ?path to one or more catalog files? What if ?paths? is non-empty but all entries are invalid and no catalog file is read - will it read the system property? line 46: s/path argument/paths argument/ Same comment applied to other factory methods. CatalogImpl.java 117 if (file.length > 0) { - Do you expect file to be null? If so, NPE will be thrown. If not, file == null in line 125 is not needed. Otherwise the change looks okay. Mandy From kubota.yuji at gmail.com Tue Jan 5 08:56:30 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Tue, 5 Jan 2016 17:56:30 +0900 Subject: [PING] Potential infinite waiting at JMXConnection#createConnection In-Reply-To: <568A86DF.2030609@oracle.com> References: <568A86DF.2030609@oracle.com> Message-ID: Hi Jaroslav and core-libs-dev, Thank Jaroslav for your kindness! For core-libs-dev members, links the information about this issue. * details of problem http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002152.html * patch http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch * testcase for reproduce http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/testProgram http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-December/018415.html Could you please review these reports? Hope this patch helps to community. Thanks, Yuji 2016-01-04 23:51 GMT+09:00 Jaroslav Bachorik : > Hi Yuji, > > On 4.1.2016 15:14, KUBOTA Yuji wrote: >> >> Hi all, >> >> Could you please review this patch? > > > Sorry for the long delay. Shanliang has not been present for some time and > probably this slipped the attention of the others. > > However, core-libs mailing list might be more appropriate place to review > this change since you are dealing with s.r.t.t.TCPChannel > (http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch) > > Regards, > > -JB- From nadeesh.tv at oracle.com Tue Jan 5 18:05:19 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Tue, 05 Jan 2016 23:35:19 +0530 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better Message-ID: <568C05DF.4090606@oracle.com> Hi all, Please review a fix for https://bugs.openjdk.java.net/browse/JDK-8068803 web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ Special thanks for Stephen for providing the source code patch -- Thanks and Regards, Nadeesh TV From Roger.Riggs at Oracle.com Tue Jan 5 18:24:28 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jan 2016 13:24:28 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base Message-ID: <568C0A5C.7090204@Oracle.com> The follow on work to adding the Cleaner is to replace uses of finalization with uses of the Cleaner. For the 'easy' cases in the java.base module, it is useful to introduce a private Cleaner for the java.base module. It is proposed to be held weakly, to allow it to terminate on a lightly loaded system. Webrev for Review: http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ The 2nd step is using the Cleaner. - Empty finalize methods should (I think) be removed; but since they are part of the public spec the process needs two full releases; so the proposal is to deprecate them first. (The JEP 277 necessary changes will be updated when JEP 277 semantics are finalized) - In a few cases, the code in the finalizer is redundant and if removed would allow an optimization of the handling of the finalizer and future removal of the finalize method. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html Thanks for comments and suggestions, Roger From Roger.Riggs at Oracle.com Tue Jan 5 18:41:36 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jan 2016 13:41:36 -0500 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: <568C05DF.4090606@oracle.com> References: <568C05DF.4090606@oracle.com> Message-ID: <568C0E60.2030008@Oracle.com> Hi Nadeesh, LocalDate.java: +1374: For the most common case of dom > 0 && <= 28, I would have explicitly and immediately returned the new LocalDate. if (dom > 0 && dom <= 28) { return LocalDate.of(year, month, (int) dom); } ... TCKLocalDate.java: - Since the test_plusDays_normal is being replaced, its test case should be included in the DataProvider {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} Thanks, Roger On 1/5/2016 1:05 PM, nadeesh tv wrote: > Hi all, > > Please review a fix for > https://bugs.openjdk.java.net/browse/JDK-8068803 > web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ > > Special thanks for Stephen for providing the source code patch > From daniel.fuchs at oracle.com Tue Jan 5 18:45:16 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 5 Jan 2016 19:45:16 +0100 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568C0A5C.7090204@Oracle.com> References: <568C0A5C.7090204@Oracle.com> Message-ID: <568C0F3C.2000906@oracle.com> Hi Roger, Some early feedback: I see that prior to your changes, MeteredStream.close() would be called by finalize. This will no longer be the case after http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html I see that MeteredStream has a subclass (KeepAliveStream) that overrides close() - so KeepAliveStream probably requires a cleanup as well - doesn't it? Finally I'd suggest making the variable 'pi' final - since you're passing that to the lambda you don't want to allow its value to be modified afterwards (fortunately it's not - and I believe it should be good to ensure it won't be). I haven't looked at the other classes yet... best regards, -- daniel On 05/01/16 19:24, Roger Riggs wrote: > The follow on work to adding the Cleaner is to replace uses of > finalization with uses of the Cleaner. > For the 'easy' cases in the java.base module, it is useful to introduce > a private Cleaner for the > java.base module. It is proposed to be held weakly, to allow it to > terminate on a lightly loaded > system. > > Webrev for Review: > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ > > > The 2nd step is using the Cleaner. > - Empty finalize methods should (I think) be removed; but since they > are part of the public spec > the process needs two full releases; so the proposal is to deprecate > them first. > (The JEP 277 necessary changes will be updated when JEP 277 > semantics are finalized) > > - In a few cases, the code in the finalizer is redundant and if removed > would allow > an optimization of the handling of the finalizer and future removal > of the finalize method. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html > > Thanks for comments and suggestions, Roger > > > > From nadeesh.tv at oracle.com Tue Jan 5 19:08:36 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Wed, 06 Jan 2016 00:38:36 +0530 Subject: RFR:8146489:@since tag missed Message-ID: <568C14B4.10309@oracle.com> Hi all, Please review a fix for BugID - https://bugs.openjdk.java.net/browse/JDK-8146489 Issue - while fixing JDK8142936 , I forgot to add @since 9 tag. webrev - http://cr.openjdk.java.net/~ntv/8146489/webrev.00/ -- Thanks and Regards, Nadeesh TV From lance.andersen at oracle.com Tue Jan 5 19:17:43 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 5 Jan 2016 14:17:43 -0500 Subject: RFR:8146489:@since tag missed In-Reply-To: <568C14B4.10309@oracle.com> References: <568C14B4.10309@oracle.com> Message-ID: +1 On Jan 5, 2016, at 2:08 PM, nadeesh tv wrote: > Hi all, > Please review a fix for > BugID - https://bugs.openjdk.java.net/browse/JDK-8146489 > Issue - while fixing JDK8142936 , I forgot to add @since 9 tag. > webrev - http://cr.openjdk.java.net/~ntv/8146489/webrev.00/ > > -- > Thanks and Regards, > Nadeesh TV > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Tue Jan 5 19:31:38 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jan 2016 14:31:38 -0500 Subject: RFR:8146489:@since tag missed In-Reply-To: References: <568C14B4.10309@oracle.com> Message-ID: <568C1A1A.5010406@Oracle.com> +1 Thanks, Roger On 1/5/2016 2:17 PM, Lance Andersen wrote: > +1 > On Jan 5, 2016, at 2:08 PM, nadeesh tv wrote: > >> Hi all, >> Please review a fix for >> BugID - https://bugs.openjdk.java.net/browse/JDK-8146489 >> Issue - while fixing JDK8142936 , I forgot to add @since 9 tag. >> webrev - http://cr.openjdk.java.net/~ntv/8146489/webrev.00/ >> >> -- >> Thanks and Regards, >> Nadeesh TV >> > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From chris.hegarty at oracle.com Tue Jan 5 19:33:37 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 5 Jan 2016 19:33:37 +0000 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568C0A5C.7090204@Oracle.com> References: <568C0A5C.7090204@Oracle.com> Message-ID: <06A536F9-13AC-4A82-9FFB-129086494CFC@oracle.com> On 5 Jan 2016, at 18:24, Roger Riggs wrote: > The follow on work to adding the Cleaner is to replace uses of finalization with uses of the Cleaner. > For the 'easy' cases in the java.base module, it is useful to introduce a private Cleaner for the > java.base module. It is proposed to be held weakly, to allow it to terminate on a lightly loaded > system. > > Webrev for Review: > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ Looks ok Roger. Does it make sense to put CleanerFactory, and maybe CleanerImpl, into their own internal package, so that it can be used as a JDK platform Cleaner ( rather than a base module only cleaner, or having to qualify exports of jdk.internal.misc to other JDK modules ) ? Is it possible of the NIO Buffers to use this Cleaner? -Chris. From Roger.Riggs at Oracle.com Tue Jan 5 21:16:08 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jan 2016 16:16:08 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568C0F3C.2000906@oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> Message-ID: <568C3298.7030200@Oracle.com> Hi Daniel, webrev updated to revert changes to MeteredStream as too risky. http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html On 1/5/2016 1:45 PM, Daniel Fuchs wrote: > Hi Roger, > > Some early feedback: > > I see that prior to your changes, MeteredStream.close() would > be called by finalize. > This will no longer be the case after > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html In the case where finalize would call close(), there can be no queuedForCleanup activity because there can be no strong reference from the KeepAliveStreamCleaner thread or the queue that holds the pending cleanups. It cannot be in the Cache of connections being kept alive or the thread that keeps them alive. It might be possible that the underlying connection is still open; but there is no advantage of trying to drain it. There is the possibility of resurrecting it by virtue of deciding that it should be queued for cleanup and starting a new thread to do the cleanup. All in all, there is a risk of misunderstanding the dynamic behavior and it would be safer to leave this using finalize. > > I see that MeteredStream has a subclass (KeepAliveStream) that > overrides close() - so KeepAliveStream probably requires a cleanup > as well - doesn't it? > > Finally I'd suggest making the variable 'pi' final - since you're > passing that to the lambda you don't want to allow its value to > be modified afterwards (fortunately it's not - and I believe it > should be good to ensure it won't be). ok, it would not go unnoticed, the compiler should complain if the variable is no effectively final. Thanks, Roger > > I haven't looked at the other classes yet... > > best regards, > > -- daniel > > On 05/01/16 19:24, Roger Riggs wrote: >> The follow on work to adding the Cleaner is to replace uses of >> finalization with uses of the Cleaner. >> For the 'easy' cases in the java.base module, it is useful to introduce >> a private Cleaner for the >> java.base module. It is proposed to be held weakly, to allow it to >> terminate on a lightly loaded >> system. >> >> Webrev for Review: >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >> >> >> The 2nd step is using the Cleaner. >> - Empty finalize methods should (I think) be removed; but since they >> are part of the public spec >> the process needs two full releases; so the proposal is to deprecate >> them first. >> (The JEP 277 necessary changes will be updated when JEP 277 >> semantics are finalized) >> >> - In a few cases, the code in the finalizer is redundant and if removed >> would allow >> an optimization of the handling of the finalizer and future removal >> of the finalize method. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >> >> Thanks for comments and suggestions, Roger >> >> >> >> > From joel.borggren.franck at gmail.com Tue Jan 5 21:16:56 2016 From: joel.borggren.franck at gmail.com (=?UTF-8?Q?Joel_Borggr=C3=A9n=2DFranck?=) Date: Tue, 5 Jan 2016 22:16:56 +0100 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: Message-ID: Hi Liam, I think you need to do skipMemberValue the correct number of times so that the cursor in buf is correct, or? I modified your test slightly to provoke an AnnotationTypeMismatchException that I think shouldn't be there: @AnnotationAnnotation({@ClassArrayAnnotation({Missing.class, String.class}), @ClassArrayAnnotation({String.class})}) public class F {} and @Retention(RUNTIME) public @interface AnnotationAnnotation { ClassArrayAnnotation[] value(); } giving: java.lang.RuntimeException: java.lang.annotation.AnnotationTypeMismatchException: Incorrectly typed data found for annotation element public abstract ClassArrayAnnotation[] AnnotationAnnotation.value() (Found data of type Array with component tag: 99) Also, in MissingArrayElementTest, why not just rethrow t? 52 if (t.getClass() != EnumConstantNotPresentException.class) { 53 System.err.printf( 54 "expected %s, saw %s\n", EnumConstantNotPresentException.class, t); 55 throw new RuntimeException(t); 56 } cheers /Joel On Mon, Dec 21, 2015 at 9:42 PM, Liam Miller-Cushon wrote: > If an annotation value is an array of class literals or enum constants, > calling Class.getAnnotation() fails with ArrayStoreException if the element > type is not found. > > This patch implements the proposed fix from the bug thread, which allows > getAnnotation() to succeed and a TypeNotPresentException to be thrown from > Annotation.value(). > > bug: https://bugs.openjdk.java.net/browse/JDK-7183985 > > The patch is attached. From Roger.Riggs at Oracle.com Tue Jan 5 21:24:35 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 5 Jan 2016 16:24:35 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <06A536F9-13AC-4A82-9FFB-129086494CFC@oracle.com> References: <568C0A5C.7090204@Oracle.com> <06A536F9-13AC-4A82-9FFB-129086494CFC@oracle.com> Message-ID: <568C3493.10806@Oracle.com> Hi Chris, On 1/5/2016 2:33 PM, Chris Hegarty wrote: > On 5 Jan 2016, at 18:24, Roger Riggs wrote: > >> The follow on work to adding the Cleaner is to replace uses of finalization with uses of the Cleaner. >> For the 'easy' cases in the java.base module, it is useful to introduce a private Cleaner for the >> java.base module. It is proposed to be held weakly, to allow it to terminate on a lightly loaded >> system. >> >> Webrev for Review: >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ > Looks ok Roger. Does it make sense to put CleanerFactory, and maybe > CleanerImpl, into their own internal package, so that it can be used as a > JDK platform Cleaner ( rather than a base module only cleaner, or having > to qualify exports of jdk.internal.misc to other JDK modules ) ? That could be useful; but it raises the question about the right/good granularity. It might make sense that the desktop module might reasonably share the Cleaner. Is there any comparable functionality/package already identified? > > Is it possible of the NIO Buffers to use this Cleaner? I did prototype using the java.lang.ref.Cleaner instead of sun.misc.Cleaner and it didn't work so well. Some of the tests failed with out of memory exceptions. I suspected that the fast path through the reference processing for the Cleaner was managing to free memory more quickly than was possible with the new Cleaner. Roger > > -Chris. From xueming.shen at oracle.com Tue Jan 5 22:10:14 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 05 Jan 2016 14:10:14 -0800 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> Message-ID: <568C3F46.3040801@oracle.com> Yuji, I'm not convinced that the ZipCryption is a public interface we'd like to expose, at least for now, given the proprietary nature of the "strong encryption" defined by PKWARE. As I said in my previous email, it might be desired to hide the "passwd/encryption" support for the "traditional zip encryption" as an implementation detail. Looking at the existing methods that deal with "entry" in ZipFile, ZipInputStream and ZipOutputStream, ZipFile.getInputStream(ZipEntry e); ZipOutputStream.putNextEntry(ZipEntry e); ZipInputStream.getNextEntry(); it appears that instead of adding "password" specific method to these classes directly, it might be more appropriate to extend the ZipEntry class for such "password" functionality. For example, with a pair of new methods boolean ZipEntry.isTraditionalEncryption(). void ZipEntry.setTraditionalEncryption(String password); The encryption support should/can be added naturally/smoothly with ZipFile.getInputStream(e), ZipInputstream and ZipOutputStream.putNextEntry(e), with no extra new method in these two classes. The implementation checks the flag (bit0, no bit 6) first and then verifies the password, as an implementation details. For ZipFile and ZipInputStream, we can add note to the api doc to force the invoker to check if the returned ZipEntry indicates it's an encrypted entry. If yes, it must to set the appropriate password to the returned ZipEntry via ZipEntry.setTraditionalEncryption(password); before reading any byte from the input stream. Again, we should not have any "encryption" related public field/method in DeflaterOutputStream/InflaterInputStream. Ideally these two classes really should not be aware of "encryption" at all. -Sherman On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: > Hi Sherman and all, > > Happy new year to everyone! > > Please let know your feedback about this proposal. :-) > > Thanks, > Yuji > > 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >> Hi Sherman, >> >> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>> It is no longer necessary to touch the native code (zip_util.c/h) after the >>> native ZipFile implementation has been moved up to the java level. Those >>> native code are for vm access only now, which I dont think care about the >>> password support at all. >> Thanks for your information. We do not take care the native. >> >> I discussed with Yasumasa, and our thought is as below. >> >>> (1) what's the benefit of exposing the public interface ZipCryption? the real >>> question is whether or not this interface is good enough for other encryption >>> implementation to plugin their implementation to support the ZipFile/Input/ >>> OutputStream to their encryption spec. >> We aimed that the public interface ZipCryption supports the >> extensibillity for other encrypt engine. The JDK core libs developers >> have to implementation ZipyCryption only. If not provide, the JDK >> developers must implement ZipStream/Entry by JDK API to design the >> data structure of entry. >> If you want to use binary key data such as PKI, you can implement new >> encrypt/decrypt engine by ZipCryption interface. >> So we think we should provide this interface to be clearly how to >> implement a new engine, e.g., cipher algorithm, cipher strength and >> converting the header, etc. >> >>> (2) it seems like it might be possible to hide most of the implementation >>> and only expose the "String password" (instead of the ZipCryption) as the >>> public interface to support the "traditional" encryption. This depends on the >>> result of (1) though. >> Thanks for your clues. We think the string password at first. However, >> we should also create a new binary interface given we support PKI in >> the future. >> >>> (3) I'm concerned of pushing ZipCryption into InflaterInputStream/DeflaterOutputStream. >>> It might be worth considering to replace the ZipCryption implementation with >>> a pair of FilterOutput/InputStream. It would be easy and reasonable to use >>> the FilterOutputStream for the ZipOutputStream and the FilterInputStream for the >>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue ... >> Thanks for your clues, too. Honestly speaking, we think the current >> zip implementation may break the data when used PushbackInputStream >> for the following reasons. >> >> * PushbackInputStream uses an unique internal buffer for re-read operation. >> * But, InflaterInputStream provide date to Inflater per reads and >> buffer by JNI (zlib). >> * So we think PushbackInputStream is poor compatibility with >> InflaterInputStream. >> >> We generally use InputStream through ZipEntry#getInputStream(). We do >> not touch FileInputStream for reading ZIP data. If we call unread() >> when we use PushbackInputStream as reading ZIP archive, we guess that >> it will break the reading data. >> So, our approach do not affect the PushbackInputStream. >> What do you think about this? >> >>> (4) It seems the ZipOutputStream only supports the "stream based" password, while >>> the ZipInputStream supports the "entry based" password. Do we really need >>> "entry based" support here? >> As your suggestion, we should support "entry based". We will start to >> implement "entry based" after this discussion is closed. >> >> Thanks, >> Yuji >> >>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>> Hi Jason, >>>> >>>> Thank you for your comment. >>>> I've fixed it in new webrev: >>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>> >>>> >>>> Thanks, >>>> >>>> Yasumasa >>>> >>>> >>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is in >>>>> conflict with line 350 and 348. >>>>> >>>>> ________________________________________ >>>>> From: core-libs-dev on behalf of >>>>> Yasumasa Suenaga >>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>> To: Sergey Bylokhov; Xueming Shen >>>>> Cc: core-libs-dev at openjdk.java.net >>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>> protection to Zip entries >>>>> >>>>> Hi Sergey, >>>>> >>>>> Thank you for your comment. >>>>> >>>>> I added that description in new webrev: >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>> Should the new methods describe how they will work in case of null >>>>>> params? >>>>>> >>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>> >>>>>>> Could you review it? >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>> Hi Sherman, >>>>>>>> >>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>> Thus we will create a new webrev for current (after 8142508) jdk9/dev >>>>>>>> repos. >>>>>>>> >>>>>>>> Do you have any comments about current webrev? >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>> >>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>> Hi Sherman, >>>>>>>>> >>>>>>>>> Thanks for your quick response :) >>>>>>>>> >>>>>>>>> I aimed to implement the "traditional" at this proposal by the below >>>>>>>>> reasons. >>>>>>>>> >>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>> * Many people use the "traditional" in problem-free scope like a >>>>>>>>> temporary file. >>>>>>>>> * We do not know which implementation of the "stronger" is best >>>>>>>>> for >>>>>>>>> openjdk. >>>>>>>>> * PKWare claims that they have patents about the "stronger" on >>>>>>>>> Zip[1]. >>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>> "stronger" [2][3]. >>>>>>>>> * Instead, we prepared the extensibility by ZipCryption interface >>>>>>>>> to >>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>> >>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>> In the future, anyone who want to implement the "stronger" can easily >>>>>>>>> add their code by virtue of this proposal. >>>>>>>>> >>>>>>>>> [1] https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption Specification) >>>>>>>>> [2] >>>>>>>>> >>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>> >>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Yuji >>>>>>>>> >>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming Shen: >>>>>>>>>> >>>>>>>>>> Hi Yuji, >>>>>>>>>> >>>>>>>>>> I will take a look at your PoC. Might need some time and even bring >>>>>>>>>> in the >>>>>>>>>> security guy >>>>>>>>>> to evaluate the proposal. It seems like you are only interested in >>>>>>>>>> the >>>>>>>>>> "traditional PKWare >>>>>>>>>> decryption", which is, based on the wiki, "known to be seriously >>>>>>>>>> flawed, and >>>>>>>>>> in particular >>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to support >>>>>>>>>> "stronger" encryption >>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>> >>>>>>>>>> Regards, >>>>>>>>>> Sherman >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>> >>>>>>>>>>> Hi all, >>>>>>>>>>> >>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Yuji >>>>>>>>>>> >>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent before >>>>>>>>>>>> get >>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>> >>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>> However, >>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we must use >>>>>>>>>>>> third party library so far. >>>>>>>>>>>> >>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>> >>>>>>>>>>>> The current zlib in JDK is completely unaffected by this proposal. >>>>>>>>>>>> The >>>>>>>>>>>> traditional zip encryption encrypts a data after it is has been >>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing zlib >>>>>>>>>>>> implementation. >>>>>>>>>>>> >>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> Test code is as below. This code will let you know how this >>>>>>>>>>>> PoC >>>>>>>>>>>> works. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>> >>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are providing >>>>>>>>>>>> many >>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>> implement to >>>>>>>>>>>> handle password-protected zip file. I guess that this proposal is >>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>> >>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>> >>>>>>>>>>>> [1]: https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Yuji >>>>>>>>>> >>>>>>>>>> >>>>>> From david.holmes at oracle.com Wed Jan 6 04:20:05 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 6 Jan 2016 14:20:05 +1000 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568A7B7A.5060001@oracle.com> References: <568A7B7A.5060001@oracle.com> Message-ID: <568C95F5.4040101@oracle.com> Hi Chris, Hotspot comment change looks okay.:) I see a lot of hotspot tests that include @modules java.base/sun.misc but I don't understand why it is present in the few cases I looked at eg: hotspot/test/gc/g1/TestShrinkAuxiliaryData15.java so not sure when it needs to be converted to jdk.internal.misc Thanks, David On 5/01/2016 12:02 AM, Chris Hegarty wrote: > sun.misc.VM provides a low-level interface for a small number > of specific operations with the VM. In preparation for JEP 260, > this class should be moved out of sun.misc and located in a > non-exported package [. > > http://cr.openjdk.java.net/~chegar/8145544/00/ > > Note: as in other areas some tests that require access to > internal APIs have been updated to use types from a more > stable package, sun.security.x509. > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8145544 From chris.hegarty at oracle.com Wed Jan 6 05:28:42 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 6 Jan 2016 05:28:42 +0000 Subject: RFR [9] 8146484: Examine sun.misc.MessageUtils Message-ID: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> sun.misc.MessageUtils provides low-level access for writing ?error? messages to stdout and stderr. It is, effectively, used in just one place in the JDK, String coding, when the specific encoding is not available. Since this is the only usage, the minimal functionality required by String coding can be moved to StringCoding, and MessageUtils removed. http://cr.openjdk.java.net/~chegar/8146484/ Note: there are a couple of unused reference to MessageUtils in the desktop module and a build tool that need to be cleaned up. -Chris. From chris.hegarty at oracle.com Wed Jan 6 06:10:39 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 6 Jan 2016 06:10:39 +0000 Subject: RFR [9] 8145544: Move sun.misc.VM to jdk.internal.misc In-Reply-To: <568C95F5.4040101@oracle.com> References: <568A7B7A.5060001@oracle.com> <568C95F5.4040101@oracle.com> Message-ID: <42EE23DC-1CD2-4816-949A-0EC4875909F5@oracle.com> On 6 Jan 2016, at 04:20, David Holmes wrote: > Hi Chris, > > Hotspot comment change looks okay.:) Thanks David. > I see a lot of hotspot tests that include > > @modules java.base/sun.misc > > but I don't understand why it is present in the few cases I looked at eg: > > hotspot/test/gc/g1/TestShrinkAuxiliaryData15.java The dependency on sun.misc is coming from the test library, specifically jdk.test.lib.Utils. Quite a few tests in the hotspot repo use the test library, so have this dependency ( on sun.misc.Unsafe ). > so not sure when it needs to be converted to jdk.internal.misc It is on my list to update these tests, see 8140608 [1]. I punted doing this with the Unsafe move, as I was unhappy with the test coverage of sun.misc.Unsafe. Paul has since added some additional tests [2] that have variants that cover both jdk.internal.misc.Unsafe and sun.misc.Unsafe, so I think the hotspot tests can now be updated to only use jdk.internal.misc. I?ll move 8140608 forward over the next few weeks. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8140608 [2] http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/rev/d84bd22ab531 > Thanks, > David > > On 5/01/2016 12:02 AM, Chris Hegarty wrote: >> sun.misc.VM provides a low-level interface for a small number >> of specific operations with the VM. In preparation for JEP 260, >> this class should be moved out of sun.misc and located in a >> non-exported package [. >> >> http://cr.openjdk.java.net/~chegar/8145544/00/ >> >> Note: as in other areas some tests that require access to >> internal APIs have been updated to use types from a more >> stable package, sun.security.x509. >> >> -Chris. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8145544 From xueming.shen at oracle.com Wed Jan 6 06:16:39 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 5 Jan 2016 22:16:39 -0800 Subject: RFR [9] 8146484: Examine sun.misc.MessageUtils In-Reply-To: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> References: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> Message-ID: <568CB147.8090508@oracle.com> Looks good. On 1/5/16 9:28 PM, Chris Hegarty wrote: > sun.misc.MessageUtils provides low-level access for writing ?error? messages > to stdout and stderr. It is, effectively, used in just one place in the JDK, String > coding, when the specific encoding is not available. Since this is the only usage, > the minimal functionality required by String coding can be moved to StringCoding, > and MessageUtils removed. > > http://cr.openjdk.java.net/~chegar/8146484/ > > Note: there are a couple of unused reference to MessageUtils in the desktop > module and a build tool that need to be cleaned up. > > -Chris. From Alan.Bateman at oracle.com Wed Jan 6 10:12:48 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jan 2016 10:12:48 +0000 Subject: RFR [9] 8146484: Examine sun.misc.MessageUtils In-Reply-To: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> References: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> Message-ID: <568CE8A0.5080400@oracle.com> On 06/01/2016 05:28, Chris Hegarty wrote: > sun.misc.MessageUtils provides low-level access for writing ?error? messages > to stdout and stderr. It is, effectively, used in just one place in the JDK, String > coding, when the specific encoding is not available. Since this is the only usage, > the minimal functionality required by String coding can be moved to StringCoding, > and MessageUtils removed. > > http://cr.openjdk.java.net/~chegar/8146484/ > > Note: there are a couple of unused reference to MessageUtils in the desktop > module and a build tool that need to be cleaned up. > This looks to me. I initially assumed the resource files in sun.misc.resources.* were used by MessageUtils but it seems not, something for another bug I think. -Alan From scolebourne at joda.org Wed Jan 6 10:27:22 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 6 Jan 2016 10:27:22 +0000 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: <568C3F46.3040801@oracle.com> References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> Message-ID: Just to note that I once needed this and even commented on Stack Overflow about it: http://stackoverflow.com/questions/166340/write-a-password-protected-zip-file-in-java I'd just note that setting the password on each entry is a bit painful, as you'd normally expect to just set the password once. But having the feature in the JDK is better than not. Having pluggable encryption will I suspect be useful though, because the encryption used will no doubt change over time. Stephen On 5 January 2016 at 22:10, Xueming Shen wrote: > Yuji, > > I'm not convinced that the ZipCryption is a public interface we'd like to > expose, > at least for now, given the proprietary nature of the "strong encryption" > defined > by PKWARE. > > As I said in my previous email, it might be desired to hide the > "passwd/encryption" > support for the "traditional zip encryption" as an implementation detail. > > Looking at the existing methods that deal with "entry" in ZipFile, > ZipInputStream > and ZipOutputStream, > > ZipFile.getInputStream(ZipEntry e); > ZipOutputStream.putNextEntry(ZipEntry e); > ZipInputStream.getNextEntry(); > > it appears that instead of adding "password" specific method to these > classes > directly, it might be more appropriate to extend the ZipEntry class for > such > "password" functionality. For example, with a pair of new methods > > boolean ZipEntry.isTraditionalEncryption(). > void ZipEntry.setTraditionalEncryption(String password); > > The encryption support should/can be added naturally/smoothly with > ZipFile.getInputStream(e), ZipInputstream and > ZipOutputStream.putNextEntry(e), > with no extra new method in these two classes. The implementation checks > the flag (bit0, no bit 6) first and then verifies the password, as an > implementation > details. > > For ZipFile and ZipInputStream, we can add note to the api doc to force the > invoker to check if the returned ZipEntry indicates it's an encrypted > entry. If yes, > it must to set the appropriate password to the returned ZipEntry via > ZipEntry.setTraditionalEncryption(password); before reading any byte from > the > input stream. > > Again, we should not have any "encryption" related public field/method in > DeflaterOutputStream/InflaterInputStream. Ideally these two classes really > should > not be aware of "encryption" at all. > > -Sherman > > > > On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: > >> Hi Sherman and all, >> >> Happy new year to everyone! >> >> Please let know your feedback about this proposal. :-) >> >> Thanks, >> Yuji >> >> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >> >>> Hi Sherman, >>> >>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>> >>>> It is no longer necessary to touch the native code (zip_util.c/h) after >>>> the >>>> native ZipFile implementation has been moved up to the java level. Those >>>> native code are for vm access only now, which I dont think care about >>>> the >>>> password support at all. >>>> >>> Thanks for your information. We do not take care the native. >>> >>> I discussed with Yasumasa, and our thought is as below. >>> >>> (1) what's the benefit of exposing the public interface ZipCryption? the >>>> real >>>> question is whether or not this interface is good enough for other >>>> encryption >>>> implementation to plugin their implementation to support the >>>> ZipFile/Input/ >>>> OutputStream to their encryption spec. >>>> >>> We aimed that the public interface ZipCryption supports the >>> extensibillity for other encrypt engine. The JDK core libs developers >>> have to implementation ZipyCryption only. If not provide, the JDK >>> developers must implement ZipStream/Entry by JDK API to design the >>> data structure of entry. >>> If you want to use binary key data such as PKI, you can implement new >>> encrypt/decrypt engine by ZipCryption interface. >>> So we think we should provide this interface to be clearly how to >>> implement a new engine, e.g., cipher algorithm, cipher strength and >>> converting the header, etc. >>> >>> (2) it seems like it might be possible to hide most of the implementation >>>> and only expose the "String password" (instead of the ZipCryption) as >>>> the >>>> public interface to support the "traditional" encryption. This depends >>>> on the >>>> result of (1) though. >>>> >>> Thanks for your clues. We think the string password at first. However, >>> we should also create a new binary interface given we support PKI in >>> the future. >>> >>> (3) I'm concerned of pushing ZipCryption into >>>> InflaterInputStream/DeflaterOutputStream. >>>> It might be worth considering to replace the ZipCryption implementation >>>> with >>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>> use >>>> the FilterOutputStream for the ZipOutputStream and the >>>> FilterInputStream for the >>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue ... >>>> >>> Thanks for your clues, too. Honestly speaking, we think the current >>> zip implementation may break the data when used PushbackInputStream >>> for the following reasons. >>> >>> * PushbackInputStream uses an unique internal buffer for re-read >>> operation. >>> * But, InflaterInputStream provide date to Inflater per reads and >>> buffer by JNI (zlib). >>> * So we think PushbackInputStream is poor compatibility with >>> InflaterInputStream. >>> >>> We generally use InputStream through ZipEntry#getInputStream(). We do >>> not touch FileInputStream for reading ZIP data. If we call unread() >>> when we use PushbackInputStream as reading ZIP archive, we guess that >>> it will break the reading data. >>> So, our approach do not affect the PushbackInputStream. >>> What do you think about this? >>> >>> (4) It seems the ZipOutputStream only supports the "stream based" >>>> password, while >>>> the ZipInputStream supports the "entry based" password. Do we really >>>> need >>>> "entry based" support here? >>>> >>> As your suggestion, we should support "entry based". We will start to >>> implement "entry based" after this discussion is closed. >>> >>> Thanks, >>> Yuji >>> >>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>> >>>>> Hi Jason, >>>>> >>>>> Thank you for your comment. >>>>> I've fixed it in new webrev: >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>> >>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is in >>>>>> conflict with line 350 and 348. >>>>>> >>>>>> ________________________________________ >>>>>> From: core-libs-dev on >>>>>> behalf of >>>>>> Yasumasa Suenaga >>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>> protection to Zip entries >>>>>> >>>>>> Hi Sergey, >>>>>> >>>>>> Thank you for your comment. >>>>>> >>>>>> I added that description in new webrev: >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>> >>>>>>> Should the new methods describe how they will work in case of null >>>>>>> params? >>>>>>> >>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>> >>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>> >>>>>>>> Could you review it? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>>> Hi Sherman, >>>>>>>>> >>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>> jdk9/dev >>>>>>>>> repos. >>>>>>>>> >>>>>>>>> Do you have any comments about current webrev? >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>> >>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>> >>>>>>>>>> Hi Sherman, >>>>>>>>>> >>>>>>>>>> Thanks for your quick response :) >>>>>>>>>> >>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>> below >>>>>>>>>> reasons. >>>>>>>>>> >>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>> like a >>>>>>>>>> temporary file. >>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>> best >>>>>>>>>> for >>>>>>>>>> openjdk. >>>>>>>>>> * PKWare claims that they have patents about the "stronger" >>>>>>>>>> on >>>>>>>>>> Zip[1]. >>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>> "stronger" [2][3]. >>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>> interface >>>>>>>>>> to >>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>> >>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>> easily >>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>> >>>>>>>>>> [1] https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption Specification) >>>>>>>>>> >>>>>>>>>> [2] >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>> >>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Yuji >>>>>>>>>> >>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming Shen: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Yuji, >>>>>>>>>>> >>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>> bring >>>>>>>>>>> in the >>>>>>>>>>> security guy >>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>> in >>>>>>>>>>> the >>>>>>>>>>> "traditional PKWare >>>>>>>>>>> decryption", which is, based on the wiki, "known to be seriously >>>>>>>>>>> flawed, and >>>>>>>>>>> in particular >>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>> support >>>>>>>>>>> "stronger" encryption >>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Sherman >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Yuji >>>>>>>>>>>> >>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>> before >>>>>>>>>>>>> get >>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>> >>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>> However, >>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we must >>>>>>>>>>>>> use >>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>> >>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>> >>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>> proposal. >>>>>>>>>>>>> The >>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has been >>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>> zlib >>>>>>>>>>>>> implementation. >>>>>>>>>>>>> >>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Test code is as below. This code will let you know how >>>>>>>>>>>>> this >>>>>>>>>>>>> PoC >>>>>>>>>>>>> works. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>> >>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are providing >>>>>>>>>>>>> many >>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>> implement to >>>>>>>>>>>>> handle password-protected zip file. I guess that this proposal >>>>>>>>>>>>> is >>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>> >>>>>>>>>>>>> [1]: >>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Yuji >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>> > From chris.hegarty at oracle.com Wed Jan 6 10:37:42 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 6 Jan 2016 10:37:42 +0000 Subject: RFR [9] 8146484: Examine sun.misc.MessageUtils In-Reply-To: <568CE8A0.5080400@oracle.com> References: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> <568CE8A0.5080400@oracle.com> Message-ID: <568CEE76.4010401@oracle.com> On 06/01/16 10:12, Alan Bateman wrote: > On 06/01/2016 05:28, Chris Hegarty wrote: >> sun.misc.MessageUtils provides low-level access for writing ?error? >> messages >> to stdout and stderr. It is, effectively, used in just one place in >> the JDK, String >> coding, when the specific encoding is not available. Since this is the >> only usage, >> the minimal functionality required by String coding can be moved to >> StringCoding, >> and MessageUtils removed. >> >> http://cr.openjdk.java.net/~chegar/8146484/ >> >> Note: there are a couple of unused reference to MessageUtils in the >> desktop >> module and a build tool that need to be cleaned up. >> > This looks to me. Thanks Alan. > I initially assumed the resource files in > sun.misc.resources.* were used by MessageUtils but it seems not, > something for another bug I think. Yes, they are on the hit list too :-) -Chris. From scolebourne at joda.org Wed Jan 6 11:48:17 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 6 Jan 2016 11:48:17 +0000 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <1461285505.20151226222913@gmail.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> Message-ID: Thanks for the patch. The docs say that if the end date is before the start date, the stream is empty. While I can see just about why LongStream.range() works that way, I don't think this API should. The minimum is an exception, but it would be easy to support negative in the days-only case or the months-only case. The problem is where there are both months/years and days and those should be exceptions. The single-arg implementation uses plusDays() with an incrementing number. When the performance patch goes in, the proposed streaming implementation will be optimal for small streams but less optimal for large ones. The fastest way to loop over a list of dates would be to manually generate them by incrementing the day until it exceeds the length of month, and so on. However, this would be serial. As such, I think the best way to write this, taking account of how plusDays() is implemented, is as follows: LongStream.range(start.toEpochDay(), end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); This ensures that toEpochDay() is only called twice, rather than almost every time (within plusDays). In the period-based method, it would be best to capture the case where totalMonths == 0 and days > 0 and forward to another method that ignores months. That private method can share implementation with the public single-arg method (passing in 1). The docs for the period-based method should specify the strategy that produces the results (in abstract terms). This needs to cover that the result is equivalent to always adding to the start period a multiple of the period. I haven't looked at the maths in detail at this point. Hopefully the test cases cover it. Some nits: I prefer to avoid @link in the first sentence. Just using 'stream' would be sufficient. The first sentence should be a summary. In this case it probably has a bit too much detail. The @return has 'values' on a new line when it could be on the same line. If statements need braces. thanks Stephen On 26 December 2015 at 16:29, Tagir F. Valeev wrote: > Hello! > > Thank you for your comments. I logged an issue: > https://bugs.openjdk.java.net/browse/JDK-8146218 > > And prepared a patch: > http://amaembo.github.io/cr/8146218/webrev-1/ > (sorry for not using official code-review server, I cannot access it > currently, hopefully this problem will be solved soon). Please review, > especially the specification: English is not my native language. > > The patch introduces two methods in LocalDate class: > datesUntil(LocalDate endExclusive) and > datesUntil(LocalDate endExclusive, Period step). > > The implementation of the former one is just a one-liner. I decided to > produce an empty stream if endExclusive is before than this to stay in > sync with IntStream.range/LongStream.range. > > The latter method is more tricky. It could be implemented using the > takeWhile() method, but I wanted to make the sized stream, so it can > parallelize perfectly and its count method is guaranteed to work in > constant time (see JDK-8067969). Thus it's necessary to precompute > exactly the size of the stream. > > Zero and negative periods are not supported. On the one hand it might > be useful in some scenarios (assuming the ending date is before the > start date). On the other hand it's possible to define some weird > period like Period.of(0,1,-30) which may go in both directions > depending on current date. Period like Period.of(400,0,-146097) is > equivalent to ZERO, so it would never advance, but isZero would return > false (and even normalization will not help). Theoretically it's > possible to handle all such cases, but it would make the method and > its specification much more complex. Thus I would like to disable > negative periods at all. Though I'm open to discussion. > > Probably I added too many test cases. If it poses the problem I may > try to reduce the number of tests. > > With best regards, > Tagir Valeev. > > SC> JSR-310 was developed in parallel with the Stream API. While we > SC> briefly considered adding some stream methods, it wasn't obvious what > SC> would be of value. > > SC> A key question is whether the methods are either specific to a narrow > SC> use case or more general. Specific methods like months() on Year are > SC> very limited. You'd need: > SC> Year::months() > SC> Year::dates() > SC> YearMonth::dates() > SC> Only three methods, so not too bad. There would be some use cases, but > SC> enough to include them? Not sure. > > SC> A method that provides all days between two dates: > SC> Stream datesUntil(LocalDate endExclusive); > SC> would have some value as it is more common and trickier to write. > > SC> A method that provides Period addition looping might make more sense: > SC> Stream datesUntil(LocalDate endExclusive, Period step); > SC> This is actually quite hard to write correctly, as it requires > SC> multiplying the period by the loop index, not adding the period each > SC> time (to handle end-of-month effects). > > SC> But where to stop? > SC> Stream instantsUntil(Instant endExclusive, Duration step); > SC> Stream instantsUntil(Instant endExclusive, Duration > step); > SC> Stream instantsUntil(Instant endExclusive, Duration > step); > SC> ... > > > SC> I think the place to start is the two methods on LocalDate which are > SC> tricky enough to be justified. Then perhaps the methods on > SC> Year/YearMonth. > > SC> Stephen > > > SC> On 10 December 2015 at 16:31, Tagir F. Valeev > wrote: > >> - in class Year: > >> > >> // Returns sequential ordered stream of all months within this Year: > >> Stream months(); > >> // Returns sequential ordered stream of all days within this Year: > >> Stream days(); > >> // Returns sequential ordered stream of all years starting from this > >> // Year until the supplied year, exclusive > >> Stream yearsUntil(Temporal endExclusive); > >> > >> - in class YearMonth: > >> > >> // Returns sequential ordered stream of all days within this YearMonth: > >> Stream days(); > >> // Returns sequential ordered stream of all months starting from this > >> // YearMonth until the supplied YearMonth, exclusive > >> Stream monthsUntil(Temporal endExclusive); > >> > >> - in class LocalDate: > >> > >> // Returns sequential ordered stream of all months starting from this > >> // LocalDate until the supplied LocalDate, exclusive > >> Stream daysUntil(Temporal endExclusive); > >> > >> The implementation of these methods could be quite simple. For example: > >> > >> class Year { > >> public Stream days() { > >> return IntStream.rangeClosed(1, length()).mapToObj(this::atDay); > >> } > >> } > >> > >> What do you think? > >> > >> With best regards, > >> Tagir Valeev. > >> > > From kubota.yuji at gmail.com Wed Jan 6 13:25:51 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Wed, 6 Jan 2016 22:25:51 +0900 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: <568C3F46.3040801@oracle.com> References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> Message-ID: Hi Sherman, Thank you for your feedback! At first, I'd like to confirm your thought about ZipCryption to prevent misunderstand. By following reasons, we should remove (or modified) ZipCryption to not have any "encryption" related field/method in DeflaterOutputStream / InflaterInputStream. 1. For security, to hide "passwd/encryption" support by not just adding "password" specific method. 2. To avoid that JDK API developers implement "passwd/encryption" through ZipCryption. Is it right? In my previous email, I thought we should have the public interface to provide scalability for supporting any open-source encryption implementation such as WinZip AES. If the reason is right, your suggestion sounds reasonable to us as below. 2016-01-06 7:10 GMT+09:00 Xueming Shen : > it appears that instead of adding "password" specific method to these > classes directly, it might be more appropriate to extend the ZipEntry class > for such "password" functionality. For example, with a pair of new methods > > boolean ZipEntry.isTraditionalEncryption(). > void ZipEntry.setTraditionalEncryption(String password); Thanks advice, I agree. We should re-design the API to extend the ZipEntry class. > The encryption support should/can be added naturally/smoothly with > ZipFile.getInputStream(e), ZipInputstream and > ZipOutputStream.putNextEntry(e), > with no extra new method in these two classes. The implementation checks > the flag (bit0, no bit 6) first and then verifies the password, as an > implementation details. Agree. For this proposal, we aim to support only traditional encryption. So I think we should also check bit 6. > For ZipFile and ZipInputStream, we can add note to the api doc to force the > invoker to check if the returned ZipEntry indicates it's an encrypted entry. > If yes, it must to set the appropriate password to the returned ZipEntry via > ZipEntry.setTraditionalEncryption(password); before reading any byte from > the input stream. Yes, we have to add note the flow of codes to the JavaDoc. > Again, we should not have any "encryption" related public field/method in > DeflaterOutputStream/InflaterInputStream. Ideally these two classes really > should not be aware of "encryption" at all. Agree, but I think we might be faced technical difficulty about a processing between zlib and the internal buffer of InflaterInputStream / DeflaterOutputStream. Please give us time to implement. > > -Sherman Thanks, Yuji > > On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: >> >> Hi Sherman and all, >> >> Happy new year to everyone! >> >> Please let know your feedback about this proposal. :-) >> >> Thanks, >> Yuji >> >> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >>> >>> Hi Sherman, >>> >>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>>> >>>> It is no longer necessary to touch the native code (zip_util.c/h) after >>>> the >>>> native ZipFile implementation has been moved up to the java level. Those >>>> native code are for vm access only now, which I dont think care about >>>> the >>>> password support at all. >>> >>> Thanks for your information. We do not take care the native. >>> >>> I discussed with Yasumasa, and our thought is as below. >>> >>>> (1) what's the benefit of exposing the public interface ZipCryption? the >>>> real >>>> question is whether or not this interface is good enough for other >>>> encryption >>>> implementation to plugin their implementation to support the >>>> ZipFile/Input/ >>>> OutputStream to their encryption spec. >>> >>> We aimed that the public interface ZipCryption supports the >>> extensibillity for other encrypt engine. The JDK core libs developers >>> have to implementation ZipyCryption only. If not provide, the JDK >>> developers must implement ZipStream/Entry by JDK API to design the >>> data structure of entry. >>> If you want to use binary key data such as PKI, you can implement new >>> encrypt/decrypt engine by ZipCryption interface. >>> So we think we should provide this interface to be clearly how to >>> implement a new engine, e.g., cipher algorithm, cipher strength and >>> converting the header, etc. >>> >>>> (2) it seems like it might be possible to hide most of the >>>> implementation >>>> and only expose the "String password" (instead of the ZipCryption) as >>>> the >>>> public interface to support the "traditional" encryption. This depends >>>> on the >>>> result of (1) though. >>> >>> Thanks for your clues. We think the string password at first. However, >>> we should also create a new binary interface given we support PKI in >>> the future. >>> >>>> (3) I'm concerned of pushing ZipCryption into >>>> InflaterInputStream/DeflaterOutputStream. >>>> It might be worth considering to replace the ZipCryption implementation >>>> with >>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>> use >>>> the FilterOutputStream for the ZipOutputStream and the FilterInputStream >>>> for the >>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue ... >>> >>> Thanks for your clues, too. Honestly speaking, we think the current >>> zip implementation may break the data when used PushbackInputStream >>> for the following reasons. >>> >>> * PushbackInputStream uses an unique internal buffer for re-read >>> operation. >>> * But, InflaterInputStream provide date to Inflater per reads and >>> buffer by JNI (zlib). >>> * So we think PushbackInputStream is poor compatibility with >>> InflaterInputStream. >>> >>> We generally use InputStream through ZipEntry#getInputStream(). We do >>> not touch FileInputStream for reading ZIP data. If we call unread() >>> when we use PushbackInputStream as reading ZIP archive, we guess that >>> it will break the reading data. >>> So, our approach do not affect the PushbackInputStream. >>> What do you think about this? >>> >>>> (4) It seems the ZipOutputStream only supports the "stream based" >>>> password, while >>>> the ZipInputStream supports the "entry based" password. Do we really >>>> need >>>> "entry based" support here? >>> >>> As your suggestion, we should support "entry based". We will start to >>> implement "entry based" after this discussion is closed. >>> >>> Thanks, >>> Yuji >>> >>>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>>> >>>>> Hi Jason, >>>>> >>>>> Thank you for your comment. >>>>> I've fixed it in new webrev: >>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> Yasumasa >>>>> >>>>> >>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>>> >>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is in >>>>>> conflict with line 350 and 348. >>>>>> >>>>>> ________________________________________ >>>>>> From: core-libs-dev on behalf >>>>>> of >>>>>> Yasumasa Suenaga >>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>> protection to Zip entries >>>>>> >>>>>> Hi Sergey, >>>>>> >>>>>> Thank you for your comment. >>>>>> >>>>>> I added that description in new webrev: >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>>> >>>>>>> Should the new methods describe how they will work in case of null >>>>>>> params? >>>>>>> >>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>> >>>>>>>> Could you review it? >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> Hi Sherman, >>>>>>>>> >>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>> jdk9/dev >>>>>>>>> repos. >>>>>>>>> >>>>>>>>> Do you have any comments about current webrev? >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>> >>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>>> >>>>>>>>>> Hi Sherman, >>>>>>>>>> >>>>>>>>>> Thanks for your quick response :) >>>>>>>>>> >>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>> below >>>>>>>>>> reasons. >>>>>>>>>> >>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>> like a >>>>>>>>>> temporary file. >>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>> best >>>>>>>>>> for >>>>>>>>>> openjdk. >>>>>>>>>> * PKWare claims that they have patents about the "stronger" >>>>>>>>>> on >>>>>>>>>> Zip[1]. >>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>> "stronger" [2][3]. >>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>> interface >>>>>>>>>> to >>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>> >>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>> easily >>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>> >>>>>>>>>> [1] https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption Specification) >>>>>>>>>> >>>>>>>>>> [2] >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>> >>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Yuji >>>>>>>>>> >>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming Shen: >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Yuji, >>>>>>>>>>> >>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>> bring >>>>>>>>>>> in the >>>>>>>>>>> security guy >>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>> in >>>>>>>>>>> the >>>>>>>>>>> "traditional PKWare >>>>>>>>>>> decryption", which is, based on the wiki, "known to be seriously >>>>>>>>>>> flawed, and >>>>>>>>>>> in particular >>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>> support >>>>>>>>>>> "stronger" encryption >>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>> >>>>>>>>>>> Regards, >>>>>>>>>>> Sherman >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi all, >>>>>>>>>>>> >>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Yuji >>>>>>>>>>>> >>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>> before >>>>>>>>>>>>> get >>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>> >>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>> However, >>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we must >>>>>>>>>>>>> use >>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>> >>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>> >>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>> proposal. >>>>>>>>>>>>> The >>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has been >>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>> zlib >>>>>>>>>>>>> implementation. >>>>>>>>>>>>> >>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> Test code is as below. This code will let you know how >>>>>>>>>>>>> this >>>>>>>>>>>>> PoC >>>>>>>>>>>>> works. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>> >>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are providing >>>>>>>>>>>>> many >>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>> implement to >>>>>>>>>>>>> handle password-protected zip file. I guess that this proposal >>>>>>>>>>>>> is >>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>> >>>>>>>>>>>>> [1]: >>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Yuji >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>> > From kubota.yuji at gmail.com Wed Jan 6 13:45:07 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Wed, 6 Jan 2016 22:45:07 +0900 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> Message-ID: Hi Stephen, Thank you for your note. After discussion about the design, we will re-implement API. Please check it, and give feedback us when the time comes :) Thanks, Yuji 2016-01-06 19:27 GMT+09:00 Stephen Colebourne : > Just to note that I once needed this and even commented on Stack Overflow > about it: > http://stackoverflow.com/questions/166340/write-a-password-protected-zip-file-in-java > > I'd just note that setting the password on each entry is a bit painful, as > you'd normally expect to just set the password once. But having the feature > in the JDK is better than not. Having pluggable encryption will I suspect > be useful though, because the encryption used will no doubt change over > time. > > Stephen > > > On 5 January 2016 at 22:10, Xueming Shen wrote: > >> Yuji, >> >> I'm not convinced that the ZipCryption is a public interface we'd like to >> expose, >> at least for now, given the proprietary nature of the "strong encryption" >> defined >> by PKWARE. >> >> As I said in my previous email, it might be desired to hide the >> "passwd/encryption" >> support for the "traditional zip encryption" as an implementation detail. >> >> Looking at the existing methods that deal with "entry" in ZipFile, >> ZipInputStream >> and ZipOutputStream, >> >> ZipFile.getInputStream(ZipEntry e); >> ZipOutputStream.putNextEntry(ZipEntry e); >> ZipInputStream.getNextEntry(); >> >> it appears that instead of adding "password" specific method to these >> classes >> directly, it might be more appropriate to extend the ZipEntry class for >> such >> "password" functionality. For example, with a pair of new methods >> >> boolean ZipEntry.isTraditionalEncryption(). >> void ZipEntry.setTraditionalEncryption(String password); >> >> The encryption support should/can be added naturally/smoothly with >> ZipFile.getInputStream(e), ZipInputstream and >> ZipOutputStream.putNextEntry(e), >> with no extra new method in these two classes. The implementation checks >> the flag (bit0, no bit 6) first and then verifies the password, as an >> implementation >> details. >> >> For ZipFile and ZipInputStream, we can add note to the api doc to force the >> invoker to check if the returned ZipEntry indicates it's an encrypted >> entry. If yes, >> it must to set the appropriate password to the returned ZipEntry via >> ZipEntry.setTraditionalEncryption(password); before reading any byte from >> the >> input stream. >> >> Again, we should not have any "encryption" related public field/method in >> DeflaterOutputStream/InflaterInputStream. Ideally these two classes really >> should >> not be aware of "encryption" at all. >> >> -Sherman >> >> >> >> On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: >> >>> Hi Sherman and all, >>> >>> Happy new year to everyone! >>> >>> Please let know your feedback about this proposal. :-) >>> >>> Thanks, >>> Yuji >>> >>> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >>> >>>> Hi Sherman, >>>> >>>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>>> >>>>> It is no longer necessary to touch the native code (zip_util.c/h) after >>>>> the >>>>> native ZipFile implementation has been moved up to the java level. Those >>>>> native code are for vm access only now, which I dont think care about >>>>> the >>>>> password support at all. >>>>> >>>> Thanks for your information. We do not take care the native. >>>> >>>> I discussed with Yasumasa, and our thought is as below. >>>> >>>> (1) what's the benefit of exposing the public interface ZipCryption? the >>>>> real >>>>> question is whether or not this interface is good enough for other >>>>> encryption >>>>> implementation to plugin their implementation to support the >>>>> ZipFile/Input/ >>>>> OutputStream to their encryption spec. >>>>> >>>> We aimed that the public interface ZipCryption supports the >>>> extensibillity for other encrypt engine. The JDK core libs developers >>>> have to implementation ZipyCryption only. If not provide, the JDK >>>> developers must implement ZipStream/Entry by JDK API to design the >>>> data structure of entry. >>>> If you want to use binary key data such as PKI, you can implement new >>>> encrypt/decrypt engine by ZipCryption interface. >>>> So we think we should provide this interface to be clearly how to >>>> implement a new engine, e.g., cipher algorithm, cipher strength and >>>> converting the header, etc. >>>> >>>> (2) it seems like it might be possible to hide most of the implementation >>>>> and only expose the "String password" (instead of the ZipCryption) as >>>>> the >>>>> public interface to support the "traditional" encryption. This depends >>>>> on the >>>>> result of (1) though. >>>>> >>>> Thanks for your clues. We think the string password at first. However, >>>> we should also create a new binary interface given we support PKI in >>>> the future. >>>> >>>> (3) I'm concerned of pushing ZipCryption into >>>>> InflaterInputStream/DeflaterOutputStream. >>>>> It might be worth considering to replace the ZipCryption implementation >>>>> with >>>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>>> use >>>>> the FilterOutputStream for the ZipOutputStream and the >>>>> FilterInputStream for the >>>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue ... >>>>> >>>> Thanks for your clues, too. Honestly speaking, we think the current >>>> zip implementation may break the data when used PushbackInputStream >>>> for the following reasons. >>>> >>>> * PushbackInputStream uses an unique internal buffer for re-read >>>> operation. >>>> * But, InflaterInputStream provide date to Inflater per reads and >>>> buffer by JNI (zlib). >>>> * So we think PushbackInputStream is poor compatibility with >>>> InflaterInputStream. >>>> >>>> We generally use InputStream through ZipEntry#getInputStream(). We do >>>> not touch FileInputStream for reading ZIP data. If we call unread() >>>> when we use PushbackInputStream as reading ZIP archive, we guess that >>>> it will break the reading data. >>>> So, our approach do not affect the PushbackInputStream. >>>> What do you think about this? >>>> >>>> (4) It seems the ZipOutputStream only supports the "stream based" >>>>> password, while >>>>> the ZipInputStream supports the "entry based" password. Do we really >>>>> need >>>>> "entry based" support here? >>>>> >>>> As your suggestion, we should support "entry based". We will start to >>>> implement "entry based" after this discussion is closed. >>>> >>>> Thanks, >>>> Yuji >>>> >>>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>>> >>>>>> Hi Jason, >>>>>> >>>>>> Thank you for your comment. >>>>>> I've fixed it in new webrev: >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>>> >>>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is in >>>>>>> conflict with line 350 and 348. >>>>>>> >>>>>>> ________________________________________ >>>>>>> From: core-libs-dev on >>>>>>> behalf of >>>>>>> Yasumasa Suenaga >>>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>>> protection to Zip entries >>>>>>> >>>>>>> Hi Sergey, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> >>>>>>> I added that description in new webrev: >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>>> >>>>>>>> Should the new methods describe how they will work in case of null >>>>>>>> params? >>>>>>>> >>>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>>> >>>>>>>>> Could you review it? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>>> Hi Sherman, >>>>>>>>>> >>>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>>> jdk9/dev >>>>>>>>>> repos. >>>>>>>>>> >>>>>>>>>> Do you have any comments about current webrev? >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>> >>>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>>> >>>>>>>>>>> Hi Sherman, >>>>>>>>>>> >>>>>>>>>>> Thanks for your quick response :) >>>>>>>>>>> >>>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>>> below >>>>>>>>>>> reasons. >>>>>>>>>>> >>>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>>> like a >>>>>>>>>>> temporary file. >>>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>>> best >>>>>>>>>>> for >>>>>>>>>>> openjdk. >>>>>>>>>>> * PKWare claims that they have patents about the "stronger" >>>>>>>>>>> on >>>>>>>>>>> Zip[1]. >>>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>>> "stronger" [2][3]. >>>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>>> interface >>>>>>>>>>> to >>>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>>> >>>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>>> easily >>>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>>> >>>>>>>>>>> [1] https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption Specification) >>>>>>>>>>> >>>>>>>>>>> [2] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>>> >>>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Yuji >>>>>>>>>>> >>>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming Shen: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi Yuji, >>>>>>>>>>>> >>>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>>> bring >>>>>>>>>>>> in the >>>>>>>>>>>> security guy >>>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>>> in >>>>>>>>>>>> the >>>>>>>>>>>> "traditional PKWare >>>>>>>>>>>> decryption", which is, based on the wiki, "known to be seriously >>>>>>>>>>>> flawed, and >>>>>>>>>>>> in particular >>>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>>> support >>>>>>>>>>>> "stronger" encryption >>>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Sherman >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Yuji >>>>>>>>>>>>> >>>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>>> before >>>>>>>>>>>>>> get >>>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>>> However, >>>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we must >>>>>>>>>>>>>> use >>>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>> The >>>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has been >>>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>>> zlib >>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Test code is as below. This code will let you know how >>>>>>>>>>>>>> this >>>>>>>>>>>>>> PoC >>>>>>>>>>>>>> works. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>>> >>>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are providing >>>>>>>>>>>>>> many >>>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>>> implement to >>>>>>>>>>>>>> handle password-protected zip file. I guess that this proposal >>>>>>>>>>>>>> is >>>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1]: >>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>> >> From chris.hegarty at oracle.com Wed Jan 6 14:39:26 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 6 Jan 2016 14:39:26 +0000 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568C3493.10806@Oracle.com> References: <568C0A5C.7090204@Oracle.com> <06A536F9-13AC-4A82-9FFB-129086494CFC@oracle.com> <568C3493.10806@Oracle.com> Message-ID: <568D271E.4000809@oracle.com> Roger, On 05/01/16 21:24, Roger Riggs wrote: > Hi Chris, > > On 1/5/2016 2:33 PM, Chris Hegarty wrote: >> On 5 Jan 2016, at 18:24, Roger Riggs wrote: >> >>> The follow on work to adding the Cleaner is to replace uses of finalization with uses of the Cleaner. >>> For the 'easy' cases in the java.base module, it is useful to introduce a private Cleaner for the >>> java.base module. It is proposed to be held weakly, to allow it to terminate on a lightly loaded >>> system. >>> >>> Webrev for Review: >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >> Looks ok Roger. Does it make sense to put CleanerFactory, and maybe >> CleanerImpl, into their own internal package, so that it can be used as a >> JDK platform Cleaner ( rather than a base module only cleaner, or having >> to qualify exports of jdk.internal.misc to other JDK modules ) ? > That could be useful; but it raises the question about the right/good > granularity. > It might make sense that the desktop module might reasonably share the > Cleaner. > Is there any comparable functionality/package already identified? I'm not sure that this has come up before. I'm just trying to prevent jdk.internal.misc from becoming a general dumping ground ( this is not to say that the Cleaner is rubbish, it is not ). We can revisit this later, if needed. >> Is it possible of the NIO Buffers to use this Cleaner? > I did prototype using the java.lang.ref.Cleaner instead of > sun.misc.Cleaner and it didn't work so > well. Some of the tests failed with out of memory exceptions. I > suspected that the fast path > through the reference processing for the Cleaner was managing to free > memory more quickly than > was possible with the new Cleaner. Ah, I was not aware that the VM had intimate knowledge of sun.misc.Cleaner. So, from a JEP 260 point of view, the base module will need to retain an internal mechanism for releasing NIO buffers ( I was hoping it could use the new Cleaner ). -Chris. From daniel.fuchs at oracle.com Wed Jan 6 14:52:18 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 6 Jan 2016 15:52:18 +0100 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568C3298.7030200@Oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> Message-ID: <568D2A22.1070307@oracle.com> Hi Roger, I was wondering whether URLJarFile::finalize could be simply removed. The change you are proposing looks strange to me: if this method remains, then either the method should continue to call close() or it should call super.finalize(), shouldn't it? Since this is a sun.net class then maybe simply removing it would be the safest thing... best regards, -- daniel On 05/01/16 22:16, Roger Riggs wrote: > Hi Daniel, > > webrev updated to revert changes to MeteredStream as too risky. > > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html > > On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >> Hi Roger, >> >> Some early feedback: >> >> I see that prior to your changes, MeteredStream.close() would >> be called by finalize. >> This will no longer be the case after >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html > In the case where finalize would call close(), there can be no > queuedForCleanup activity > because there can be no strong reference from the KeepAliveStreamCleaner > thread > or the queue that holds the pending cleanups. > It cannot be in the Cache of connections being kept alive or the thread > that keeps them alive. > > It might be possible that the underlying connection is still open; but > there is no advantage > of trying to drain it. There is the possibility of resurrecting it by > virtue of deciding that > it should be queued for cleanup and starting a new thread to do the cleanup. > > All in all, there is a risk of misunderstanding the dynamic behavior and > it would be safer > to leave this using finalize. > >> >> I see that MeteredStream has a subclass (KeepAliveStream) that >> overrides close() - so KeepAliveStream probably requires a cleanup >> as well - doesn't it? >> >> Finally I'd suggest making the variable 'pi' final - since you're >> passing that to the lambda you don't want to allow its value to >> be modified afterwards (fortunately it's not - and I believe it >> should be good to ensure it won't be). > ok, it would not go unnoticed, the compiler should complain if the > variable is no effectively final. > > Thanks, Roger > >> >> I haven't looked at the other classes yet... >> >> best regards, >> >> -- daniel >> >> On 05/01/16 19:24, Roger Riggs wrote: >>> The follow on work to adding the Cleaner is to replace uses of >>> finalization with uses of the Cleaner. >>> For the 'easy' cases in the java.base module, it is useful to introduce >>> a private Cleaner for the >>> java.base module. It is proposed to be held weakly, to allow it to >>> terminate on a lightly loaded >>> system. >>> >>> Webrev for Review: >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>> >>> >>> The 2nd step is using the Cleaner. >>> - Empty finalize methods should (I think) be removed; but since they >>> are part of the public spec >>> the process needs two full releases; so the proposal is to deprecate >>> them first. >>> (The JEP 277 necessary changes will be updated when JEP 277 >>> semantics are finalized) >>> >>> - In a few cases, the code in the finalizer is redundant and if removed >>> would allow >>> an optimization of the handling of the finalizer and future removal >>> of the finalize method. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>> >>> Thanks for comments and suggestions, Roger >>> >>> >>> >>> >> > From Roger.Riggs at Oracle.com Wed Jan 6 14:54:43 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Jan 2016 09:54:43 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568D271E.4000809@oracle.com> References: <568C0A5C.7090204@Oracle.com> <06A536F9-13AC-4A82-9FFB-129086494CFC@oracle.com> <568C3493.10806@Oracle.com> <568D271E.4000809@oracle.com> Message-ID: <568D2AB3.3080303@Oracle.com> Hi Chris, On 1/6/2016 9:39 AM, Chris Hegarty wrote: > Roger, > > On 05/01/16 21:24, Roger Riggs wrote: >> Hi Chris, >> >> On 1/5/2016 2:33 PM, Chris Hegarty wrote: >>> On 5 Jan 2016, at 18:24, Roger Riggs wrote: >>> >>>> The follow on work to adding the Cleaner is to replace uses of >>>> finalization with uses of the Cleaner. >>>> For the 'easy' cases in the java.base module, it is useful to >>>> introduce a private Cleaner for the >>>> java.base module. It is proposed to be held weakly, to allow it to >>>> terminate on a lightly loaded >>>> system. >>>> >>>> Webrev for Review: >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>> Looks ok Roger. Does it make sense to put CleanerFactory, and maybe >>> CleanerImpl, into their own internal package, so that it can be used >>> as a >>> JDK platform Cleaner ( rather than a base module only cleaner, or >>> having >>> to qualify exports of jdk.internal.misc to other JDK modules ) ? >> That could be useful; but it raises the question about the right/good >> granularity. >> It might make sense that the desktop module might reasonably share the >> Cleaner. >> Is there any comparable functionality/package already identified? > > I'm not sure that this has come up before. I'm just trying to > prevent jdk.internal.misc from becoming a general dumping ground > ( this is not to say that the Cleaner is rubbish, it is not ). > We can revisit this later, if needed. I'd want to make a distinction between classes being made available to OpenJDK for internal use vs implementation classes. Perhaps a jdk.internal.ref package (or jdk.internal.util) for the CleanerFactory and the subclassable Cleanable types and leave jdk.internal.misc as a dumping ground for base module implementations. > >>> Is it possible of the NIO Buffers to use this Cleaner? >> I did prototype using the java.lang.ref.Cleaner instead of >> sun.misc.Cleaner and it didn't work so >> well. Some of the tests failed with out of memory exceptions. I >> suspected that the fast path >> through the reference processing for the Cleaner was managing to free >> memory more quickly than >> was possible with the new Cleaner. > > Ah, I was not aware that the VM had intimate knowledge of > sun.misc.Cleaner. So, from a JEP 260 point of view, the base > module will need to retain an internal mechanism for releasing > NIO buffers ( I was hoping it could use the new Cleaner ). The VM doesn't know about sun.misc.Cleaner, just the Reference handling processing in Reference.tryHandlePending. There is a fast path dequeuing References that need processing from the VM. Roger > > -Chris. From Roger.Riggs at Oracle.com Wed Jan 6 15:05:05 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Jan 2016 10:05:05 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568D2A22.1070307@oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> <568D2A22.1070307@oracle.com> Message-ID: <568D2D21.20105@Oracle.com> Hi Daniel, On 1/6/2016 9:52 AM, Daniel Fuchs wrote: > Hi Roger, > > I was wondering whether URLJarFile::finalize could be simply removed. > > The change you are proposing looks strange to me: if this method > remains, then either the method should continue to call close() or > it should call super.finalize(), shouldn't it? yes, I didn't look far enough for super.finalize; ZipFile has a finalize method to be called. (There would be no use in calling Object.finalize) > > Since this is a sun.net class then maybe simply removing it would > be the safest thing... Better yet! Thanks, Roger > > best regards, > > -- daniel > > On 05/01/16 22:16, Roger Riggs wrote: >> Hi Daniel, >> >> webrev updated to revert changes to MeteredStream as too risky. >> >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >> >> On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >>> Hi Roger, >>> >>> Some early feedback: >>> >>> I see that prior to your changes, MeteredStream.close() would >>> be called by finalize. >>> This will no longer be the case after >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>> >> In the case where finalize would call close(), there can be no >> queuedForCleanup activity >> because there can be no strong reference from the KeepAliveStreamCleaner >> thread >> or the queue that holds the pending cleanups. >> It cannot be in the Cache of connections being kept alive or the thread >> that keeps them alive. >> >> It might be possible that the underlying connection is still open; but >> there is no advantage >> of trying to drain it. There is the possibility of resurrecting it by >> virtue of deciding that >> it should be queued for cleanup and starting a new thread to do the >> cleanup. >> >> All in all, there is a risk of misunderstanding the dynamic behavior and >> it would be safer >> to leave this using finalize. >> >>> >>> I see that MeteredStream has a subclass (KeepAliveStream) that >>> overrides close() - so KeepAliveStream probably requires a cleanup >>> as well - doesn't it? >>> >>> Finally I'd suggest making the variable 'pi' final - since you're >>> passing that to the lambda you don't want to allow its value to >>> be modified afterwards (fortunately it's not - and I believe it >>> should be good to ensure it won't be). >> ok, it would not go unnoticed, the compiler should complain if the >> variable is no effectively final. >> >> Thanks, Roger >> >>> >>> I haven't looked at the other classes yet... >>> >>> best regards, >>> >>> -- daniel >>> >>> On 05/01/16 19:24, Roger Riggs wrote: >>>> The follow on work to adding the Cleaner is to replace uses of >>>> finalization with uses of the Cleaner. >>>> For the 'easy' cases in the java.base module, it is useful to >>>> introduce >>>> a private Cleaner for the >>>> java.base module. It is proposed to be held weakly, to allow it to >>>> terminate on a lightly loaded >>>> system. >>>> >>>> Webrev for Review: >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>>> >>>> >>>> The 2nd step is using the Cleaner. >>>> - Empty finalize methods should (I think) be removed; but since they >>>> are part of the public spec >>>> the process needs two full releases; so the proposal is to >>>> deprecate >>>> them first. >>>> (The JEP 277 necessary changes will be updated when JEP 277 >>>> semantics are finalized) >>>> >>>> - In a few cases, the code in the finalizer is redundant and if >>>> removed >>>> would allow >>>> an optimization of the handling of the finalizer and future removal >>>> of the finalize method. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>> >>>> >>>> Thanks for comments and suggestions, Roger >>>> >>>> >>>> >>>> >>> >> > From Alan.Bateman at oracle.com Wed Jan 6 15:32:28 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 6 Jan 2016 15:32:28 +0000 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568C0A5C.7090204@Oracle.com> References: <568C0A5C.7090204@Oracle.com> Message-ID: <568D338C.4000105@oracle.com> On 05/01/2016 18:24, Roger Riggs wrote: > The follow on work to adding the Cleaner is to replace uses of > finalization with uses of the Cleaner. > For the 'easy' cases in the java.base module, it is useful to > introduce a private Cleaner for the > java.base module. It is proposed to be held weakly, to allow it to > terminate on a lightly loaded > system. > > Webrev for Review: > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ This looks okay but can't commonCleaner be final? > > > The 2nd step is using the Cleaner. > - Empty finalize methods should (I think) be removed; but since they > are part of the public spec > the process needs two full releases; so the proposal is to > deprecate them first. > (The JEP 277 necessary changes will be updated when JEP 277 > semantics are finalized) I assume some of these could just be removed - for example SocketInputStream/SocketOutputStream are package private and aren't part of the API. But maybe you mean that you don't want to touch some java.* classes? -Alan. From mandy.chung at oracle.com Wed Jan 6 16:15:12 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 6 Jan 2016 08:15:12 -0800 Subject: RFR [9] 8146484: Examine sun.misc.MessageUtils In-Reply-To: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> References: <9800DDCC-AEAB-4884-9226-1A1628C91BC1@oracle.com> Message-ID: +1 Mandy > On Jan 5, 2016, at 9:28 PM, Chris Hegarty wrote: > > sun.misc.MessageUtils provides low-level access for writing ?error? messages > to stdout and stderr. It is, effectively, used in just one place in the JDK, String > coding, when the specific encoding is not available. Since this is the only usage, > the minimal functionality required by String coding can be moved to StringCoding, > and MessageUtils removed. > > http://cr.openjdk.java.net/~chegar/8146484/ > > Note: there are a couple of unused reference to MessageUtils in the desktop > module and a build tool that need to be cleaned up. > > -Chris. From chris.hegarty at oracle.com Wed Jan 6 16:15:11 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 6 Jan 2016 16:15:11 +0000 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568D2AB3.3080303@Oracle.com> References: <568C0A5C.7090204@Oracle.com> <06A536F9-13AC-4A82-9FFB-129086494CFC@oracle.com> <568C3493.10806@Oracle.com> <568D271E.4000809@oracle.com> <568D2AB3.3080303@Oracle.com> Message-ID: <568D3D8F.2000707@oracle.com> On 06/01/16 14:54, Roger Riggs wrote: > Hi Chris, > > On 1/6/2016 9:39 AM, Chris Hegarty wrote: >> Roger, >> >> On 05/01/16 21:24, Roger Riggs wrote: >>> Hi Chris, >>> >>> On 1/5/2016 2:33 PM, Chris Hegarty wrote: >>>> On 5 Jan 2016, at 18:24, Roger Riggs wrote: >>>> >>>>> The follow on work to adding the Cleaner is to replace uses of >>>>> finalization with uses of the Cleaner. >>>>> For the 'easy' cases in the java.base module, it is useful to >>>>> introduce a private Cleaner for the >>>>> java.base module. It is proposed to be held weakly, to allow it to >>>>> terminate on a lightly loaded >>>>> system. >>>>> >>>>> Webrev for Review: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>>> Looks ok Roger. Does it make sense to put CleanerFactory, and maybe >>>> CleanerImpl, into their own internal package, so that it can be used >>>> as a >>>> JDK platform Cleaner ( rather than a base module only cleaner, or >>>> having >>>> to qualify exports of jdk.internal.misc to other JDK modules ) ? >>> That could be useful; but it raises the question about the right/good >>> granularity. >>> It might make sense that the desktop module might reasonably share the >>> Cleaner. >>> Is there any comparable functionality/package already identified? >> >> I'm not sure that this has come up before. I'm just trying to >> prevent jdk.internal.misc from becoming a general dumping ground >> ( this is not to say that the Cleaner is rubbish, it is not ). >> We can revisit this later, if needed. > I'd want to make a distinction between classes being made available to > OpenJDK for internal use vs implementation classes. > > Perhaps a jdk.internal.ref package (or jdk.internal.util) for the > CleanerFactory and > the subclassable Cleanable types and leave jdk.internal.misc as a > dumping ground for > base module implementations. That sounds ok to me. >>>> Is it possible of the NIO Buffers to use this Cleaner? >>> I did prototype using the java.lang.ref.Cleaner instead of >>> sun.misc.Cleaner and it didn't work so >>> well. Some of the tests failed with out of memory exceptions. I >>> suspected that the fast path >>> through the reference processing for the Cleaner was managing to free >>> memory more quickly than >>> was possible with the new Cleaner. >> >> Ah, I was not aware that the VM had intimate knowledge of >> sun.misc.Cleaner. So, from a JEP 260 point of view, the base >> module will need to retain an internal mechanism for releasing >> NIO buffers ( I was hoping it could use the new Cleaner ). > The VM doesn't know about sun.misc.Cleaner, just the Reference handling > processing in > Reference.tryHandlePending. There is a fast path dequeuing References > that need processing > from the VM. Oh. I found some references to sun.misc.Cleaner in the VM [1] [2], but maybe it does not affect how/when these get queued. -Chris. [1] http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/tip/src/share/vm/memory/referenceType.hpp#l39 [2] http://hg.openjdk.java.net/jdk9/jdk9/hotspot/file/tip/src/share/vm/classfile/systemDictionary.cpp#l1977 From Roger.Riggs at Oracle.com Wed Jan 6 18:14:53 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Jan 2016 13:14:53 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568D338C.4000105@oracle.com> References: <568C0A5C.7090204@Oracle.com> <568D338C.4000105@oracle.com> Message-ID: <568D599D.8060209@Oracle.com> Hi Alan, On 1/6/2016 10:32 AM, Alan Bateman wrote: > > > On 05/01/2016 18:24, Roger Riggs wrote: >> The follow on work to adding the Cleaner is to replace uses of >> finalization with uses of the Cleaner. >> For the 'easy' cases in the java.base module, it is useful to >> introduce a private Cleaner for the >> java.base module. It is proposed to be held weakly, to allow it to >> terminate on a lightly loaded >> system. >> >> Webrev for Review: >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ > This looks okay but can't commonCleaner be final? yes, fixed. I prototype a version that kept a WeakReference to the Cleaner so it could terminate if not used. But it is more complex to make sure the Cleaner would be referenced *only* as long as there were pending Cleanables and depending on its use it might thrash starting a cleaner (thread) and then terminating, necessitating re-creating on the next use. > >> >> >> The 2nd step is using the Cleaner. >> - Empty finalize methods should (I think) be removed; but since they >> are part of the public spec >> the process needs two full releases; so the proposal is to >> deprecate them first. >> (The JEP 277 necessary changes will be updated when JEP 277 >> semantics are finalized) > I assume some of these could just be removed - for example > SocketInputStream/SocketOutputStream are package private and aren't > part of the API. But maybe you mean that you don't want to touch some > java.* classes? Will remove them; its clearer. I'll split the finalizers being removed into a separate webrev; since they don't fit the original description as replacements. Thanks, Roger > > -Alan. From peter.levart at gmail.com Wed Jan 6 19:02:32 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 6 Jan 2016 20:02:32 +0100 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568C3298.7030200@Oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> Message-ID: <568D64C8.7030401@gmail.com> On 01/05/2016 10:16 PM, Roger Riggs wrote: > Hi Daniel, > > webrev updated to revert changes to MeteredStream as too risky. > > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html Hi Roger, I briefly skimmed over the webrev, and found the following issue in ProcessImpl: 420 // Register a cleaning function to close the handle 421 CleanerFactory.getCleaner().register(this, () -> closeHandle(handle)); 422 ... 'handle' is an instance variable, which means that 'this' is captured by the lambda. You will have to assign handle to a local var and capture it instead to prevent 'this' to be captured. Regards, Peter > > On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >> Hi Roger, >> >> Some early feedback: >> >> I see that prior to your changes, MeteredStream.close() would >> be called by finalize. >> This will no longer be the case after >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html > In the case where finalize would call close(), there can be no > queuedForCleanup activity > because there can be no strong reference from the > KeepAliveStreamCleaner thread > or the queue that holds the pending cleanups. > It cannot be in the Cache of connections being kept alive or the > thread that keeps them alive. > > It might be possible that the underlying connection is still open; but > there is no advantage > of trying to drain it. There is the possibility of resurrecting it by > virtue of deciding that > it should be queued for cleanup and starting a new thread to do the > cleanup. > > All in all, there is a risk of misunderstanding the dynamic behavior > and it would be safer > to leave this using finalize. > >> >> I see that MeteredStream has a subclass (KeepAliveStream) that >> overrides close() - so KeepAliveStream probably requires a cleanup >> as well - doesn't it? >> >> Finally I'd suggest making the variable 'pi' final - since you're >> passing that to the lambda you don't want to allow its value to >> be modified afterwards (fortunately it's not - and I believe it >> should be good to ensure it won't be). > ok, it would not go unnoticed, the compiler should complain if the > variable is no effectively final. > > Thanks, Roger > >> >> I haven't looked at the other classes yet... >> >> best regards, >> >> -- daniel >> >> On 05/01/16 19:24, Roger Riggs wrote: >>> The follow on work to adding the Cleaner is to replace uses of >>> finalization with uses of the Cleaner. >>> For the 'easy' cases in the java.base module, it is useful to introduce >>> a private Cleaner for the >>> java.base module. It is proposed to be held weakly, to allow it to >>> terminate on a lightly loaded >>> system. >>> >>> Webrev for Review: >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>> >>> >>> The 2nd step is using the Cleaner. >>> - Empty finalize methods should (I think) be removed; but since they >>> are part of the public spec >>> the process needs two full releases; so the proposal is to >>> deprecate >>> them first. >>> (The JEP 277 necessary changes will be updated when JEP 277 >>> semantics are finalized) >>> >>> - In a few cases, the code in the finalizer is redundant and if removed >>> would allow >>> an optimization of the handling of the finalizer and future removal >>> of the finalize method. >>> >>> Webrev: >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>> >>> >>> Thanks for comments and suggestions, Roger >>> >>> >>> >>> >> > From xueming.shen at oracle.com Wed Jan 6 19:04:51 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 6 Jan 2016 11:04:51 -0800 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> Message-ID: <568D6553.4090108@oracle.com> On 1/6/16 5:25 AM, KUBOTA Yuji wrote: > Hi Sherman, > > Thank you for your feedback! > > At first, I'd like to confirm your thought about ZipCryption to > prevent misunderstand. > By following reasons, we should remove (or modified) ZipCryption to > not have any "encryption" related field/method in DeflaterOutputStream > / InflaterInputStream. > > 1. For security, to hide "passwd/encryption" support by not just > adding "password" specific method. > 2. To avoid that JDK API developers implement "passwd/encryption" > through ZipCryption. > > Is it right? In my previous email, I thought we should have the public > interface to provide scalability for supporting any open-source > encryption implementation such as WinZip AES. > If the reason is right, your suggestion sounds reasonable to us as below. The reason that I'm not convinced that we really need a public interface of ZipCryption here is I don't know how useful/helpful/likely it would be going forward that someone might really use this interface to implement other encryption(s), especially the pkware proprietary one, I doubt it might be not that straightforward. In fact I did have a draft implementation that supports WinZip AES about 5-6 years ago :-) (which also supports compression methods bzip and lzma, btw) Here is the top class, It appears a general interface might not be that helpful and it might be ideal to simply implement it inside the JDK, as what is proposed here, when it's really desired. http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java It is a ZipFile based implementation, so it does not have the headache that ZipInputStream has, such as to push back the bytes belong to next entry, since the loc might not have the needed info regarding the size/csize in stream mode. From abstract point of view. The "encryption" and "compression" are different layers, it would be ideal to have them in separate classes logically, instead of mixing the encryption into compression. Sure, it might be convenient and probably may have better performance to mix them in certain use scenario, but the "encryption" should never appear in the public interface of those compression classes. Package private interface should be fine, if have to. -Sherman > > 2016-01-06 7:10 GMT+09:00 Xueming Shen : >> it appears that instead of adding "password" specific method to these >> classes directly, it might be more appropriate to extend the ZipEntry class >> for such "password" functionality. For example, with a pair of new methods >> >> boolean ZipEntry.isTraditionalEncryption(). >> void ZipEntry.setTraditionalEncryption(String password); > Thanks advice, I agree. We should re-design the API to extend the > ZipEntry class. > >> The encryption support should/can be added naturally/smoothly with >> ZipFile.getInputStream(e), ZipInputstream and >> ZipOutputStream.putNextEntry(e), >> with no extra new method in these two classes. The implementation checks >> the flag (bit0, no bit 6) first and then verifies the password, as an >> implementation details. > Agree. For this proposal, we aim to support only traditional > encryption. So I think we should also check bit 6. > >> For ZipFile and ZipInputStream, we can add note to the api doc to force the >> invoker to check if the returned ZipEntry indicates it's an encrypted entry. >> If yes, it must to set the appropriate password to the returned ZipEntry via >> ZipEntry.setTraditionalEncryption(password); before reading any byte from >> the input stream. > Yes, we have to add note the flow of codes to the JavaDoc. > >> Again, we should not have any "encryption" related public field/method in >> DeflaterOutputStream/InflaterInputStream. Ideally these two classes really >> should not be aware of "encryption" at all. > Agree, but I think we might be faced technical difficulty about a > processing between zlib and the internal buffer of InflaterInputStream > / DeflaterOutputStream. Please give us time to implement. > >> -Sherman > Thanks, > Yuji > > >> On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: >>> Hi Sherman and all, >>> >>> Happy new year to everyone! >>> >>> Please let know your feedback about this proposal. :-) >>> >>> Thanks, >>> Yuji >>> >>> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >>>> Hi Sherman, >>>> >>>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>>>> It is no longer necessary to touch the native code (zip_util.c/h) after >>>>> the >>>>> native ZipFile implementation has been moved up to the java level. Those >>>>> native code are for vm access only now, which I dont think care about >>>>> the >>>>> password support at all. >>>> Thanks for your information. We do not take care the native. >>>> >>>> I discussed with Yasumasa, and our thought is as below. >>>> >>>>> (1) what's the benefit of exposing the public interface ZipCryption? the >>>>> real >>>>> question is whether or not this interface is good enough for other >>>>> encryption >>>>> implementation to plugin their implementation to support the >>>>> ZipFile/Input/ >>>>> OutputStream to their encryption spec. >>>> We aimed that the public interface ZipCryption supports the >>>> extensibillity for other encrypt engine. The JDK core libs developers >>>> have to implementation ZipyCryption only. If not provide, the JDK >>>> developers must implement ZipStream/Entry by JDK API to design the >>>> data structure of entry. >>>> If you want to use binary key data such as PKI, you can implement new >>>> encrypt/decrypt engine by ZipCryption interface. >>>> So we think we should provide this interface to be clearly how to >>>> implement a new engine, e.g., cipher algorithm, cipher strength and >>>> converting the header, etc. >>>> >>>>> (2) it seems like it might be possible to hide most of the >>>>> implementation >>>>> and only expose the "String password" (instead of the ZipCryption) as >>>>> the >>>>> public interface to support the "traditional" encryption. This depends >>>>> on the >>>>> result of (1) though. >>>> Thanks for your clues. We think the string password at first. However, >>>> we should also create a new binary interface given we support PKI in >>>> the future. >>>> >>>>> (3) I'm concerned of pushing ZipCryption into >>>>> InflaterInputStream/DeflaterOutputStream. >>>>> It might be worth considering to replace the ZipCryption implementation >>>>> with >>>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>>> use >>>>> the FilterOutputStream for the ZipOutputStream and the FilterInputStream >>>>> for the >>>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue ... >>>> Thanks for your clues, too. Honestly speaking, we think the current >>>> zip implementation may break the data when used PushbackInputStream >>>> for the following reasons. >>>> >>>> * PushbackInputStream uses an unique internal buffer for re-read >>>> operation. >>>> * But, InflaterInputStream provide date to Inflater per reads and >>>> buffer by JNI (zlib). >>>> * So we think PushbackInputStream is poor compatibility with >>>> InflaterInputStream. >>>> >>>> We generally use InputStream through ZipEntry#getInputStream(). We do >>>> not touch FileInputStream for reading ZIP data. If we call unread() >>>> when we use PushbackInputStream as reading ZIP archive, we guess that >>>> it will break the reading data. >>>> So, our approach do not affect the PushbackInputStream. >>>> What do you think about this? >>>> >>>>> (4) It seems the ZipOutputStream only supports the "stream based" >>>>> password, while >>>>> the ZipInputStream supports the "entry based" password. Do we really >>>>> need >>>>> "entry based" support here? >>>> As your suggestion, we should support "entry based". We will start to >>>> implement "entry based" after this discussion is closed. >>>> >>>> Thanks, >>>> Yuji >>>> >>>>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>>>> Hi Jason, >>>>>> >>>>>> Thank you for your comment. >>>>>> I've fixed it in new webrev: >>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Yasumasa >>>>>> >>>>>> >>>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is in >>>>>>> conflict with line 350 and 348. >>>>>>> >>>>>>> ________________________________________ >>>>>>> From: core-libs-dev on behalf >>>>>>> of >>>>>>> Yasumasa Suenaga >>>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>>> protection to Zip entries >>>>>>> >>>>>>> Hi Sergey, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> >>>>>>> I added that description in new webrev: >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>>>> Should the new methods describe how they will work in case of null >>>>>>>> params? >>>>>>>> >>>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>>> >>>>>>>>> Could you review it? >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>>>> Hi Sherman, >>>>>>>>>> >>>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>>> jdk9/dev >>>>>>>>>> repos. >>>>>>>>>> >>>>>>>>>> Do you have any comments about current webrev? >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>> >>>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>>>> Hi Sherman, >>>>>>>>>>> >>>>>>>>>>> Thanks for your quick response :) >>>>>>>>>>> >>>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>>> below >>>>>>>>>>> reasons. >>>>>>>>>>> >>>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>>> like a >>>>>>>>>>> temporary file. >>>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>>> best >>>>>>>>>>> for >>>>>>>>>>> openjdk. >>>>>>>>>>> * PKWare claims that they have patents about the "stronger" >>>>>>>>>>> on >>>>>>>>>>> Zip[1]. >>>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>>> "stronger" [2][3]. >>>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>>> interface >>>>>>>>>>> to >>>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>>> >>>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>>> easily >>>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>>> >>>>>>>>>>> [1] https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption Specification) >>>>>>>>>>> >>>>>>>>>>> [2] >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>>> >>>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> Yuji >>>>>>>>>>> >>>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming Shen: >>>>>>>>>>>> >>>>>>>>>>>> Hi Yuji, >>>>>>>>>>>> >>>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>>> bring >>>>>>>>>>>> in the >>>>>>>>>>>> security guy >>>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>>> in >>>>>>>>>>>> the >>>>>>>>>>>> "traditional PKWare >>>>>>>>>>>> decryption", which is, based on the wiki, "known to be seriously >>>>>>>>>>>> flawed, and >>>>>>>>>>>> in particular >>>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>>> support >>>>>>>>>>>> "stronger" encryption >>>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>>> >>>>>>>>>>>> Regards, >>>>>>>>>>>> Sherman >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi all, >>>>>>>>>>>>> >>>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Yuji >>>>>>>>>>>>> >>>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>>> before >>>>>>>>>>>>>> get >>>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>>> However, >>>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we must >>>>>>>>>>>>>> use >>>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>> The >>>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has been >>>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>>> zlib >>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>> >>>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Test code is as below. This code will let you know how >>>>>>>>>>>>>> this >>>>>>>>>>>>>> PoC >>>>>>>>>>>>>> works. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>>> >>>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are providing >>>>>>>>>>>>>> many >>>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>>> implement to >>>>>>>>>>>>>> handle password-protected zip file. I guess that this proposal >>>>>>>>>>>>>> is >>>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1]: >>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Yuji >>>>>>>>>>>> >>>>>>>>>>>> From Roger.Riggs at Oracle.com Wed Jan 6 19:12:17 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Jan 2016 14:12:17 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568D64C8.7030401@gmail.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> <568D64C8.7030401@gmail.com> Message-ID: <568D6711.9090504@Oracle.com> Hi Peter, Thanks for the review and catching that. Will fix. Roger On 1/6/2016 2:02 PM, Peter Levart wrote: > > > On 01/05/2016 10:16 PM, Roger Riggs wrote: >> Hi Daniel, >> >> webrev updated to revert changes to MeteredStream as too risky. >> >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html > > > Hi Roger, > > I briefly skimmed over the webrev, and found the following issue in > ProcessImpl: > > 420 // Register a cleaning function to close the handle > 421 CleanerFactory.getCleaner().register(this, () -> > closeHandle(handle)); > 422 > > ... 'handle' is an instance variable, which means that 'this' is > captured by the lambda. You will have to assign handle to a local var > and capture it instead to prevent 'this' to be captured. > > Regards, Peter > > >> >> On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >>> Hi Roger, >>> >>> Some early feedback: >>> >>> I see that prior to your changes, MeteredStream.close() would >>> be called by finalize. >>> This will no longer be the case after >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>> >> In the case where finalize would call close(), there can be no >> queuedForCleanup activity >> because there can be no strong reference from the >> KeepAliveStreamCleaner thread >> or the queue that holds the pending cleanups. >> It cannot be in the Cache of connections being kept alive or the >> thread that keeps them alive. >> >> It might be possible that the underlying connection is still open; >> but there is no advantage >> of trying to drain it. There is the possibility of resurrecting it >> by virtue of deciding that >> it should be queued for cleanup and starting a new thread to do the >> cleanup. >> >> All in all, there is a risk of misunderstanding the dynamic behavior >> and it would be safer >> to leave this using finalize. >> >>> >>> I see that MeteredStream has a subclass (KeepAliveStream) that >>> overrides close() - so KeepAliveStream probably requires a cleanup >>> as well - doesn't it? >>> >>> Finally I'd suggest making the variable 'pi' final - since you're >>> passing that to the lambda you don't want to allow its value to >>> be modified afterwards (fortunately it's not - and I believe it >>> should be good to ensure it won't be). >> ok, it would not go unnoticed, the compiler should complain if the >> variable is no effectively final. >> >> Thanks, Roger >> >>> >>> I haven't looked at the other classes yet... >>> >>> best regards, >>> >>> -- daniel >>> >>> On 05/01/16 19:24, Roger Riggs wrote: >>>> The follow on work to adding the Cleaner is to replace uses of >>>> finalization with uses of the Cleaner. >>>> For the 'easy' cases in the java.base module, it is useful to >>>> introduce >>>> a private Cleaner for the >>>> java.base module. It is proposed to be held weakly, to allow it to >>>> terminate on a lightly loaded >>>> system. >>>> >>>> Webrev for Review: >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>>> >>>> >>>> The 2nd step is using the Cleaner. >>>> - Empty finalize methods should (I think) be removed; but since they >>>> are part of the public spec >>>> the process needs two full releases; so the proposal is to >>>> deprecate >>>> them first. >>>> (The JEP 277 necessary changes will be updated when JEP 277 >>>> semantics are finalized) >>>> >>>> - In a few cases, the code in the finalizer is redundant and if >>>> removed >>>> would allow >>>> an optimization of the handling of the finalizer and future removal >>>> of the finalize method. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>> >>>> >>>> Thanks for comments and suggestions, Roger >>>> >>>> >>>> >>>> >>> >> > From Roger.Riggs at Oracle.com Wed Jan 6 22:32:06 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 6 Jan 2016 17:32:06 -0500 Subject: RFR 9: 8146574 (process) ProcessHandle test cleanup Message-ID: <568D95E6.4060107@Oracle.com> Please review a few cleanups of the ProcessHandle tests. - Switch to the jdk.test.lib (from lib/testlibrary ) - Cleanup import statements - Remove TEST.properties and use explicit @run testng argument in each test Webrev: http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-8146574/ Issue: https://bugs.openjdk.java.net/browse/JDK-8146574 Thanks, Roger From martinrb at google.com Wed Jan 6 23:21:16 2016 From: martinrb at google.com (Martin Buchholz) Date: Wed, 6 Jan 2016 15:21:16 -0800 Subject: use of Unsafe for ASCII detection Message-ID: Hi guys, It turns out that much of the world's computers is busy doing UTF-8 decoding, so lots of people have expended much effort to make it faster. Much of that text is ASCII, so it's definitely worth having special ASCII-only loops. Like this: static boolean isAscii1(byte[] bytes) { for (byte b : bytes) if (b < 0) return false; return true; } Hotspot does a pretty good job on that, but we can go faster if we reach for Unsafe, processing data a long at a time. static boolean isAscii2(byte[] bytes) { long i, limit; for (i = ABASE, limit = ABASE + bytes.length - 7; i < limit; i += 8) if ((U.getLong(bytes, i) & 0x8080808080808080L) != 0) return false; for (limit = ABASE + bytes.length; i < limit; i += 1) if (U.getByte(bytes, i) < 0) return false; return true; } We can also try the same strategy with ByteBuffer.getLong, hoping it will do atomic reads for us: static boolean isAscii3(byte[] bytes) { ByteBuffer b = ByteBuffer.wrap(bytes); int i, limit; for (i = 0, limit = bytes.length - 7; i < limit; i += 8) if ((b.getLong(i) & 0x8080808080808080L) != 0) return false; for (limit = bytes.length; i < limit; i += 1) if (b.get(i) < 0) return false; return true; } A stupid benchmark (sorry Aleksey - maybe you already have a jmh version of this?) http://cr.openjdk.java.net/~martin/Utf8Bench.java gives me: (jdk9) Unsafe stride 8: 0.35626781581093897 Buffer stride 8: 0.4825880453758097 (jdk8) Unsafe stride 8: 0.33818706070673143 Buffer stride 8: 1.9958109701624323 That is, the Unsafe code is 3x faster than the simple code. The ByteBuffer code used to be 2x slower and is now 2x faster - well done - crowd goes wild! I see it uses new and well-hidden Unsafe.getLongUnaligned ... all the performance fanatics want to use that, but y'all don't want to give it to us. And we don't want to allocate a temp ByteBuffer object. What to do? If you provided a static variant of ByteBuffer.getLong for byte[], it would probably have enough performance for us to live without Unsafe. It's hard to resist Unsafe when it gives 3x improvement in a performance hotspot. Maybe C2 can optimize the simple loop even more than it is today? Maybe we can add a static method on j.u.Arrays? Maybe static int asciiCount(byte[] bytes, int start, int end) Still a performance hack, but far less of one. From huizhe.wang at oracle.com Thu Jan 7 00:11:20 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 06 Jan 2016 16:11:20 -0800 Subject: RFR (JAXP): 8144966: Catalog API: Null handling and reference to Reader In-Reply-To: <27D13FC1-CF10-4DDD-8F63-1D70FE8DBAE4@oracle.com> References: <567B264E.1020502@oracle.com> <27D13FC1-CF10-4DDD-8F63-1D70FE8DBAE4@oracle.com> Message-ID: <568DAD28.50204@oracle.com> Thanks Mandy. Please see inline. On 1/4/2016 6:50 PM, Mandy Chung wrote: >> On Dec 23, 2015, at 2:55 PM, huizhe wang wrote: >> >> Hi, >> >> This is an improvement to the new Catalog API on null handling. At the package level, a null argument shall result in NPE. However, an exception was made to the methods in CatalogManager so that they can used conveniently. With this request, we're in favor of consistency over the minor convenience, which means a resolver can no longer be created by: >> CatalogManager.catalogResolver(null); >> >> Instead, the following may be used to achieve the same result: >> CatalogManager.catalogResolver(CatalogFeatures.defaults()); >> >> While path can be empty, it can no longer be null. The following then would also get a NPE: >> CatalogManager.catalogResolver(CatalogFeatures.defaults(), null); >> > This change is reasonable and it?s good to pass CatalogFeatures.defaults() explicitly rather than special casing null to represent that. > >> In consistence with ignoring invalid catalogs as required by the Catalog standard, the statements that required CatalogException when no catalog is found are removed. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8144966 >> webrev: http://cr.openjdk.java.net/~joehw/jdk9/8144966/webrev/ >> > 41 * Creates a {@code Catalog} object using the specified feature settings and path to > 42 * a catalog file. If the path is empty, > > Nit: should it say ?path to one or more catalog files? Fixed. > > What if ?paths? is non-empty but all entries are invalid and no catalog file is read - will it read the system property? It will not read the system property. The assumption is that the paths specified are meant to be used. If they are invalid, it would indicate programming error. I added a couple statements to make it clearer. > > line 46: s/path argument/paths argument/ > > Same comment applied to other factory methods. Fixed. > > > CatalogImpl.java > 117 if (file.length > 0) { > - Do you expect file to be null? If so, NPE will be thrown. If not, file == null in line 125 is not needed. removed "file == null". > > Otherwise the change looks okay. Thanks. I also added specdiff for easier reading: JBS:https://bugs.openjdk.java.net/browse/JDK-8144966 webrev:http://cr.openjdk.java.net/~joehw/jdk9/8144966/webrev/ specdiff: http://cr.openjdk.java.net/~joehw/jdk9/8144966/specdiff/javax/xml/catalog/package-summary.html Joe > Mandy From kishor.kharbas at intel.com Thu Jan 7 00:18:48 2016 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Thu, 7 Jan 2016 00:18:48 +0000 Subject: RFR (M): 8135250: Replace custom check/range functionality with check index/range methods in java.util.Arrays Message-ID: Hello all, Requesting the community to review a patch which replaces custom bound checking functionality with calls to methods in java.util.Objects. The methods affected are CounterMode.crypt() and AESCrypt.encryptBlock(), AESCrypt.decryptBlock() Bug id: https://bugs.openjdk.java.net/browse/JDK-8135250 Webrev: http://cr.openjdk.java.net/~vdeshpande/8135250/webrev.00/ Regards Kishor Kharbas From mandy.chung at oracle.com Thu Jan 7 00:53:09 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 6 Jan 2016 16:53:09 -0800 Subject: RFR (JAXP): 8144966: Catalog API: Null handling and reference to Reader In-Reply-To: <568DAD28.50204@oracle.com> References: <567B264E.1020502@oracle.com> <27D13FC1-CF10-4DDD-8F63-1D70FE8DBAE4@oracle.com> <568DAD28.50204@oracle.com> Message-ID: <8BA66C26-8D4A-45B8-9BF9-6DB30B37F11A@oracle.com> > On Jan 6, 2016, at 4:11 PM, huizhe wang wrote: > > >> >> What if ?paths? is non-empty but all entries are invalid and no catalog file is read - will it read the system property? > > It will not read the system property. The assumption is that the paths specified are meant to be used. If they are invalid, it would indicate programming error. I added a couple statements to make it clearer. The new statement: "In case all entries are invalid, the resolver will return as no mapping is found.? It will return an object with no catalog file, right? Is it equivalent to calling with empty paths and the system property javax.xml.catalog.files is not set? I think this sentence is not necessary and can take it out. "If {@code paths} is specified, no attempt will be made to read the system property even if all entries are invalid or point to invalid catalogs." Otherwise the updated webrev looks fine. Mandy From paul.sandoz at oracle.com Thu Jan 7 08:15:42 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 7 Jan 2016 09:15:42 +0100 Subject: RFR (M): 8135250: Replace custom check/range functionality with check index/range methods in java.util.Arrays In-Reply-To: References: Message-ID: <84E6E74C-CE18-49DE-828B-A7E282904EEF@oracle.com> Hi Kishor, Looks good to me. A small but nice improvement. I guess you are ok with these throwing IndexOutOfBounds rather than ArrayIndexOutOfBounds. ? Note that i am going to revisit the exception message reporting and with nudging by John the BiFunction to ensure no bounds information is lost. It should not affect this code, but should produce better error messages, and after it might be an opportunity to revisit if you want these to throw ArrayIndexOutOfBounds. Paul. > On 7 Jan 2016, at 01:18, Kharbas, Kishor wrote: > > Hello all, > > Requesting the community to review a patch which replaces custom bound checking functionality with calls to methods in java.util.Objects. > > The methods affected are CounterMode.crypt() and AESCrypt.encryptBlock(), AESCrypt.decryptBlock() > > > Bug id: > https://bugs.openjdk.java.net/browse/JDK-8135250 > Webrev: > http://cr.openjdk.java.net/~vdeshpande/8135250/webrev.00/ > > Regards > Kishor Kharbas From amy.lu at oracle.com Thu Jan 7 08:21:48 2016 From: amy.lu at oracle.com (Amy Lu) Date: Thu, 7 Jan 2016 16:21:48 +0800 Subject: JDK 9 RFR of JDK-8146587: Mark tools/pack200/Pack200Test.java as intermittently failing Message-ID: <568E201C.4080707@oracle.com> Test tools/pack200/Pack200Test.java is known to fail intermittently, this patch is to mark the test accordingly with keyword 'intermittent'. bug: https://bugs.openjdk.java.net/browse/JDK-8146587 webrev: http://cr.openjdk.java.net/~amlu/8146587/webrev.00/ Thanks, Amy --- old/test/tools/pack200/Pack200Test.java 2016-01-07 11:07:47.000000000 +0800 +++ new/test/tools/pack200/Pack200Test.java 2016-01-07 11:07:47.000000000 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2007, 2014, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2007, 2016, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -31,6 +31,7 @@ /* * @test * @bug 6521334 6712743 8007902 + * @key intermittent * @summary check for memory leaks, test general packer/unpacker functionality\ * using native and java unpackers * @compile -XDignore.symbol.file Utils.java Pack200Test.java From paul.sandoz at oracle.com Thu Jan 7 08:27:33 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 7 Jan 2016 09:27:33 +0100 Subject: RFR (M): 8135250: Replace custom check/range functionality with check index/range methods in java.util.Arrays In-Reply-To: <84E6E74C-CE18-49DE-828B-A7E282904EEF@oracle.com> References: <84E6E74C-CE18-49DE-828B-A7E282904EEF@oracle.com> Message-ID: <7CBCE02C-DA05-4C2B-B687-28DB5CD9C4FD@oracle.com> > On 7 Jan 2016, at 09:15, Paul Sandoz wrote: > > Hi Kishor, > > Looks good to me. A small but nice improvement. > > I guess you are ok with these throwing IndexOutOfBounds rather than ArrayIndexOutOfBounds. > Actually i just realised you are reusing issue JDK-8135250, which is logged to track a wider range of identified areas in the JDK. Would it be possible to create a new issue [*] related to JDK-8135250 just for the crypto stuff. Paul. [*] If you don?t have the "power ups? to do that i can create one for you. > ? > > Note that i am going to revisit the exception message reporting and with nudging by John the BiFunction to ensure no bounds information is lost. It should not affect this code, but should produce better error messages, and after it might be an opportunity to revisit if you want these to throw ArrayIndexOutOfBounds. > > Paul. > >> On 7 Jan 2016, at 01:18, Kharbas, Kishor wrote: >> >> Hello all, >> >> Requesting the community to review a patch which replaces custom bound checking functionality with calls to methods in java.util.Objects. >> >> The methods affected are CounterMode.crypt() and AESCrypt.encryptBlock(), AESCrypt.decryptBlock() >> >> >> Bug id: >> https://bugs.openjdk.java.net/browse/JDK-8135250 >> Webrev: >> http://cr.openjdk.java.net/~vdeshpande/8135250/webrev.00/ >> >> Regards >> Kishor Kharbas > From chris.hegarty at oracle.com Thu Jan 7 09:03:34 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 7 Jan 2016 09:03:34 +0000 Subject: RFR 9: 8146574 (process) ProcessHandle test cleanup In-Reply-To: <568D95E6.4060107@Oracle.com> References: <568D95E6.4060107@Oracle.com> Message-ID: <83E3D34B-60E7-4AE2-98DE-68EFA2B35139@oracle.com> On 6 Jan 2016, at 22:32, Roger Riggs wrote: > Please review a few cleanups of the ProcessHandle tests. > - Switch to the jdk.test.lib (from lib/testlibrary ) > - Cleanup import statements > - Remove TEST.properties and use explicit @run testng argument in each test > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-testlibrary-8146574/ This looks good to me Roger. -Chris. > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146574 > > Thanks, Roger > From nadeesh.tv at oracle.com Thu Jan 7 09:20:17 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Thu, 07 Jan 2016 14:50:17 +0530 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: <568C0E60.2030008@Oracle.com> References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> Message-ID: <568E2DD1.5010900@oracle.com> Hi , Please see the updated webrev http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ Thanks and regards, Nadeesh On 1/6/2016 12:11 AM, Roger Riggs wrote: > Hi Nadeesh, > > LocalDate.java: +1374: > For the most common case of dom > 0 && <= 28, I would have > explicitly and immediately returned the new LocalDate. > if (dom > 0 && dom <= 28) { > return LocalDate.of(year, month, (int) dom); > } > ... > > TCKLocalDate.java: > - Since the test_plusDays_normal is being replaced, its test case > should be included in the DataProvider > {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} > Thanks, Roger > > On 1/5/2016 1:05 PM, nadeesh tv wrote: >> Hi all, >> >> Please review a fix for >> https://bugs.openjdk.java.net/browse/JDK-8068803 >> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >> >> Special thanks for Stephen for providing the source code patch >> > -- Thanks and Regards, Nadeesh TV From paul.sandoz at oracle.com Thu Jan 7 09:32:27 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 7 Jan 2016 10:32:27 +0100 Subject: use of Unsafe for ASCII detection In-Reply-To: References: Message-ID: <87C41750-1815-46AF-8D54-3199969F2CA2@oracle.com> Hi Martin, It?s possible to do that with a VarHandle, see the proposed MethodHandles.byteArrrayViewHandle factory method: http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/java/lang/invoke/MethodHandles-report.html#method:byteArrayViewVarHandle(java.lang.Class, boolean, boolean) I have not tested in your context but i would expect using a VarHandle should produce almost identical generated code as direct use of Unsafe, any bounds checks will be dominated by those of the loop bounds. If not it?s a bug! Some recent HotSpot compiler work by Roland should ensure that unrolled loops using Unsafe will be on par with direct array access. Once that gets rolled into jdk9/dev (i don?t think it is there yet) you might see another incremental improvement in performance. ? Longer term i think we can do better. A low-level, and safe, vectorization API would provide support for widths wider that 64 bits on supported platforms. A stream implementation utilising that API would be even better, so one could simply express it as Stream.of(bytes).allMatch(b -> b & 0x80 != 0), but that is much harder. Paul. > On 7 Jan 2016, at 00:21, Martin Buchholz wrote: > > Hi guys, > > It turns out that much of the world's computers is busy doing UTF-8 > decoding, so lots of people have expended much effort to make it > faster. Much of that text is ASCII, so it's definitely worth having > special ASCII-only loops. Like this: > > static boolean isAscii1(byte[] bytes) { > for (byte b : bytes) > if (b < 0) return false; > return true; > } > > Hotspot does a pretty good job on that, but we can go faster if we > reach for Unsafe, processing data a long at a time. > > static boolean isAscii2(byte[] bytes) { > long i, limit; > for (i = ABASE, limit = ABASE + bytes.length - 7; i < limit; i += 8) > if ((U.getLong(bytes, i) & 0x8080808080808080L) != 0) > return false; > for (limit = ABASE + bytes.length; i < limit; i += 1) > if (U.getByte(bytes, i) < 0) > return false; > return true; > } > > We can also try the same strategy with ByteBuffer.getLong, hoping it > will do atomic reads for us: > > static boolean isAscii3(byte[] bytes) { > ByteBuffer b = ByteBuffer.wrap(bytes); > int i, limit; > for (i = 0, limit = bytes.length - 7; i < limit; i += 8) > if ((b.getLong(i) & 0x8080808080808080L) != 0) > return false; > for (limit = bytes.length; i < limit; i += 1) > if (b.get(i) < 0) > return false; > return true; > } > > A stupid benchmark (sorry Aleksey - maybe you already have a jmh > version of this?) > http://cr.openjdk.java.net/~martin/Utf8Bench.java > gives me: > (jdk9) > Unsafe stride 8: 0.35626781581093897 > Buffer stride 8: 0.4825880453758097 > (jdk8) > Unsafe stride 8: 0.33818706070673143 > Buffer stride 8: 1.9958109701624323 > > That is, the Unsafe code is 3x faster than the simple code. The > ByteBuffer code used to be 2x slower and is now 2x faster - well done > - crowd goes wild! I see it uses new and well-hidden > Unsafe.getLongUnaligned ... all the performance fanatics want to use > that, but y'all don't want to give it to us. And we don't want to > allocate a temp ByteBuffer object. What to do? If you provided a > static variant of ByteBuffer.getLong for byte[], it would probably > have enough performance for us to live without Unsafe. It's hard to > resist Unsafe when it gives 3x improvement in a performance hotspot. > Maybe C2 can optimize the simple loop even more than it is today? > Maybe we can add a static method on j.u.Arrays? Maybe > > static int asciiCount(byte[] bytes, int start, int end) > > Still a performance hack, but far less of one. From aph at redhat.com Thu Jan 7 10:04:43 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 7 Jan 2016 10:04:43 +0000 Subject: use of Unsafe for ASCII detection In-Reply-To: References: Message-ID: <568E383B.8050201@redhat.com> On 06/01/16 23:21, Martin Buchholz wrote: > That is, the Unsafe code is 3x faster than the simple code. The > ByteBuffer code used to be 2x slower and is now 2x faster - well > done - crowd goes wild! Why, thank you. > I see it uses new and well-hidden Unsafe.getLongUnaligned ... all > the performance fanatics want to use that, but y'all don't want to > give it to us. And we don't want to allocate a temp ByteBuffer > object. What to do? If you allocate a temp ByteBuffer object carefully so that it does not escape, it will be removed and you should get the same code as directly using Unsafe. I certainly tested it during the work on Unsafe.getXXUnaligned and it was the same. I'll have a look at your example. Andrew. From uschindler at apache.org Thu Jan 7 10:30:06 2016 From: uschindler at apache.org (Uwe Schindler) Date: Thu, 7 Jan 2016 11:30:06 +0100 Subject: use of Unsafe for ASCII detection In-Reply-To: <568E383B.8050201@redhat.com> References: <568E383B.8050201@redhat.com> Message-ID: <013c01d14936$63c0b3d0$2b421b70$@apache.org> Hi, > On 06/01/16 23:21, Martin Buchholz wrote: > > > That is, the Unsafe code is 3x faster than the simple code. The > > ByteBuffer code used to be 2x slower and is now 2x faster - well > > done - crowd goes wild! > > Why, thank you. > > > I see it uses new and well-hidden Unsafe.getLongUnaligned ... all > > the performance fanatics want to use that, but y'all don't want to > > give it to us. And we don't want to allocate a temp ByteBuffer > > object. What to do? > > If you allocate a temp ByteBuffer object carefully so that it does not > escape, it will be removed and you should get the same code as > directly using Unsafe. I certainly tested it during the work on > Unsafe.getXXUnaligned and it was the same. I'll have a look at your > example. I think the benchmark is not good: It does not have a warmup time. So when code starts, ByteBuffer access is not yet fully optimized and inlined. So it is initially slower, because it takes longer until it is fully optimized. To fix the benchmark, use JMH or add a non-measured extra loop *before* each measurement to warm up compiler/optimizer. The results should then really be almost identical to Unsafe. The alternative is to try the new VarHandles code (see previous mails). We are looking at that for future Lucene developments around our ByteBufferIndexInput and access to other byte[] arrays as different type views. Uwe From Alan.Bateman at oracle.com Thu Jan 7 11:15:35 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 7 Jan 2016 11:15:35 +0000 Subject: 8049422: Remove @jdk.Exported Message-ID: <568E48D7.5030006@oracle.com> It's time to remove @jdk.Exported. As background, this annotation was introduced by JEP 179 in JDK 8 to document the supportness of JDK-specific APIs. It will become redundant and confusing once we bring the module system into JDK 9. The proposal in JEP 261 is to remove it and it seems saner to remove this in jdk9/dev in advance rather than trying to carry this in jigsaw/jake. The webrevs (top, jdk, langtools and nashorn repos) are here: http://cr.openjdk.java.net/~alanb/8049422/ The changes are trivial with all usages removed with find src -name "*.java" -exec sed -i '/@jdk.Exported/d' {} \; Skimming the .patch files should be enough to check for any blunders. The only "real" changes are the hg rm of Exported.java in the jdk repo, and the updates to modules.xml and the package list for the javadoc build in the top-level repo Note that jdeps has already been changed to not use this annotation so there are no jdeps changes required, only residual comments in one test. I think I got everything. Joe Darcy created this annotation and knows this removal is coming, I'll at least wait for Joe to review before pushing this change. -Alan. From aph at redhat.com Thu Jan 7 11:54:03 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 7 Jan 2016 11:54:03 +0000 Subject: use of Unsafe for ASCII detection In-Reply-To: <568E383B.8050201@redhat.com> References: <568E383B.8050201@redhat.com> Message-ID: <568E51DB.2090904@redhat.com> On 07/01/16 10:04, Andrew Haley wrote: > If you allocate a temp ByteBuffer object carefully so that it does not > escape, it will be removed and you should get the same code as > directly using Unsafe. I certainly tested it during the work on > Unsafe.getXXUnaligned and it was the same. I'll have a look at your > example. I tried your test. The ByteBuffer allocation is not being removed because it's not inlined: @ 1 java.nio.ByteBuffer::wrap (8 bytes) @ 4 java.nio.ByteBuffer::wrap (20 bytes) @ 7 java.nio.HeapByteBuffer:: (14 bytes) @ 10 java.nio.ByteBuffer:: (45 bytes) callee is too large Oops: we're really missing a trick. So, I'll try to increase the inline limit so that it's big enough to inline everything: I set it to 100 with -XX:MaxInlineSize=100 . The other thing is that you're doing a lot of byte-reversing because ByteBuffer.getLong() is always big-endian. Do this: ByteBuffer b = ByteBuffer.wrap(bytes).order(ByteOrder.nativeOrder()); I get: Unsafe stride 8: 0.22454525012941629 Buffer stride 8: 0.22788670497576385 Buffer stride 1: 1.0007217052423052 Yippee! :-) There are some lessons to be learned here. We really should try to inline ByteBuffer constructors to give escape analysis a chance. And if you want buffers in native byte order you have to ask for them. And finally, wrapped ByteBuffers can be a perfectly good substitute for Unsafe access to byte arrays. Andrew. From scolebourne at joda.org Thu Jan 7 12:45:56 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Thu, 7 Jan 2016 12:45:56 +0000 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: <568E2DD1.5010900@oracle.com> References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> Message-ID: I updated the benchmark with this change and another one: https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java Marginally fastest is this pattern as it avoids one if statement: if (dom > 0) { if (dom <= 28) { // same month return ... } if (dom <= 59) { // 59th Jan is 28th Feb return ... } } Rather than doing: return LocalDate.of(year, month, (int) dom); we can also do return new LocalDate(year, month, (int) dom); This is safe, because we know that the year, month and day are valid. (I can't easily test the performance of this change, but it should be noticeable as it avoids a lot of unnecessary checks). I'd like a few more test cases. Addition around 27/28/29/30 from the first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of Feb. Stephen On 7 January 2016 at 09:20, nadeesh tv wrote: > Hi , > Please see the updated webrev > http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ > Thanks and regards, > Nadeesh > > On 1/6/2016 12:11 AM, Roger Riggs wrote: > > Hi Nadeesh, > > LocalDate.java: +1374: > For the most common case of dom > 0 && <= 28, I would have explicitly and > immediately returned the new LocalDate. > > if (dom > 0 && dom <= 28) { > return LocalDate.of(year, month, (int) dom); > } > ... > > > TCKLocalDate.java: > - Since the test_plusDays_normal is being replaced, its test case should be > included in the DataProvider > > {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} > > Thanks, Roger > > On 1/5/2016 1:05 PM, nadeesh tv wrote: > > Hi all, > > Please review a fix for https://bugs.openjdk.java.net/browse/JDK-8068803 > web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ > > Special thanks for Stephen for providing the source code patch > > > > -- > Thanks and Regards, > Nadeesh TV > From daniel.fuchs at oracle.com Thu Jan 7 13:24:35 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 7 Jan 2016 14:24:35 +0100 Subject: 8049422: Remove @jdk.Exported In-Reply-To: <568E48D7.5030006@oracle.com> References: <568E48D7.5030006@oracle.com> Message-ID: <568E6713.8010509@oracle.com> Hi Alan, I looked at the jdk/ changes - these look good to me. I didn't spot any error in the sed result ;-) best regards, -- daniel On 07/01/16 12:15, Alan Bateman wrote: > > It's time to remove @jdk.Exported. > > As background, this annotation was introduced by JEP 179 in JDK 8 to > document the supportness of JDK-specific APIs. It will become redundant > and confusing once we bring the module system into JDK 9. The proposal > in JEP 261 is to remove it and it seems saner to remove this in jdk9/dev > in advance rather than trying to carry this in jigsaw/jake. > > The webrevs (top, jdk, langtools and nashorn repos) are here: > http://cr.openjdk.java.net/~alanb/8049422/ > > The changes are trivial with all usages removed with > find src -name "*.java" -exec sed -i '/@jdk.Exported/d' {} \; > > Skimming the .patch files should be enough to check for any blunders. > > The only "real" changes are the hg rm of Exported.java in the jdk repo, > and the updates to modules.xml and the package list for the javadoc > build in the top-level repo > > Note that jdeps has already been changed to not use this annotation so > there are no jdeps changes required, only residual comments in one test. > > I think I got everything. Joe Darcy created this annotation and knows > this removal is coming, I'll at least wait for Joe to review before > pushing this change. > > -Alan. From kubota.yuji at gmail.com Thu Jan 7 15:01:16 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Fri, 8 Jan 2016 00:01:16 +0900 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: <568D6553.4090108@oracle.com> References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> <568D6553.4090108@oracle.com> Message-ID: Hi Sherman, Thank you for sharing! 2016-01-07 4:04 GMT+09:00 Xueming Shen : > The reason that I'm not convinced that we really need a public interface of > ZipCryption here > is I don't know how useful/helpful/likely it would be going forward that > someone might really > use this interface to implement other encryption(s), especially the pkware > proprietary one, > I doubt it might be not that straightforward. In this proposal, we aim to support "traditional" because most people need it in secure environment. BTW, could you please share the reason why you did not support WinZip AES? Do you have a plan to support in the future? If you can share the reason, we want to decide the way of implementation with consideration for your information. I think we can implement by two way as below. 1. Implementing by reference to http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java This is good simply API. If we need to implement other encryption(s), try to refactor it. 2. Implementing with a package private interface of ZipCryption for next step. This has two problems as your advice. We agree with that the "encryption" and "compression" should be separated logically. However, current implementation compress the encrypted data, and buffering it. It is too tightly-coupled, so we need refactoring to separate the managing buffer processing and the stream processing of InflaterInputStream / DeflaterOutputStream. About "push back the bytes belong to next entry", we think InflaterInputStream.originBuf of our PoC do not required the needed info. Do this implements have problem? http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/src/java.base/share/classes/java/util/zip/InflaterInputStream.java.cdiff.html Thanks, Yuji > In fact I did have a draft implementation that supports WinZip AES about 5-6 > years ago :-) > (which also supports compression methods bzip and lzma, btw) Here is the > top class, It appears > a general interface might not be that helpful and it might be ideal to > simply implement it inside > the JDK, as what is proposed here, when it's really desired. > > http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java > > It is a ZipFile based implementation, so it does not have the headache that > ZipInputStream has, > such as to push back the bytes belong to next entry, since the loc might not > have the needed > info regarding the size/csize in stream mode. > > From abstract point of view. The "encryption" and "compression" are > different layers, it would > be ideal to have them in separate classes logically, instead of mixing the > encryption into > compression. Sure, it might be convenient and probably may have better > performance to mix > them in certain use scenario, but the "encryption" should never appear in > the public interface > of those compression classes. Package private interface should be fine, if > have to. > > -Sherman > > >> >> 2016-01-06 7:10 GMT+09:00 Xueming Shen : >>> >>> it appears that instead of adding "password" specific method to these >>> classes directly, it might be more appropriate to extend the ZipEntry >>> class >>> for such "password" functionality. For example, with a pair of new >>> methods >>> >>> boolean ZipEntry.isTraditionalEncryption(). >>> void ZipEntry.setTraditionalEncryption(String password); >> >> Thanks advice, I agree. We should re-design the API to extend the >> ZipEntry class. >> >>> The encryption support should/can be added naturally/smoothly with >>> ZipFile.getInputStream(e), ZipInputstream and >>> ZipOutputStream.putNextEntry(e), >>> with no extra new method in these two classes. The implementation checks >>> the flag (bit0, no bit 6) first and then verifies the password, as an >>> implementation details. >> >> Agree. For this proposal, we aim to support only traditional >> encryption. So I think we should also check bit 6. >> >>> For ZipFile and ZipInputStream, we can add note to the api doc to force >>> the >>> invoker to check if the returned ZipEntry indicates it's an encrypted >>> entry. >>> If yes, it must to set the appropriate password to the returned ZipEntry >>> via >>> ZipEntry.setTraditionalEncryption(password); before reading any byte from >>> the input stream. >> >> Yes, we have to add note the flow of codes to the JavaDoc. >> >>> Again, we should not have any "encryption" related public field/method in >>> DeflaterOutputStream/InflaterInputStream. Ideally these two classes >>> really >>> should not be aware of "encryption" at all. >> >> Agree, but I think we might be faced technical difficulty about a >> processing between zlib and the internal buffer of InflaterInputStream >> / DeflaterOutputStream. Please give us time to implement. >> >>> -Sherman >> >> Thanks, >> Yuji >> >> >>> On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: >>>> >>>> Hi Sherman and all, >>>> >>>> Happy new year to everyone! >>>> >>>> Please let know your feedback about this proposal. :-) >>>> >>>> Thanks, >>>> Yuji >>>> >>>> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >>>>> >>>>> Hi Sherman, >>>>> >>>>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>>>>> >>>>>> It is no longer necessary to touch the native code (zip_util.c/h) >>>>>> after >>>>>> the >>>>>> native ZipFile implementation has been moved up to the java level. >>>>>> Those >>>>>> native code are for vm access only now, which I dont think care about >>>>>> the >>>>>> password support at all. >>>>> >>>>> Thanks for your information. We do not take care the native. >>>>> >>>>> I discussed with Yasumasa, and our thought is as below. >>>>> >>>>>> (1) what's the benefit of exposing the public interface ZipCryption? >>>>>> the >>>>>> real >>>>>> question is whether or not this interface is good enough for other >>>>>> encryption >>>>>> implementation to plugin their implementation to support the >>>>>> ZipFile/Input/ >>>>>> OutputStream to their encryption spec. >>>>> >>>>> We aimed that the public interface ZipCryption supports the >>>>> extensibillity for other encrypt engine. The JDK core libs developers >>>>> have to implementation ZipyCryption only. If not provide, the JDK >>>>> developers must implement ZipStream/Entry by JDK API to design the >>>>> data structure of entry. >>>>> If you want to use binary key data such as PKI, you can implement new >>>>> encrypt/decrypt engine by ZipCryption interface. >>>>> So we think we should provide this interface to be clearly how to >>>>> implement a new engine, e.g., cipher algorithm, cipher strength and >>>>> converting the header, etc. >>>>> >>>>>> (2) it seems like it might be possible to hide most of the >>>>>> implementation >>>>>> and only expose the "String password" (instead of the ZipCryption) as >>>>>> the >>>>>> public interface to support the "traditional" encryption. This depends >>>>>> on the >>>>>> result of (1) though. >>>>> >>>>> Thanks for your clues. We think the string password at first. However, >>>>> we should also create a new binary interface given we support PKI in >>>>> the future. >>>>> >>>>>> (3) I'm concerned of pushing ZipCryption into >>>>>> InflaterInputStream/DeflaterOutputStream. >>>>>> It might be worth considering to replace the ZipCryption >>>>>> implementation >>>>>> with >>>>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>>>> use >>>>>> the FilterOutputStream for the ZipOutputStream and the >>>>>> FilterInputStream >>>>>> for the >>>>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue >>>>>> ... >>>>> >>>>> Thanks for your clues, too. Honestly speaking, we think the current >>>>> zip implementation may break the data when used PushbackInputStream >>>>> for the following reasons. >>>>> >>>>> * PushbackInputStream uses an unique internal buffer for re-read >>>>> operation. >>>>> * But, InflaterInputStream provide date to Inflater per reads and >>>>> buffer by JNI (zlib). >>>>> * So we think PushbackInputStream is poor compatibility with >>>>> InflaterInputStream. >>>>> >>>>> We generally use InputStream through ZipEntry#getInputStream(). We do >>>>> not touch FileInputStream for reading ZIP data. If we call unread() >>>>> when we use PushbackInputStream as reading ZIP archive, we guess that >>>>> it will break the reading data. >>>>> So, our approach do not affect the PushbackInputStream. >>>>> What do you think about this? >>>>> >>>>>> (4) It seems the ZipOutputStream only supports the "stream based" >>>>>> password, while >>>>>> the ZipInputStream supports the "entry based" password. Do we really >>>>>> need >>>>>> "entry based" support here? >>>>> >>>>> As your suggestion, we should support "entry based". We will start to >>>>> implement "entry based" after this discussion is closed. >>>>> >>>>> Thanks, >>>>> Yuji >>>>> >>>>>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>>>>> >>>>>>> Hi Jason, >>>>>>> >>>>>>> Thank you for your comment. >>>>>>> I've fixed it in new webrev: >>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Yasumasa >>>>>>> >>>>>>> >>>>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>>>>> >>>>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is >>>>>>>> in >>>>>>>> conflict with line 350 and 348. >>>>>>>> >>>>>>>> ________________________________________ >>>>>>>> From: core-libs-dev on >>>>>>>> behalf >>>>>>>> of >>>>>>>> Yasumasa Suenaga >>>>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>>>> protection to Zip entries >>>>>>>> >>>>>>>> Hi Sergey, >>>>>>>> >>>>>>>> Thank you for your comment. >>>>>>>> >>>>>>>> I added that description in new webrev: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>>>>> >>>>>>>>> Should the new methods describe how they will work in case of null >>>>>>>>> params? >>>>>>>>> >>>>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>>>> >>>>>>>>>> Could you review it? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>>>>> >>>>>>>>>>> Hi Sherman, >>>>>>>>>>> >>>>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>>>> jdk9/dev >>>>>>>>>>> repos. >>>>>>>>>>> >>>>>>>>>>> Do you have any comments about current webrev? >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>> >>>>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Sherman, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for your quick response :) >>>>>>>>>>>> >>>>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>>>> below >>>>>>>>>>>> reasons. >>>>>>>>>>>> >>>>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>>>> like a >>>>>>>>>>>> temporary file. >>>>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>>>> best >>>>>>>>>>>> for >>>>>>>>>>>> openjdk. >>>>>>>>>>>> * PKWare claims that they have patents about the >>>>>>>>>>>> "stronger" >>>>>>>>>>>> on >>>>>>>>>>>> Zip[1]. >>>>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>>>> "stronger" [2][3]. >>>>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>>>> interface >>>>>>>>>>>> to >>>>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>>>> >>>>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>>>> easily >>>>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>>>> >>>>>>>>>>>> [1] >>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption >>>>>>>>>>>> Specification) >>>>>>>>>>>> >>>>>>>>>>>> [2] >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>>>> >>>>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Yuji >>>>>>>>>>>> >>>>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming >>>>>>>>>>>> Shen: >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Yuji, >>>>>>>>>>>>> >>>>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>>>> bring >>>>>>>>>>>>> in the >>>>>>>>>>>>> security guy >>>>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>>>> in >>>>>>>>>>>>> the >>>>>>>>>>>>> "traditional PKWare >>>>>>>>>>>>> decryption", which is, based on the wiki, "known to be >>>>>>>>>>>>> seriously >>>>>>>>>>>>> flawed, and >>>>>>>>>>>>> in particular >>>>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>>>> support >>>>>>>>>>>>> "stronger" encryption >>>>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>>>> >>>>>>>>>>>>> Regards, >>>>>>>>>>>>> Sherman >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>> >>>>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>>>> before >>>>>>>>>>>>>>> get >>>>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>>>> However, >>>>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we >>>>>>>>>>>>>>> must >>>>>>>>>>>>>>> use >>>>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>>> The >>>>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has >>>>>>>>>>>>>>> been >>>>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>>>> zlib >>>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Test code is as below. This code will let you know >>>>>>>>>>>>>>> how >>>>>>>>>>>>>>> this >>>>>>>>>>>>>>> PoC >>>>>>>>>>>>>>> works. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are >>>>>>>>>>>>>>> providing >>>>>>>>>>>>>>> many >>>>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>>>> implement to >>>>>>>>>>>>>>> handle password-protected zip file. I guess that this >>>>>>>>>>>>>>> proposal >>>>>>>>>>>>>>> is >>>>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1]: >>>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> > From joe.darcy at oracle.com Thu Jan 7 16:51:31 2016 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 7 Jan 2016 08:51:31 -0800 Subject: JDK 9 RFR of JDK-8146587: Mark tools/pack200/Pack200Test.java as intermittently failing In-Reply-To: <568E201C.4080707@oracle.com> References: <568E201C.4080707@oracle.com> Message-ID: <568E9793.7010806@oracle.com> Look good; cheers, -Joe On 1/7/2016 12:21 AM, Amy Lu wrote: > Test tools/pack200/Pack200Test.java is known to fail intermittently, > this patch is to mark the test accordingly with keyword 'intermittent'. > > bug: https://bugs.openjdk.java.net/browse/JDK-8146587 > webrev: http://cr.openjdk.java.net/~amlu/8146587/webrev.00/ > > Thanks, > Amy > > --- old/test/tools/pack200/Pack200Test.java 2016-01-07 > 11:07:47.000000000 +0800 > +++ new/test/tools/pack200/Pack200Test.java 2016-01-07 > 11:07:47.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2007, 2014, Oracle and/or its affiliates. All rights > reserved. > + * Copyright (c) 2007, 2016, Oracle and/or its affiliates. All rights > reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -31,6 +31,7 @@ > /* > * @test > * @bug 6521334 6712743 8007902 > + * @key intermittent > * @summary check for memory leaks, test general packer/unpacker > functionality\ > * using native and java unpackers > * @compile -XDignore.symbol.file Utils.java Pack200Test.java > > From huizhe.wang at oracle.com Thu Jan 7 17:22:17 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 07 Jan 2016 09:22:17 -0800 Subject: RFR (JAXP): 8144966: Catalog API: Null handling and reference to Reader In-Reply-To: <8BA66C26-8D4A-45B8-9BF9-6DB30B37F11A@oracle.com> References: <567B264E.1020502@oracle.com> <27D13FC1-CF10-4DDD-8F63-1D70FE8DBAE4@oracle.com> <568DAD28.50204@oracle.com> <8BA66C26-8D4A-45B8-9BF9-6DB30B37F11A@oracle.com> Message-ID: <568E9EC9.9060708@oracle.com> On 1/6/2016 4:53 PM, Mandy Chung wrote: >> On Jan 6, 2016, at 4:11 PM, huizhe wang wrote: >> >> >>> What if ?paths? is non-empty but all entries are invalid and no catalog file is read - will it read the system property? >> It will not read the system property. The assumption is that the paths specified are meant to be used. If they are invalid, it would indicate programming error. I added a couple statements to make it clearer. > > The new statement: > "In case all entries are invalid, the resolver will return as no mapping is found.? > > It will return an object with no catalog file, right? Is it equivalent to calling with empty paths and the system property javax.xml.catalog.files is not set? The form of the Source returned will depend on the setting of the property "javax.xml.catalog.resolve" as specified by the resolvers when no mapping is found. > > I think this sentence is not necessary and can take it out. > "If {@code paths} is specified, no attempt will be made to read the system property even if all entries are invalid or point to invalid catalogs." Removed. > > Otherwise the updated webrev looks fine. Thanks, Joe > > Mandy From mandy.chung at oracle.com Thu Jan 7 18:42:02 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 7 Jan 2016 10:42:02 -0800 Subject: 8049422: Remove @jdk.Exported In-Reply-To: <568E48D7.5030006@oracle.com> References: <568E48D7.5030006@oracle.com> Message-ID: <9A76AF15-A631-4524-A10F-3BD2C1666A08@oracle.com> > On Jan 7, 2016, at 3:15 AM, Alan Bateman wrote: > > > It's time to remove @jdk.Exported. > > As background, this annotation was introduced by JEP 179 in JDK 8 to document the supportness of JDK-specific APIs. It will become redundant and confusing once we bring the module system into JDK 9. The proposal in JEP 261 is to remove it and it seems saner to remove this in jdk9/dev in advance rather than trying to carry this in jigsaw/jake. > > The webrevs (top, jdk, langtools and nashorn repos) are here: > http://cr.openjdk.java.net/~alanb/8049422/ I looked through this change and I think you got everything. Removing @jdk.Exported is okay. Supported JDK-specific API is currently documented in modules.xml as an interim solution and it will be replaced the module definitions when the module system is in. Mandy From cushon at google.com Thu Jan 7 18:59:25 2016 From: cushon at google.com (Liam Miller-Cushon) Date: Thu, 7 Jan 2016 10:59:25 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: Message-ID: Hi Joel - On Tue, Jan 5, 2016 at 1:16 PM, Joel Borggr?n-Franck < joel.borggren.franck at gmail.com> wrote: > I think you need to do skipMemberValue the correct number of times so > that the cursor in buf is correct, or? > Thanks for the catch. I added your test case, and the exception proxy is now stored and returned after the entire array is read. > Also, in MissingArrayElementTest, why not just rethrow t? > Done. Thanks! -------------- next part -------------- A non-text attachment was scrubbed... Name: 7183985.2.patch Type: text/x-patch Size: 17369 bytes Desc: not available URL: From marcins at microsoft.com Thu Jan 7 19:48:39 2016 From: marcins at microsoft.com (Martin Sawicki) Date: Thu, 7 Jan 2016 19:48:39 +0000 Subject: RFR 8124977 cmdline encoding challenges on Windows In-Reply-To: References: <558C017A.2020708@oracle.com> <5651E9AA.9060908@oracle.com> Message-ID: Hi Kumar, just wondering if there are any updates on processing this submission. Thanks! -----Original Message----- From: Vladimir Shcherbakov Sent: Wednesday, November 25, 2015 2:38 PM To: Kumar Srinivasan ; Martin Sawicki Cc: Kirk Shoop ; core-libs-dev Libs Subject: RE: RFR 8124977 cmdline encoding challenges on Windows Hi Kumar, Please find updated webreview here: http://cr.openjdk.java.net/~kshoop/8124977/webrev.04/ Thanks, Vladimir. -----Original Message----- From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] Sent: Sunday, November 22, 2015 8:14 AM To: Martin Sawicki Cc: Kirk Shoop ; Vladimir Shcherbakov ; core-libs-dev Libs Subject: Re: RFR 8124977 cmdline encoding challenges on Windows Hi Martin, et. al., Sorry for not getting back earlier, I am very busy right now with my other large commitments for JDK9. I will sponsor this "enhancement/bug fix" sometime in the new year, meanwhile, there is the changeset [1] which is likely to cause merge conflicts, and perhaps logic issues. Thanks Kumar [1] https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.openjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7Mg4lt2P0sdzLS%2fT9Q%3d > Hi all > Here's an updated webrev attempting to take into account the various pieces of feedback we have received: > > Issue: > > https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. > openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40micro > soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7c > d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E%3 > d > Webrev: > > https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.openj > dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashch% > 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af9 > 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJnWY > 0%3d > > (Vladimir Shcherbakov is now working on this from our side) > > Looking forward to any other feedback. > Thanks > > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Kumar Srinivasan > Sent: Thursday, June 25, 2015 6:26 AM > To: Kirk Shoop (MS OPEN TECH) > Cc: Valery Kopylov (Akvelon) ; core-libs-dev > Libs > Subject: Re: RFR 8124977 cmdline encoding challenges on Windows > > Hi Kirk, > > Thanks for proposing this change. > > If you notice all the posix calls are wrapped in JLI_* this gives us the ability to use "W" functions. I almost got it done, several years ago, but we upgraded to VS2010 and my work based on VS2003 keeled over, meanwhile my focus was "shifted" to something else. > > main.c: is really envisioned to be a stub compiled by the tool > launchers, like java, javac, javah, jar etc. I prefer to see all the > heavy logic in this file moved to the platform specific file > windows/java_md.* > > For the reason specified above we need to move fprintf or any naked posix calls to JLI_* indirections. > > I don't see any tests ? The tests must be written in java and placed in jdk/test/tools/launcher, there is a helper framework TestHelper.java. > > There are other changes in nio, charsets etc, this will be reviewed by my colleague specializing in that area (Sherman) cc'ed. > > > Thanks > > Kumar > > > > > > > On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote: >> Hi, >> >> Issue: >> >> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs >> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic >> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2 >> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq >> 9E%3d >> >> Webrev: >> >> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open >> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40microsof >> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7cd0 >> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M%3d >> >> This webrev intends to address interaction between Windows console and java apps. >> >> Two switches were added that change the behavior of the launcher. The defaults do not change the launcher behavior. >> >> -Dwindows.UnicodeConsole=true - switches on Unicode support in the Windows console. This optional switch causes the launcher to call GetCommandLineW() and parse the arguments in unicode. It also modifies how the codepage for console output is selected. >> >> -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset to use; If not specified, UTF-8 is used by default. Ignored when windows.UnicodeConsole is not set to true. When the first switch is used, this optional switch allows the codepage for console output to be controlled. >> >> I would like to get feedback on the approach here and any additional work that is required solve these particular Unicode issues on Windows. >> >> Kirk From snfuchs at gmx.de Thu Jan 7 07:21:37 2016 From: snfuchs at gmx.de (Stefan Fuchs) Date: Thu, 7 Jan 2016 08:21:37 +0100 Subject: Backport of JDK-8041501? Message-ID: <568E1201.4010005@gmx.de> Hi, I don't know, if this is the correct mailinglist, but one of our customers was recently hit by https://bugs.openjdk.java.net/browse/JDK-8041501. Would it be possible to backport the fix to Java 8 ? Regards, Stefan From kishor.kharbas at intel.com Thu Jan 7 21:32:57 2016 From: kishor.kharbas at intel.com (Kharbas, Kishor) Date: Thu, 7 Jan 2016 21:32:57 +0000 Subject: RFR (M): 8135250: Replace custom check/range functionality with check index/range methods in java.util.Arrays In-Reply-To: <7CBCE02C-DA05-4C2B-B687-28DB5CD9C4FD@oracle.com> References: <84E6E74C-CE18-49DE-828B-A7E282904EEF@oracle.com> <7CBCE02C-DA05-4C2B-B687-28DB5CD9C4FD@oracle.com> Message-ID: Thanks Paul for reviewing it. And thanks for offering to create a issue, I myself don't have the permissions on JBS, so I can use your help :-) For now IndexOutOfBoundsException looks ok; like you said I will revisit this functionality when exception message reported by the API is changed. Thanks, Kishor -----Original Message----- From: Paul Sandoz [mailto:paul.sandoz at oracle.com] Sent: Thursday, January 07, 2016 12:28 AM To: Kharbas, Kishor Cc: Anthony Scarpino; core-libs-dev at openjdk.java.net Subject: Re: RFR (M): 8135250: Replace custom check/range functionality with check index/range methods in java.util.Arrays > On 7 Jan 2016, at 09:15, Paul Sandoz wrote: > > Hi Kishor, > > Looks good to me. A small but nice improvement. > > I guess you are ok with these throwing IndexOutOfBounds rather than ArrayIndexOutOfBounds. > Actually i just realised you are reusing issue JDK-8135250, which is logged to track a wider range of identified areas in the JDK. Would it be possible to create a new issue [*] related to JDK-8135250 just for the crypto stuff. Paul. [*] If you don?t have the "power ups? to do that i can create one for you. > ? > > Note that i am going to revisit the exception message reporting and with nudging by John the BiFunction to ensure no bounds information is lost. It should not affect this code, but should produce better error messages, and after it might be an opportunity to revisit if you want these to throw ArrayIndexOutOfBounds. > > Paul. > >> On 7 Jan 2016, at 01:18, Kharbas, Kishor wrote: >> >> Hello all, >> >> Requesting the community to review a patch which replaces custom bound checking functionality with calls to methods in java.util.Objects. >> >> The methods affected are CounterMode.crypt() and AESCrypt.encryptBlock(), AESCrypt.decryptBlock() >> >> >> Bug id: >> https://bugs.openjdk.java.net/browse/JDK-8135250 >> Webrev: >> http://cr.openjdk.java.net/~vdeshpande/8135250/webrev.00/ >> >> Regards >> Kishor Kharbas > From martinrb at google.com Thu Jan 7 22:53:23 2016 From: martinrb at google.com (Martin Buchholz) Date: Thu, 7 Jan 2016 14:53:23 -0800 Subject: Backport of JDK-8041501? In-Reply-To: <568E1201.4010005@gmx.de> References: <568E1201.4010005@gmx.de> Message-ID: I suspect 2d-dev is the right mailing list. On Wed, Jan 6, 2016 at 11:21 PM, Stefan Fuchs wrote: > Hi, > > I don't know, if this is the correct mailinglist, but one of our customers > was recently hit by > https://bugs.openjdk.java.net/browse/JDK-8041501. > > Would it be possible to backport the fix to Java 8 ? > > Regards, > Stefan From brent.christian at oracle.com Fri Jan 8 00:18:25 2016 From: brent.christian at oracle.com (Brent Christian) Date: Thu, 07 Jan 2016 16:18:25 -0800 Subject: RFR 9: JDK-8144552 : java/lang/StackWalker/LocalsAndOperands.java fails with java.lang.NullPointerException Message-ID: <568F0051.4000305@oracle.com> Hi, Please review my change for 8144552 : java/lang/StackWalker/LocalsAndOperands.java fails with java.lang.NullPointerException Bug: https://bugs.openjdk.java.net/browse/JDK-8144552 Webrev: http://cr.openjdk.java.net/~bchristi/8144552/webrev.0/ It's possible for the LiveStackFrame.getLocals() array to contain null entries, in particular in the case of "dead" local variables. The (package-private) docs are being updated to state this. The test case is updated to keep the locals in the test() method alive (and checked for non-null), and otherwise be tolerant of nulls. Thanks, -Brent From joe.darcy at oracle.com Fri Jan 8 03:12:44 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Thu, 07 Jan 2016 19:12:44 -0800 Subject: 8049422: Remove @jdk.Exported In-Reply-To: <568E48D7.5030006@oracle.com> References: <568E48D7.5030006@oracle.com> Message-ID: <568F292C.3030807@oracle.com> Looks fine Alan; cheers, -Joe On 1/7/2016 3:15 AM, Alan Bateman wrote: > > It's time to remove @jdk.Exported. > > As background, this annotation was introduced by JEP 179 in JDK 8 to > document the supportness of JDK-specific APIs. It will become > redundant and confusing once we bring the module system into JDK 9. > The proposal in JEP 261 is to remove it and it seems saner to remove > this in jdk9/dev in advance rather than trying to carry this in > jigsaw/jake. > > The webrevs (top, jdk, langtools and nashorn repos) are here: > http://cr.openjdk.java.net/~alanb/8049422/ > > The changes are trivial with all usages removed with > find src -name "*.java" -exec sed -i '/@jdk.Exported/d' {} \; > > Skimming the .patch files should be enough to check for any blunders. > > The only "real" changes are the hg rm of Exported.java in the jdk > repo, and the updates to modules.xml and the package list for the > javadoc build in the top-level repo > > Note that jdeps has already been changed to not use this annotation so > there are no jdeps changes required, only residual comments in one test. > > I think I got everything. Joe Darcy created this annotation and knows > this removal is coming, I'll at least wait for Joe to review before > pushing this change. > > -Alan. From mandy.chung at oracle.com Fri Jan 8 03:55:44 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 7 Jan 2016 19:55:44 -0800 Subject: RFR 9: JDK-8144552 : java/lang/StackWalker/LocalsAndOperands.java fails with java.lang.NullPointerException In-Reply-To: <568F0051.4000305@oracle.com> References: <568F0051.4000305@oracle.com> Message-ID: <12C85E69-11CC-4437-87A2-9814CEC651EC@oracle.com> > On Jan 7, 2016, at 4:18 PM, Brent Christian wrote: > > Hi, > > Please review my change for > > 8144552 : java/lang/StackWalker/LocalsAndOperands.java fails with java.lang.NullPointerException > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8144552 > Webrev: > http://cr.openjdk.java.net/~bchristi/8144552/webrev.0/ Looks okay. Thanks Mandy From chris.hegarty at oracle.com Fri Jan 8 06:08:35 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 8 Jan 2016 06:08:35 +0000 Subject: 8049422: Remove @jdk.Exported In-Reply-To: <568E48D7.5030006@oracle.com> References: <568E48D7.5030006@oracle.com> Message-ID: <0E23E72E-8360-4CBB-BC5A-610FC94CD2F4@oracle.com> On 7 Jan 2016, at 11:15, Alan Bateman wrote: > > It's time to remove @jdk.Exported. > > As background, this annotation was introduced by JEP 179 in JDK 8 to document the supportness of JDK-specific APIs. It will become redundant and confusing once we bring the module system into JDK 9. The proposal in JEP 261 is to remove it and it seems saner to remove this in jdk9/dev in advance rather than trying to carry this in jigsaw/jake. > > The webrevs (top, jdk, langtools and nashorn repos) are here: > http://cr.openjdk.java.net/~alanb/8049422/ The change looks good to me Alan. -Chris. > The changes are trivial with all usages removed with > find src -name "*.java" -exec sed -i '/@jdk.Exported/d' {} \; > > Skimming the .patch files should be enough to check for any blunders. > > The only "real" changes are the hg rm of Exported.java in the jdk repo, and the updates to modules.xml and the package list for the javadoc build in the top-level repo > > Note that jdeps has already been changed to not use this annotation so there are no jdeps changes required, only residual comments in one test. > > I think I got everything. Joe Darcy created this annotation and knows this removal is coming, I'll at least wait for Joe to review before pushing this change. > > -Alan. From daniel.fuchs at oracle.com Fri Jan 8 09:23:09 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 8 Jan 2016 10:23:09 +0100 Subject: RFR 9: JDK-8144552 : java/lang/StackWalker/LocalsAndOperands.java fails with java.lang.NullPointerException In-Reply-To: <568F0051.4000305@oracle.com> References: <568F0051.4000305@oracle.com> Message-ID: <568F7FFD.80507@oracle.com> On 1/8/16 1:18 AM, Brent Christian wrote: > Hi, > > Please review my change for > > 8144552 : java/lang/StackWalker/LocalsAndOperands.java fails with > java.lang.NullPointerException > > Bug: > https://bugs.openjdk.java.net/browse/JDK-8144552 > Webrev: > http://cr.openjdk.java.net/~bchristi/8144552/webrev.0/ > > It's possible for the LiveStackFrame.getLocals() array to contain null > entries, in particular in the case of "dead" local variables. > > The (package-private) docs are being updated to state this. The test > case is updated to keep the locals in the test() method alive (and > checked for non-null), and otherwise be tolerant of nulls. Hi Brent, Looks good to me as well. Cheers, -- daniel > > Thanks, > -Brent From nadeesh.tv at oracle.com Fri Jan 8 10:56:39 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Fri, 08 Jan 2016 16:26:39 +0530 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> Message-ID: <568F95E7.9030304@oracle.com> Hi all, Thanks Stephen for the comments Please see the updated webrev http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ Regards, Nadeesh On 1/7/2016 6:15 PM, Stephen Colebourne wrote: > I updated the benchmark with this change and another one: > https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java > > Marginally fastest is this pattern as it avoids one if statement: > > if (dom > 0) { > if (dom <= 28) { // same month > return ... > } > if (dom <= 59) { // 59th Jan is 28th Feb > return ... > } > } > > Rather than doing: > return LocalDate.of(year, month, (int) dom); > we can also do > return new LocalDate(year, month, (int) dom); > > This is safe, because we know that the year, month and day are valid. > (I can't easily test the performance of this change, but it should be > noticeable as it avoids a lot of unnecessary checks). > > I'd like a few more test cases. Addition around 27/28/29/30 from the > first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of > Feb. > > Stephen > > > > On 7 January 2016 at 09:20, nadeesh tv wrote: >> Hi , >> Please see the updated webrev >> http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ >> Thanks and regards, >> Nadeesh >> >> On 1/6/2016 12:11 AM, Roger Riggs wrote: >> >> Hi Nadeesh, >> >> LocalDate.java: +1374: >> For the most common case of dom > 0 && <= 28, I would have explicitly and >> immediately returned the new LocalDate. >> >> if (dom > 0 && dom <= 28) { >> return LocalDate.of(year, month, (int) dom); >> } >> ... >> >> >> TCKLocalDate.java: >> - Since the test_plusDays_normal is being replaced, its test case should be >> included in the DataProvider >> >> {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} >> >> Thanks, Roger >> >> On 1/5/2016 1:05 PM, nadeesh tv wrote: >> >> Hi all, >> >> Please review a fix for https://bugs.openjdk.java.net/browse/JDK-8068803 >> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >> >> Special thanks for Stephen for providing the source code patch >> >> >> >> -- >> Thanks and Regards, >> Nadeesh TV >> -- Thanks and Regards, Nadeesh TV From scolebourne at joda.org Fri Jan 8 11:56:39 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 8 Jan 2016 11:56:39 +0000 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: <568F95E7.9030304@oracle.com> References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> <568F95E7.9030304@oracle.com> Message-ID: As I mentioned in my email: Rather than doing: return withDayOfMonth((int) dom); or return LocalDate.of(year, month, (int) dom); you can do return new LocalDate(year, month, (int) dom); (there are two occurrences) Stephen On 8 January 2016 at 10:56, nadeesh tv wrote: > Hi all, > > Thanks Stephen for the comments > Please see the updated webrev > http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ > Regards, > Nadeesh > > On 1/7/2016 6:15 PM, Stephen Colebourne wrote: >> >> I updated the benchmark with this change and another one: >> >> https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java >> >> Marginally fastest is this pattern as it avoids one if statement: >> >> if (dom > 0) { >> if (dom <= 28) { // same month >> return ... >> } >> if (dom <= 59) { // 59th Jan is 28th Feb >> return ... >> } >> } >> >> Rather than doing: >> return LocalDate.of(year, month, (int) dom); >> we can also do >> return new LocalDate(year, month, (int) dom); >> >> This is safe, because we know that the year, month and day are valid. >> (I can't easily test the performance of this change, but it should be >> noticeable as it avoids a lot of unnecessary checks). >> >> I'd like a few more test cases. Addition around 27/28/29/30 from the >> first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of >> Feb. >> >> Stephen >> >> >> >> On 7 January 2016 at 09:20, nadeesh tv wrote: >>> >>> Hi , >>> Please see the updated webrev >>> http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ >>> Thanks and regards, >>> Nadeesh >>> >>> On 1/6/2016 12:11 AM, Roger Riggs wrote: >>> >>> Hi Nadeesh, >>> >>> LocalDate.java: +1374: >>> For the most common case of dom > 0 && <= 28, I would have explicitly >>> and >>> immediately returned the new LocalDate. >>> >>> if (dom > 0 && dom <= 28) { >>> return LocalDate.of(year, month, (int) dom); >>> } >>> ... >>> >>> >>> TCKLocalDate.java: >>> - Since the test_plusDays_normal is being replaced, its test case >>> should be >>> included in the DataProvider >>> >>> {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} >>> >>> Thanks, Roger >>> >>> On 1/5/2016 1:05 PM, nadeesh tv wrote: >>> >>> Hi all, >>> >>> Please review a fix for >>> https://bugs.openjdk.java.net/browse/JDK-8068803 >>> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >>> >>> Special thanks for Stephen for providing the source code patch >>> >>> >>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> > > -- > Thanks and Regards, > Nadeesh TV > From daniel.fuchs at oracle.com Fri Jan 8 15:36:52 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 8 Jan 2016 16:36:52 +0100 Subject: RFR: 8146665: Test jdk/test/java/util/logging/LogManager/Configuration/updateConfiguration/UpdateConfigurationTest.java fails - missing expected output Message-ID: <568FD794.7030200@oracle.com> Hi, Please find below a tentative fix for: 8146665: Test jdk/test/java/util/logging/LogManager/Configuration/updateConfiguration/UpdateConfigurationTest.java fails - missing expected output https://bugs.openjdk.java.net/browse/JDK-8146665 Webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8146665/webrev.00/ The fix increases the numbers of gc performed (which may or may not fix the issue), but more importantly fixes the finally { } clause of the test(...) method, where a failure in releasing the second logger reference may be hiding the original failure. best regards, -- daniel From Roger.Riggs at Oracle.com Fri Jan 8 15:51:58 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 8 Jan 2016 10:51:58 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568D64C8.7030401@gmail.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> <568D64C8.7030401@gmail.com> Message-ID: <568FDB1E.4050300@Oracle.com> Thanks for the review and comments: While integrating the comments, some comments identified finalizers that were unneeded should be removed. I split that into a separate bug, as they are not being replaced, just removed. [2] The CleaningFactory is moved to jdk.internal.ref so it can be exported selectively to specific OpenJDK modules. The suggested refactoring of the subclassable XXXCleanables can be done separately if it turns out to be useful outside the base module. Webrev for shared CleaningFactory and its uses: http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ Webrev for removed finalizers: http://cr.openjdk.java.net/~rriggs/webrev-rmfinalizers-8146567/ Thanks, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8146028 Common Cleaner for finalization replacements in OpenJDK [2] https://bugs.openjdk.java.net/browse/JDK-8146567 Remove dead code finalizer methods On 1/6/2016 2:02 PM, Peter Levart wrote: > > > On 01/05/2016 10:16 PM, Roger Riggs wrote: >> Hi Daniel, >> >> webrev updated to revert changes to MeteredStream as too risky. >> >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html > > > Hi Roger, > > I briefly skimmed over the webrev, and found the following issue in > ProcessImpl: > > 420 // Register a cleaning function to close the handle > 421 CleanerFactory.getCleaner().register(this, () -> > closeHandle(handle)); > 422 > > ... 'handle' is an instance variable, which means that 'this' is > captured by the lambda. You will have to assign handle to a local var > and capture it instead to prevent 'this' to be captured. > > Regards, Peter > > >> >> On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >>> Hi Roger, >>> >>> Some early feedback: >>> >>> I see that prior to your changes, MeteredStream.close() would >>> be called by finalize. >>> This will no longer be the case after >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>> >> In the case where finalize would call close(), there can be no >> queuedForCleanup activity >> because there can be no strong reference from the >> KeepAliveStreamCleaner thread >> or the queue that holds the pending cleanups. >> It cannot be in the Cache of connections being kept alive or the >> thread that keeps them alive. >> >> It might be possible that the underlying connection is still open; >> but there is no advantage >> of trying to drain it. There is the possibility of resurrecting it >> by virtue of deciding that >> it should be queued for cleanup and starting a new thread to do the >> cleanup. >> >> All in all, there is a risk of misunderstanding the dynamic behavior >> and it would be safer >> to leave this using finalize. >> >>> >>> I see that MeteredStream has a subclass (KeepAliveStream) that >>> overrides close() - so KeepAliveStream probably requires a cleanup >>> as well - doesn't it? >>> >>> Finally I'd suggest making the variable 'pi' final - since you're >>> passing that to the lambda you don't want to allow its value to >>> be modified afterwards (fortunately it's not - and I believe it >>> should be good to ensure it won't be). >> ok, it would not go unnoticed, the compiler should complain if the >> variable is no effectively final. >> >> Thanks, Roger >> >>> >>> I haven't looked at the other classes yet... >>> >>> best regards, >>> >>> -- daniel >>> >>> On 05/01/16 19:24, Roger Riggs wrote: >>>> The follow on work to adding the Cleaner is to replace uses of >>>> finalization with uses of the Cleaner. >>>> For the 'easy' cases in the java.base module, it is useful to >>>> introduce >>>> a private Cleaner for the >>>> java.base module. It is proposed to be held weakly, to allow it to >>>> terminate on a lightly loaded >>>> system. >>>> >>>> Webrev for Review: >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>>> >>>> >>>> The 2nd step is using the Cleaner. >>>> - Empty finalize methods should (I think) be removed; but since they >>>> are part of the public spec >>>> the process needs two full releases; so the proposal is to >>>> deprecate >>>> them first. >>>> (The JEP 277 necessary changes will be updated when JEP 277 >>>> semantics are finalized) >>>> >>>> - In a few cases, the code in the finalizer is redundant and if >>>> removed >>>> would allow >>>> an optimization of the handling of the finalizer and future removal >>>> of the finalize method. >>>> >>>> Webrev: >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>> >>>> >>>> Thanks for comments and suggestions, Roger >>>> >>>> >>>> >>>> >>> >> > From daniel.fuchs at oracle.com Fri Jan 8 15:51:54 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 8 Jan 2016 16:51:54 +0100 Subject: RFR: 8146691: @since 9 tag missing for System.getLogger Message-ID: <568FDB1A.7060703@oracle.com> Hi, Please find below a trivial fix for 8146691: @since 9 tag missing for System.getLogger https://bugs.openjdk.java.net/browse/JDK-8146691 best regards, -- daniel --- old/src/java.base/share/classes/java/lang/System.java 2016-01-08 16:45:12.000000000 +0100 +++ new/src/java.base/share/classes/java/lang/System.java 2016-01-08 16:45:12.000000000 +0100 @@ -1535,6 +1535,8 @@ * @return an instance of {@link Logger} that can be used by the calling * class. * @throws NullPointerException if {@code name} is {@code null}. + * + * @since 9 */ @CallerSensitive public static Logger getLogger(String name) { @@ -1572,6 +1574,8 @@ * resource bundle for message localization. * @throws NullPointerException if {@code name} is {@code null} or * {@code bundle} is {@code null}. + * + * @since 9 */ @CallerSensitive public static Logger getLogger(String name, ResourceBundle bundle) { From chris.hegarty at oracle.com Fri Jan 8 15:55:05 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 8 Jan 2016 15:55:05 +0000 Subject: RFR: 8146691: @since 9 tag missing for System.getLogger In-Reply-To: <568FDB1A.7060703@oracle.com> References: <568FDB1A.7060703@oracle.com> Message-ID: <568FDBD9.1000501@oracle.com> Looks fine Daniel. -Chris. On 08/01/16 15:51, Daniel Fuchs wrote: > > > > Hi, > > Please find below a trivial fix for > 8146691: @since 9 tag missing for System.getLogger > > https://bugs.openjdk.java.net/browse/JDK-8146691 > > best regards, > > -- daniel > > > --- old/src/java.base/share/classes/java/lang/System.java 2016-01-08 > 16:45:12.000000000 +0100 > +++ new/src/java.base/share/classes/java/lang/System.java 2016-01-08 > 16:45:12.000000000 +0100 > @@ -1535,6 +1535,8 @@ > * @return an instance of {@link Logger} that can be used by the > calling > * class. > * @throws NullPointerException if {@code name} is {@code null}. > + * > + * @since 9 > */ > @CallerSensitive > public static Logger getLogger(String name) { > @@ -1572,6 +1574,8 @@ > * resource bundle for message localization. > * @throws NullPointerException if {@code name} is {@code null} or > * {@code bundle} is {@code null}. > + * > + * @since 9 > */ > @CallerSensitive > public static Logger getLogger(String name, ResourceBundle bundle) { From chris.hegarty at oracle.com Fri Jan 8 16:08:13 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 8 Jan 2016 16:08:13 +0000 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568FDB1E.4050300@Oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> <568D64C8.7030401@gmail.com> <568FDB1E.4050300@Oracle.com> Message-ID: <568FDEED.2030201@oracle.com> Hi Roger, On 08/01/16 15:51, Roger Riggs wrote: > Thanks for the review and comments: > > While integrating the comments, some comments identified finalizers that > were unneeded should be removed. > I split that into a separate bug, as they are not being replaced, just > removed. [2] > > The CleaningFactory is moved to jdk.internal.ref so it can be exported > selectively to specific > OpenJDK modules. The suggested refactoring of the subclassable > XXXCleanables can be done separately > if it turns out to be useful outside the base module. > > Webrev for shared CleaningFactory and its uses: > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ Thanks for moving this into its own package. Does is make sense to move CleanerImpl? Not for it to be used by other modules, just logically seems to belong there. CleanerTest should have its @modules tag updated to include java.base/jdk.internal.ref. Trivially, "Cleaner for use within the java.base module" -> 'OpenJDK modules'. Otherwise, looks good to me. Thanks, -Chris. > Webrev for removed finalizers: > http://cr.openjdk.java.net/~rriggs/webrev-rmfinalizers-8146567/ > > Thanks, Roger > > > [1] https://bugs.openjdk.java.net/browse/JDK-8146028 Common Cleaner for > finalization replacements in OpenJDK > [2] https://bugs.openjdk.java.net/browse/JDK-8146567 Remove dead code > finalizer methods > > On 1/6/2016 2:02 PM, Peter Levart wrote: >> >> >> On 01/05/2016 10:16 PM, Roger Riggs wrote: >>> Hi Daniel, >>> >>> webrev updated to revert changes to MeteredStream as too risky. >>> >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >> >> >> Hi Roger, >> >> I briefly skimmed over the webrev, and found the following issue in >> ProcessImpl: >> >> 420 // Register a cleaning function to close the handle >> 421 CleanerFactory.getCleaner().register(this, () -> >> closeHandle(handle)); >> 422 >> >> ... 'handle' is an instance variable, which means that 'this' is >> captured by the lambda. You will have to assign handle to a local var >> and capture it instead to prevent 'this' to be captured. >> >> Regards, Peter >> >> >>> >>> On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >>>> Hi Roger, >>>> >>>> Some early feedback: >>>> >>>> I see that prior to your changes, MeteredStream.close() would >>>> be called by finalize. >>>> This will no longer be the case after >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>> >>> In the case where finalize would call close(), there can be no >>> queuedForCleanup activity >>> because there can be no strong reference from the >>> KeepAliveStreamCleaner thread >>> or the queue that holds the pending cleanups. >>> It cannot be in the Cache of connections being kept alive or the >>> thread that keeps them alive. >>> >>> It might be possible that the underlying connection is still open; >>> but there is no advantage >>> of trying to drain it. There is the possibility of resurrecting it >>> by virtue of deciding that >>> it should be queued for cleanup and starting a new thread to do the >>> cleanup. >>> >>> All in all, there is a risk of misunderstanding the dynamic behavior >>> and it would be safer >>> to leave this using finalize. >>> >>>> >>>> I see that MeteredStream has a subclass (KeepAliveStream) that >>>> overrides close() - so KeepAliveStream probably requires a cleanup >>>> as well - doesn't it? >>>> >>>> Finally I'd suggest making the variable 'pi' final - since you're >>>> passing that to the lambda you don't want to allow its value to >>>> be modified afterwards (fortunately it's not - and I believe it >>>> should be good to ensure it won't be). >>> ok, it would not go unnoticed, the compiler should complain if the >>> variable is no effectively final. >>> >>> Thanks, Roger >>> >>>> >>>> I haven't looked at the other classes yet... >>>> >>>> best regards, >>>> >>>> -- daniel >>>> >>>> On 05/01/16 19:24, Roger Riggs wrote: >>>>> The follow on work to adding the Cleaner is to replace uses of >>>>> finalization with uses of the Cleaner. >>>>> For the 'easy' cases in the java.base module, it is useful to >>>>> introduce >>>>> a private Cleaner for the >>>>> java.base module. It is proposed to be held weakly, to allow it to >>>>> terminate on a lightly loaded >>>>> system. >>>>> >>>>> Webrev for Review: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>>>> >>>>> >>>>> The 2nd step is using the Cleaner. >>>>> - Empty finalize methods should (I think) be removed; but since they >>>>> are part of the public spec >>>>> the process needs two full releases; so the proposal is to >>>>> deprecate >>>>> them first. >>>>> (The JEP 277 necessary changes will be updated when JEP 277 >>>>> semantics are finalized) >>>>> >>>>> - In a few cases, the code in the finalizer is redundant and if >>>>> removed >>>>> would allow >>>>> an optimization of the handling of the finalizer and future removal >>>>> of the finalize method. >>>>> >>>>> Webrev: >>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>>> >>>>> >>>>> Thanks for comments and suggestions, Roger >>>>> >>>>> >>>>> >>>>> >>>> >>> >> > From daniel.fuchs at oracle.com Fri Jan 8 16:20:46 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 8 Jan 2016 17:20:46 +0100 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568FDB1E.4050300@Oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> <568D64C8.7030401@gmail.com> <568FDB1E.4050300@Oracle.com> Message-ID: <568FE1DE.3040807@oracle.com> On 08/01/16 16:51, Roger Riggs wrote: > Webrev for removed finalizers: > http://cr.openjdk.java.net/~rriggs/webrev-rmfinalizers-8146567/ Looks good to me Roger! best regards, -- daniel From mandy.chung at oracle.com Fri Jan 8 17:22:48 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 8 Jan 2016 09:22:48 -0800 Subject: RFR: 8146691: @since 9 tag missing for System.getLogger In-Reply-To: <568FDB1A.7060703@oracle.com> References: <568FDB1A.7060703@oracle.com> Message-ID: <49304F38-2A9D-43EF-B91A-6CB3C485A84D@oracle.com> +1 Mandy > On Jan 8, 2016, at 7:51 AM, Daniel Fuchs wrote: > > > > > Hi, > > Please find below a trivial fix for > 8146691: @since 9 tag missing for System.getLogger > > https://bugs.openjdk.java.net/browse/JDK-8146691 > > best regards, > > -- daniel > > > --- old/src/java.base/share/classes/java/lang/System.java 2016-01-08 16:45:12.000000000 +0100 > +++ new/src/java.base/share/classes/java/lang/System.java 2016-01-08 16:45:12.000000000 +0100 > @@ -1535,6 +1535,8 @@ > * @return an instance of {@link Logger} that can be used by the calling > * class. > * @throws NullPointerException if {@code name} is {@code null}. > + * > + * @since 9 > */ > @CallerSensitive > public static Logger getLogger(String name) { > @@ -1572,6 +1574,8 @@ > * resource bundle for message localization. > * @throws NullPointerException if {@code name} is {@code null} or > * {@code bundle} is {@code null}. > + * > + * @since 9 > */ > @CallerSensitive > public static Logger getLogger(String name, ResourceBundle bundle) { From mark.sheppard at oracle.com Fri Jan 8 17:49:25 2016 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Fri, 8 Jan 2016 17:49:25 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection Message-ID: <568FF6A5.4080700@oracle.com> Hi please oblige and review the following changes http://cr.openjdk.java.net/~msheppar/8144144/webrev/ which addresses the issue https://bugs.openjdk.java.net/browse/JDK-8144144 the changes ensure that an Acceptor and its associated ServerSocket/ServerSocketChannel are closed. regards Mark From chris.hegarty at oracle.com Fri Jan 8 19:09:15 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 8 Jan 2016 19:09:15 +0000 Subject: RFR [9] Move sun.misc.Perf* classes to jdk.internal.perf Message-ID: There are a small number, three, of classes in sun.misc that are used by the platform for monitor and logging of the core platform itself. In preparation for JEP 260, these can be moved into a new internal package, jdk.internal.perf. http://cr.openjdk.java.net/~chegar/sun.misc.Perf/ -Chris. From Roger.Riggs at Oracle.com Fri Jan 8 19:22:36 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 8 Jan 2016 14:22:36 -0500 Subject: RFR [9] Move sun.misc.Perf* classes to jdk.internal.perf In-Reply-To: References: Message-ID: <56900C7C.2000900@Oracle.com> Hi Chris, Looks fine. Is there any particular reason in XToolKit.java that sun.misc.ManagedLocalsThread is both imported and fully qualified in the code (line 282+) The conversion to use the new Cleaner looks correct too; though I haven't pushed it yet. (trying to refactor to avoid exposing, too much, implementation as public methods) Roger On 1/8/2016 2:09 PM, Chris Hegarty wrote: > There are a small number, three, of classes in sun.misc that are used > by the platform for monitor and logging of the core platform itself. In > preparation for JEP 260, these can be moved into a new internal > package, jdk.internal.perf. > > http://cr.openjdk.java.net/~chegar/sun.misc.Perf/ > > -Chris. From chris.hegarty at oracle.com Fri Jan 8 19:26:05 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 8 Jan 2016 19:26:05 +0000 Subject: RFR [9] Move sun.misc.Perf* classes to jdk.internal.perf In-Reply-To: <56900C7C.2000900@Oracle.com> References: <56900C7C.2000900@Oracle.com> Message-ID: <6EB76FE0-F833-4EF7-95F8-E33B8B14A50E@oracle.com> On 8 Jan 2016, at 19:22, Roger Riggs wrote: > Hi Chris, > > Looks fine. Thanks Roger. > Is there any particular reason in XToolKit.java that sun.misc.ManagedLocalsThread > is both imported and fully qualified in the code (line 282+) That is temporary. I?m in the process of filling a bug to replace all usages of ManagedLocalsThread with the new shiny Thread ( suppress inheriting inheritable thread-local initial values ). > The conversion to use the new Cleaner looks correct too; though I haven't pushed it yet. > (trying to refactor to avoid exposing, too much, implementation as public methods) Got ya. -Chris. > Roger > > > > On 1/8/2016 2:09 PM, Chris Hegarty wrote: >> There are a small number, three, of classes in sun.misc that are used >> by the platform for monitor and logging of the core platform itself. In >> preparation for JEP 260, these can be moved into a new internal >> package, jdk.internal.perf. >> >> http://cr.openjdk.java.net/~chegar/sun.misc.Perf/ >> >> -Chris. > From mandy.chung at oracle.com Fri Jan 8 19:51:47 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 8 Jan 2016 11:51:47 -0800 Subject: RFR: 8146665: Test jdk/test/java/util/logging/LogManager/Configuration/updateConfiguration/UpdateConfigurationTest.java fails - missing expected output In-Reply-To: <568FD794.7030200@oracle.com> References: <568FD794.7030200@oracle.com> Message-ID: <1829FDB2-5AA2-453A-8847-E416CBA05414@oracle.com> > On Jan 8, 2016, at 7:36 AM, Daniel Fuchs wrote: > > Hi, > > Please find below a tentative fix for: > > 8146665: Test jdk/test/java/util/logging/LogManager/Configuration/updateConfiguration/UpdateConfigurationTest.java fails - missing expected output > https://bugs.openjdk.java.net/browse/JDK-8146665 > > Webrev: > http://cr.openjdk.java.net/~dfuchs/webrev_8146665/webrev.00/ > > The fix increases the numbers of gc performed (which may or may > not fix the issue), but more importantly fixes the finally { } clause > of the test(...) method, where a failure in releasing the second > logger reference may be hiding the original failure. line 280 adding the suppressed == null check is okay. Is the additional GC needed? Are you able to reproduce the failure? Mandy From Roger.Riggs at Oracle.com Fri Jan 8 19:57:50 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 8 Jan 2016 14:57:50 -0500 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <568FDEED.2030201@oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> <568D64C8.7030401@gmail.com> <568FDB1E.4050300@Oracle.com> <568FDEED.2030201@oracle.com> Message-ID: <569014BE.2090201@Oracle.com> Hi Chris, Updated webrev: http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ On 1/8/2016 11:08 AM, Chris Hegarty wrote: > Hi Roger, > > On 08/01/16 15:51, Roger Riggs wrote: >> Thanks for the review and comments: >> >> While integrating the comments, some comments identified finalizers that >> were unneeded should be removed. >> I split that into a separate bug, as they are not being replaced, just >> removed. [2] >> >> The CleaningFactory is moved to jdk.internal.ref so it can be exported >> selectively to specific >> OpenJDK modules. The suggested refactoring of the subclassable >> XXXCleanables can be done separately >> if it turns out to be useful outside the base module. >> >> Webrev for shared CleaningFactory and its uses: >> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ > > Thanks for moving this into its own package. Does is make sense to > move CleanerImpl? Not for it to be used by other modules, just > logically seems to belong there. Ok, moved CleanerImpl to jdk.internal.ref; in the the process refactored it to expose fewer of the implementation methods as public to avoid temptations of misuse. The XXXCleanable classes are top level classes and not nested in CleanerImpl. > > CleanerTest should have its @modules tag updated to include > java.base/jdk.internal.ref. Added > > Trivially, "Cleaner for use within the java.base module" -> > 'OpenJDK modules'. Fixed Thanks, Roger > > Otherwise, looks good to me. > > Thanks, > -Chris. > >> Webrev for removed finalizers: >> http://cr.openjdk.java.net/~rriggs/webrev-rmfinalizers-8146567/ >> >> Thanks, Roger >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8146028 Common Cleaner for >> finalization replacements in OpenJDK >> [2] https://bugs.openjdk.java.net/browse/JDK-8146567 Remove dead code >> finalizer methods >> >> On 1/6/2016 2:02 PM, Peter Levart wrote: >>> >>> >>> On 01/05/2016 10:16 PM, Roger Riggs wrote: >>>> Hi Daniel, >>>> >>>> webrev updated to revert changes to MeteredStream as too risky. >>>> >>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>> >>> >>> >>> Hi Roger, >>> >>> I briefly skimmed over the webrev, and found the following issue in >>> ProcessImpl: >>> >>> 420 // Register a cleaning function to close the handle >>> 421 CleanerFactory.getCleaner().register(this, () -> >>> closeHandle(handle)); >>> 422 >>> >>> ... 'handle' is an instance variable, which means that 'this' is >>> captured by the lambda. You will have to assign handle to a local var >>> and capture it instead to prevent 'this' to be captured. >>> >>> Regards, Peter >>> >>> >>>> >>>> On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >>>>> Hi Roger, >>>>> >>>>> Some early feedback: >>>>> >>>>> I see that prior to your changes, MeteredStream.close() would >>>>> be called by finalize. >>>>> This will no longer be the case after >>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>>> >>>>> >>>> In the case where finalize would call close(), there can be no >>>> queuedForCleanup activity >>>> because there can be no strong reference from the >>>> KeepAliveStreamCleaner thread >>>> or the queue that holds the pending cleanups. >>>> It cannot be in the Cache of connections being kept alive or the >>>> thread that keeps them alive. >>>> >>>> It might be possible that the underlying connection is still open; >>>> but there is no advantage >>>> of trying to drain it. There is the possibility of resurrecting it >>>> by virtue of deciding that >>>> it should be queued for cleanup and starting a new thread to do the >>>> cleanup. >>>> >>>> All in all, there is a risk of misunderstanding the dynamic behavior >>>> and it would be safer >>>> to leave this using finalize. >>>> >>>>> >>>>> I see that MeteredStream has a subclass (KeepAliveStream) that >>>>> overrides close() - so KeepAliveStream probably requires a cleanup >>>>> as well - doesn't it? >>>>> >>>>> Finally I'd suggest making the variable 'pi' final - since you're >>>>> passing that to the lambda you don't want to allow its value to >>>>> be modified afterwards (fortunately it's not - and I believe it >>>>> should be good to ensure it won't be). >>>> ok, it would not go unnoticed, the compiler should complain if the >>>> variable is no effectively final. >>>> >>>> Thanks, Roger >>>> >>>>> >>>>> I haven't looked at the other classes yet... >>>>> >>>>> best regards, >>>>> >>>>> -- daniel >>>>> >>>>> On 05/01/16 19:24, Roger Riggs wrote: >>>>>> The follow on work to adding the Cleaner is to replace uses of >>>>>> finalization with uses of the Cleaner. >>>>>> For the 'easy' cases in the java.base module, it is useful to >>>>>> introduce >>>>>> a private Cleaner for the >>>>>> java.base module. It is proposed to be held weakly, to allow it to >>>>>> terminate on a lightly loaded >>>>>> system. >>>>>> >>>>>> Webrev for Review: >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>>>>> >>>>>> >>>>>> The 2nd step is using the Cleaner. >>>>>> - Empty finalize methods should (I think) be removed; but since >>>>>> they >>>>>> are part of the public spec >>>>>> the process needs two full releases; so the proposal is to >>>>>> deprecate >>>>>> them first. >>>>>> (The JEP 277 necessary changes will be updated when JEP 277 >>>>>> semantics are finalized) >>>>>> >>>>>> - In a few cases, the code in the finalizer is redundant and if >>>>>> removed >>>>>> would allow >>>>>> an optimization of the handling of the finalizer and future >>>>>> removal >>>>>> of the finalize method. >>>>>> >>>>>> Webrev: >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>>>> >>>>>> >>>>>> >>>>>> Thanks for comments and suggestions, Roger >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >> From chris.hegarty at oracle.com Fri Jan 8 20:00:08 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 8 Jan 2016 20:00:08 +0000 Subject: RFR 9: JDK-8146028 : Common Cleaner for finalization replacements in java.base In-Reply-To: <569014BE.2090201@Oracle.com> References: <568C0A5C.7090204@Oracle.com> <568C0F3C.2000906@oracle.com> <568C3298.7030200@Oracle.com> <568D64C8.7030401@gmail.com> <568FDB1E.4050300@Oracle.com> <568FDEED.2030201@oracle.com> <569014BE.2090201@Oracle.com> Message-ID: On 8 Jan 2016, at 19:57, Roger Riggs wrote: > Hi Chris, > > Updated webrev: > http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ Thanks Roger. This is starting to look like a nice, and useful, package ;-) -Chris. > On 1/8/2016 11:08 AM, Chris Hegarty wrote: >> Hi Roger, >> >> On 08/01/16 15:51, Roger Riggs wrote: >>> Thanks for the review and comments: >>> >>> While integrating the comments, some comments identified finalizers that >>> were unneeded should be removed. >>> I split that into a separate bug, as they are not being replaced, just >>> removed. [2] >>> >>> The CleaningFactory is moved to jdk.internal.ref so it can be exported >>> selectively to specific >>> OpenJDK modules. The suggested refactoring of the subclassable >>> XXXCleanables can be done separately >>> if it turns out to be useful outside the base module. >>> >>> Webrev for shared CleaningFactory and its uses: >>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >> >> Thanks for moving this into its own package. Does is make sense to >> move CleanerImpl? Not for it to be used by other modules, just >> logically seems to belong there. > Ok, moved CleanerImpl to jdk.internal.ref; in the the process refactored it to expose fewer > of the implementation methods as public to avoid temptations of misuse. > The XXXCleanable classes are top level classes and not nested in CleanerImpl. >> >> CleanerTest should have its @modules tag updated to include >> java.base/jdk.internal.ref. > Added >> >> Trivially, "Cleaner for use within the java.base module" -> >> 'OpenJDK modules'. > Fixed > > Thanks, Roger > >> >> Otherwise, looks good to me. >> >> Thanks, >> -Chris. >> >>> Webrev for removed finalizers: >>> http://cr.openjdk.java.net/~rriggs/webrev-rmfinalizers-8146567/ >>> >>> Thanks, Roger >>> >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8146028 Common Cleaner for >>> finalization replacements in OpenJDK >>> [2] https://bugs.openjdk.java.net/browse/JDK-8146567 Remove dead code >>> finalizer methods >>> >>> On 1/6/2016 2:02 PM, Peter Levart wrote: >>>> >>>> >>>> On 01/05/2016 10:16 PM, Roger Riggs wrote: >>>>> Hi Daniel, >>>>> >>>>> webrev updated to revert changes to MeteredStream as too risky. >>>>> >>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>> >>>> >>>> Hi Roger, >>>> >>>> I briefly skimmed over the webrev, and found the following issue in >>>> ProcessImpl: >>>> >>>> 420 // Register a cleaning function to close the handle >>>> 421 CleanerFactory.getCleaner().register(this, () -> >>>> closeHandle(handle)); >>>> 422 >>>> >>>> ... 'handle' is an instance variable, which means that 'this' is >>>> captured by the lambda. You will have to assign handle to a local var >>>> and capture it instead to prevent 'this' to be captured. >>>> >>>> Regards, Peter >>>> >>>> >>>>> >>>>> On 1/5/2016 1:45 PM, Daniel Fuchs wrote: >>>>>> Hi Roger, >>>>>> >>>>>> Some early feedback: >>>>>> >>>>>> I see that prior to your changes, MeteredStream.close() would >>>>>> be called by finalize. >>>>>> This will no longer be the case after >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>>>> >>>>> In the case where finalize would call close(), there can be no >>>>> queuedForCleanup activity >>>>> because there can be no strong reference from the >>>>> KeepAliveStreamCleaner thread >>>>> or the queue that holds the pending cleanups. >>>>> It cannot be in the Cache of connections being kept alive or the >>>>> thread that keeps them alive. >>>>> >>>>> It might be possible that the underlying connection is still open; >>>>> but there is no advantage >>>>> of trying to drain it. There is the possibility of resurrecting it >>>>> by virtue of deciding that >>>>> it should be queued for cleanup and starting a new thread to do the >>>>> cleanup. >>>>> >>>>> All in all, there is a risk of misunderstanding the dynamic behavior >>>>> and it would be safer >>>>> to leave this using finalize. >>>>> >>>>>> >>>>>> I see that MeteredStream has a subclass (KeepAliveStream) that >>>>>> overrides close() - so KeepAliveStream probably requires a cleanup >>>>>> as well - doesn't it? >>>>>> >>>>>> Finally I'd suggest making the variable 'pi' final - since you're >>>>>> passing that to the lambda you don't want to allow its value to >>>>>> be modified afterwards (fortunately it's not - and I believe it >>>>>> should be good to ensure it won't be). >>>>> ok, it would not go unnoticed, the compiler should complain if the >>>>> variable is no effectively final. >>>>> >>>>> Thanks, Roger >>>>> >>>>>> >>>>>> I haven't looked at the other classes yet... >>>>>> >>>>>> best regards, >>>>>> >>>>>> -- daniel >>>>>> >>>>>> On 05/01/16 19:24, Roger Riggs wrote: >>>>>>> The follow on work to adding the Cleaner is to replace uses of >>>>>>> finalization with uses of the Cleaner. >>>>>>> For the 'easy' cases in the java.base module, it is useful to >>>>>>> introduce >>>>>>> a private Cleaner for the >>>>>>> java.base module. It is proposed to be held weakly, to allow it to >>>>>>> terminate on a lightly loaded >>>>>>> system. >>>>>>> >>>>>>> Webrev for Review: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-factory-8146028/ >>>>>>> >>>>>>> >>>>>>> The 2nd step is using the Cleaner. >>>>>>> - Empty finalize methods should (I think) be removed; but since they >>>>>>> are part of the public spec >>>>>>> the process needs two full releases; so the proposal is to >>>>>>> deprecate >>>>>>> them first. >>>>>>> (The JEP 277 necessary changes will be updated when JEP 277 >>>>>>> semantics are finalized) >>>>>>> >>>>>>> - In a few cases, the code in the finalizer is redundant and if >>>>>>> removed >>>>>>> would allow >>>>>>> an optimization of the handling of the finalizer and future removal >>>>>>> of the finalize method. >>>>>>> >>>>>>> Webrev: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-cleaning-finalizers/index.html >>>>>>> >>>>>>> >>>>>>> Thanks for comments and suggestions, Roger From mandy.chung at oracle.com Fri Jan 8 20:20:20 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 8 Jan 2016 12:20:20 -0800 Subject: RFR [9] Move sun.misc.Perf* classes to jdk.internal.perf In-Reply-To: References: Message-ID: > On Jan 8, 2016, at 11:09 AM, Chris Hegarty wrote: > > There are a small number, three, of classes in sun.misc that are used > by the platform for monitor and logging of the core platform itself. In > preparation for JEP 260, these can be moved into a new internal > package, jdk.internal.perf. > > http://cr.openjdk.java.net/~chegar/sun.misc.Perf/ sun.misc.PerformanceLogger was added for java.desktop module to use. I?d suggest to move it to java.desktop. I assume this patch will go into jdk9/hs-rt instead of jdk9/dev. It may be better to separate the change to use jdk.internal.ref.CleanerFactory as a separate changeset that will allow it to go in sooner. Mandy From joe.darcy at oracle.com Fri Jan 8 21:30:35 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 8 Jan 2016 13:30:35 -0800 Subject: JDK 9 RFR of JDK-8146718: jdk/internal/jimage tests listed in both tier 1 and tier 2 Message-ID: <56902A7B.9050201@oracle.com> Hello, The combination of the changes for JDK-8049367 "Modular Run-Time Images" and JDK-8087181 "Move native jimage code to its own library (maybe libjimage)" put the tests in the directory jdk/internal/jimage in both tier 1 and tier 2. This is incorrect and unnecessary. Please review the patch below to remove the tests from tier 1: diff -r 44ab242c0bfc test/TEST.groups --- a/test/TEST.groups Fri Jan 08 17:39:00 2016 +0100 +++ b/test/TEST.groups Fri Jan 08 13:27:06 2016 -0800 @@ -77,7 +77,6 @@ sun/misc \ sun/reflect \ jdk/lambda \ - jdk/internal/jimage \ vm # All of the java.util package As a follow-up, I'll verify there are no other overlaps in the tiers. Thanks, -Joe From Roger.Riggs at Oracle.com Fri Jan 8 21:48:06 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 8 Jan 2016 16:48:06 -0500 Subject: JDK 9 RFR of JDK-8146718: jdk/internal/jimage tests listed in both tier 1 and tier 2 In-Reply-To: <56902A7B.9050201@oracle.com> References: <56902A7B.9050201@oracle.com> Message-ID: <56902E96.5010100@Oracle.com> Hi Joe, Looks fine. Roger On 1/8/2016 4:30 PM, joe darcy wrote: > Hello, > > The combination of the changes for JDK-8049367 "Modular Run-Time > Images" and JDK-8087181 "Move native jimage code to its own library > (maybe libjimage)" put the tests in the directory > > jdk/internal/jimage > > in both tier 1 and tier 2. This is incorrect and unnecessary. Please > review the patch below to remove the tests from tier 1: > > diff -r 44ab242c0bfc test/TEST.groups > --- a/test/TEST.groups Fri Jan 08 17:39:00 2016 +0100 > +++ b/test/TEST.groups Fri Jan 08 13:27:06 2016 -0800 > @@ -77,7 +77,6 @@ > sun/misc \ > sun/reflect \ > jdk/lambda \ > - jdk/internal/jimage \ > vm > > # All of the java.util package > > As a follow-up, I'll verify there are no other overlaps in the tiers. > > Thanks, > > -Joe From Roger.Riggs at Oracle.com Fri Jan 8 22:09:00 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 8 Jan 2016 17:09:00 -0500 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> <568F95E7.9030304@oracle.com> Message-ID: <5690337C.8050905@Oracle.com> +1 (With Stephen's update below). Roger On 1/8/2016 6:56 AM, Stephen Colebourne wrote: > As I mentioned in my email: > > Rather than doing: > return withDayOfMonth((int) dom); > or > return LocalDate.of(year, month, (int) dom); > you can do > return new LocalDate(year, month, (int) dom); > > (there are two occurrences) > > Stephen > > > > On 8 January 2016 at 10:56, nadeesh tv wrote: >> Hi all, >> >> Thanks Stephen for the comments >> Please see the updated webrev >> http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ >> Regards, >> Nadeesh >> >> On 1/7/2016 6:15 PM, Stephen Colebourne wrote: >>> I updated the benchmark with this change and another one: >>> >>> https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java >>> >>> Marginally fastest is this pattern as it avoids one if statement: >>> >>> if (dom > 0) { >>> if (dom <= 28) { // same month >>> return ... >>> } >>> if (dom <= 59) { // 59th Jan is 28th Feb >>> return ... >>> } >>> } >>> >>> Rather than doing: >>> return LocalDate.of(year, month, (int) dom); >>> we can also do >>> return new LocalDate(year, month, (int) dom); >>> >>> This is safe, because we know that the year, month and day are valid. >>> (I can't easily test the performance of this change, but it should be >>> noticeable as it avoids a lot of unnecessary checks). >>> >>> I'd like a few more test cases. Addition around 27/28/29/30 from the >>> first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of >>> Feb. >>> >>> Stephen >>> >>> >>> >>> On 7 January 2016 at 09:20, nadeesh tv wrote: >>>> Hi , >>>> Please see the updated webrev >>>> http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ >>>> Thanks and regards, >>>> Nadeesh >>>> >>>> On 1/6/2016 12:11 AM, Roger Riggs wrote: >>>> >>>> Hi Nadeesh, >>>> >>>> LocalDate.java: +1374: >>>> For the most common case of dom > 0 && <= 28, I would have explicitly >>>> and >>>> immediately returned the new LocalDate. >>>> >>>> if (dom > 0 && dom <= 28) { >>>> return LocalDate.of(year, month, (int) dom); >>>> } >>>> ... >>>> >>>> >>>> TCKLocalDate.java: >>>> - Since the test_plusDays_normal is being replaced, its test case >>>> should be >>>> included in the DataProvider >>>> >>>> {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} >>>> >>>> Thanks, Roger >>>> >>>> On 1/5/2016 1:05 PM, nadeesh tv wrote: >>>> >>>> Hi all, >>>> >>>> Please review a fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8068803 >>>> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >>>> >>>> Special thanks for Stephen for providing the source code patch >>>> >>>> >>>> >>>> -- >>>> Thanks and Regards, >>>> Nadeesh TV >>>> >> -- >> Thanks and Regards, >> Nadeesh TV >> From joe.darcy at oracle.com Fri Jan 8 23:14:46 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 8 Jan 2016 15:14:46 -0800 Subject: JDK 9 RFR of JDK-8146718: jdk/internal/jimage tests listed in both tier 1 and tier 2 In-Reply-To: <56902A7B.9050201@oracle.com> References: <56902A7B.9050201@oracle.com> Message-ID: <569042E6.3010007@oracle.com> PS After this change, no tests were in the intersection of tier 1 and tier 2. Cheers, -Joe On 1/8/2016 1:30 PM, joe darcy wrote: > Hello, > > The combination of the changes for JDK-8049367 "Modular Run-Time > Images" and JDK-8087181 "Move native jimage code to its own library > (maybe libjimage)" put the tests in the directory > > jdk/internal/jimage > > in both tier 1 and tier 2. This is incorrect and unnecessary. Please > review the patch below to remove the tests from tier 1: > > diff -r 44ab242c0bfc test/TEST.groups > --- a/test/TEST.groups Fri Jan 08 17:39:00 2016 +0100 > +++ b/test/TEST.groups Fri Jan 08 13:27:06 2016 -0800 > @@ -77,7 +77,6 @@ > sun/misc \ > sun/reflect \ > jdk/lambda \ > - jdk/internal/jimage \ > vm > > # All of the java.util package > > As a follow-up, I'll verify there are no other overlaps in the tiers. > > Thanks, > > -Joe From joe.darcy at oracle.com Sat Jan 9 00:30:19 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 8 Jan 2016 16:30:19 -0800 Subject: JDK 9 RFR of JDK-8146724:Problem list jdk/internal/jimage/JImageReadTest.java Message-ID: <5690549B.2060700@oracle.com> Hello, Please review the patch below to problem list jdk/internal/jimage/JImageReadTest.java The test has been consistently failing across platforms since the push for JDK-8144226 Sjavac's handling of include/exclude patterns is buggy, redundant and inconsistent Correcting the actual test failing is being tracked under JDK-8146712 jdk/internal/jimage/JImageReadTest.java fails on all platforms Thanks, -Joe diff -r dc56076a794a test/ProblemList.txt --- a/test/ProblemList.txt Fri Jan 08 14:06:17 2016 -0800 +++ b/test/ProblemList.txt Fri Jan 08 16:27:53 2016 -0800 @@ -384,6 +384,13 @@ ############################################################################ +# core_tools + +# 8146712 +jdk/internal/jimage/JImageReadTest.java generic-all + +############################################################################ + # svc_tools # 8031482 From jonathan.gibbons at oracle.com Sat Jan 9 02:08:38 2016 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 08 Jan 2016 18:08:38 -0800 Subject: JDK 9 RFR of JDK-8146724:Problem list jdk/internal/jimage/JImageReadTest.java In-Reply-To: <5690549B.2060700@oracle.com> References: <5690549B.2060700@oracle.com> Message-ID: <56906BA6.4080203@oracle.com> Approved. -- Jon On 01/08/2016 04:30 PM, joe darcy wrote: > Hello, > > Please review the patch below to problem list > > jdk/internal/jimage/JImageReadTest.java > > The test has been consistently failing across platforms since the push > for > > JDK-8144226 Sjavac's handling of include/exclude patterns is > buggy, redundant and inconsistent > > Correcting the actual test failing is being tracked under > > JDK-8146712 jdk/internal/jimage/JImageReadTest.java fails on all > platforms > > Thanks, > > -Joe > > diff -r dc56076a794a test/ProblemList.txt > --- a/test/ProblemList.txt Fri Jan 08 14:06:17 2016 -0800 > +++ b/test/ProblemList.txt Fri Jan 08 16:27:53 2016 -0800 > @@ -384,6 +384,13 @@ > > ############################################################################ > > > +# core_tools > + > +# 8146712 > +jdk/internal/jimage/JImageReadTest.java generic-all > + > +############################################################################ > > + > # svc_tools > > # 8031482 From Alan.Bateman at oracle.com Sat Jan 9 08:15:51 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 9 Jan 2016 08:15:51 +0000 Subject: RFR [9] Move sun.misc.Perf* classes to jdk.internal.perf In-Reply-To: References: Message-ID: <5690C1B7.50501@oracle.com> On 08/01/2016 19:09, Chris Hegarty wrote: > There are a small number, three, of classes in sun.misc that are used > by the platform for monitor and logging of the core platform itself. In > preparation for JEP 260, these can be moved into a new internal > package, jdk.internal.perf. > > http://cr.openjdk.java.net/~chegar/sun.misc.Perf/ > I agree with Mandy on PerformanceLogger, it's a logger and completely separate to the performance counters. Maybe it would be easier to separate that one out or look at moving it to somewhere like sun.util.logging. -Alan From nadeesh.tv at oracle.com Sat Jan 9 09:33:09 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Sat, 09 Jan 2016 15:03:09 +0530 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: <5690337C.8050905@Oracle.com> References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> <568F95E7.9030304@oracle.com> <5690337C.8050905@Oracle.com> Message-ID: <5690D3D5.4080006@oracle.com> Hi Stephen/Roger, Please see the updated the webrev http://cr.openjdk.java.net/~ntv/8068803/webrev.03/ Explicit "YEAR.checkValidValue(year + 1);' added in 3rd case to handle the *invalidTooLarge year case (*LocalDate.of(Year.MAX_VALUE, 12, 31).plusDays(1)) Thanks and Regards, Nadeesh On 1/9/2016 3:39 AM, Roger Riggs wrote: > +1 > > (With Stephen's update below). > > Roger > > > On 1/8/2016 6:56 AM, Stephen Colebourne wrote: >> As I mentioned in my email: >> >> Rather than doing: >> return withDayOfMonth((int) dom); >> or >> return LocalDate.of(year, month, (int) dom); >> you can do >> return new LocalDate(year, month, (int) dom); >> >> (there are two occurrences) >> >> Stephen >> >> >> >> On 8 January 2016 at 10:56, nadeesh tv wrote: >>> Hi all, >>> >>> Thanks Stephen for the comments >>> Please see the updated webrev >>> http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ >>> Regards, >>> Nadeesh >>> >>> On 1/7/2016 6:15 PM, Stephen Colebourne wrote: >>>> I updated the benchmark with this change and another one: >>>> >>>> https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java >>>> >>>> >>>> Marginally fastest is this pattern as it avoids one if statement: >>>> >>>> if (dom > 0) { >>>> if (dom <= 28) { // same month >>>> return ... >>>> } >>>> if (dom <= 59) { // 59th Jan is 28th Feb >>>> return ... >>>> } >>>> } >>>> >>>> Rather than doing: >>>> return LocalDate.of(year, month, (int) dom); >>>> we can also do >>>> return new LocalDate(year, month, (int) dom); >>>> >>>> This is safe, because we know that the year, month and day are valid. >>>> (I can't easily test the performance of this change, but it should be >>>> noticeable as it avoids a lot of unnecessary checks). >>>> >>>> I'd like a few more test cases. Addition around 27/28/29/30 from the >>>> first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of >>>> Feb. >>>> >>>> Stephen >>>> >>>> >>>> >>>> On 7 January 2016 at 09:20, nadeesh tv wrote: >>>>> Hi , >>>>> Please see the updated webrev >>>>> http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ >>>>> Thanks and regards, >>>>> Nadeesh >>>>> >>>>> On 1/6/2016 12:11 AM, Roger Riggs wrote: >>>>> >>>>> Hi Nadeesh, >>>>> >>>>> LocalDate.java: +1374: >>>>> For the most common case of dom > 0 && <= 28, I would have >>>>> explicitly >>>>> and >>>>> immediately returned the new LocalDate. >>>>> >>>>> if (dom > 0 && dom <= 28) { >>>>> return LocalDate.of(year, month, (int) dom); >>>>> } >>>>> ... >>>>> >>>>> >>>>> TCKLocalDate.java: >>>>> - Since the test_plusDays_normal is being replaced, its test case >>>>> should be >>>>> included in the DataProvider >>>>> >>>>> {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} >>>>> >>>>> Thanks, Roger >>>>> >>>>> On 1/5/2016 1:05 PM, nadeesh tv wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Please review a fix for >>>>> https://bugs.openjdk.java.net/browse/JDK-8068803 >>>>> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >>>>> >>>>> Special thanks for Stephen for providing the source code patch >>>>> >>>>> >>>>> >>>>> -- >>>>> Thanks and Regards, >>>>> Nadeesh TV >>>>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> > -- Thanks and Regards, Nadeesh TV From scolebourne at joda.org Sat Jan 9 11:28:30 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Sat, 9 Jan 2016 11:28:30 +0000 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: <5690D3D5.4080006@oracle.com> References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> <568F95E7.9030304@oracle.com> <5690337C.8050905@Oracle.com> <5690D3D5.4080006@oracle.com> Message-ID: Thanks for the update. You should have a test case for the exception thrown by the checkValidValue() Stephen On 9 January 2016 at 09:33, nadeesh tv wrote: > Hi Stephen/Roger, > > Please see the updated the webrev > http://cr.openjdk.java.net/~ntv/8068803/webrev.03/ > > Explicit "YEAR.checkValidValue(year + 1);' added in 3rd case to handle the > > invalidTooLarge year case (LocalDate.of(Year.MAX_VALUE, 12, 31).plusDays(1)) > > Thanks and Regards, > Nadeesh > > On 1/9/2016 3:39 AM, Roger Riggs wrote: > > +1 > > (With Stephen's update below). > > Roger > > > On 1/8/2016 6:56 AM, Stephen Colebourne wrote: > > As I mentioned in my email: > > Rather than doing: > return withDayOfMonth((int) dom); > or > return LocalDate.of(year, month, (int) dom); > you can do > return new LocalDate(year, month, (int) dom); > > (there are two occurrences) > > Stephen > > > > On 8 January 2016 at 10:56, nadeesh tv wrote: > > Hi all, > > Thanks Stephen for the comments > Please see the updated webrev > http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ > Regards, > Nadeesh > > On 1/7/2016 6:15 PM, Stephen Colebourne wrote: > > I updated the benchmark with this change and another one: > > https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java > > Marginally fastest is this pattern as it avoids one if statement: > > if (dom > 0) { > if (dom <= 28) { // same month > return ... > } > if (dom <= 59) { // 59th Jan is 28th Feb > return ... > } > } > > Rather than doing: > return LocalDate.of(year, month, (int) dom); > we can also do > return new LocalDate(year, month, (int) dom); > > This is safe, because we know that the year, month and day are valid. > (I can't easily test the performance of this change, but it should be > noticeable as it avoids a lot of unnecessary checks). > > I'd like a few more test cases. Addition around 27/28/29/30 from the > first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of > Feb. > > Stephen > > > > On 7 January 2016 at 09:20, nadeesh tv wrote: > > Hi , > Please see the updated webrev > http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ > Thanks and regards, > Nadeesh > > On 1/6/2016 12:11 AM, Roger Riggs wrote: > > Hi Nadeesh, > > LocalDate.java: +1374: > For the most common case of dom > 0 && <= 28, I would have explicitly > and > immediately returned the new LocalDate. > > if (dom > 0 && dom <= 28) { > return LocalDate.of(year, month, (int) dom); > } > ... > > > TCKLocalDate.java: > - Since the test_plusDays_normal is being replaced, its test case > should be > included in the DataProvider > > {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} > > Thanks, Roger > > On 1/5/2016 1:05 PM, nadeesh tv wrote: > > Hi all, > > Please review a fix for > https://bugs.openjdk.java.net/browse/JDK-8068803 > web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ > > Special thanks for Stephen for providing the source code patch > > > > -- > Thanks and Regards, > Nadeesh TV > > -- > Thanks and Regards, > Nadeesh TV > > > > -- > Thanks and Regards, > Nadeesh TV > From nadeesh.tv at oracle.com Sat Jan 9 12:44:31 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Sat, 09 Jan 2016 18:14:31 +0530 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> <568F95E7.9030304@oracle.com> <5690337C.8050905@Oracle.com> <5690D3D5.4080006@oracle.com> Message-ID: <569100AF.7040306@oracle.com> Hi Stephen, It's already covered in public void test_plusDays_invalidTooLarge() and public void test_minusDays_maximum(). Thanks and Regards, Nadeesh On 1/9/2016 4:58 PM, Stephen Colebourne wrote: > Thanks for the update. You should have a test case for the exception > thrown by the checkValidValue() > Stephen > > On 9 January 2016 at 09:33, nadeesh tv wrote: >> Hi Stephen/Roger, >> >> Please see the updated the webrev >> http://cr.openjdk.java.net/~ntv/8068803/webrev.03/ >> >> Explicit "YEAR.checkValidValue(year + 1);' added in 3rd case to handle the >> >> invalidTooLarge year case (LocalDate.of(Year.MAX_VALUE, 12, 31).plusDays(1)) >> >> Thanks and Regards, >> Nadeesh >> >> On 1/9/2016 3:39 AM, Roger Riggs wrote: >> >> +1 >> >> (With Stephen's update below). >> >> Roger >> >> >> On 1/8/2016 6:56 AM, Stephen Colebourne wrote: >> >> As I mentioned in my email: >> >> Rather than doing: >> return withDayOfMonth((int) dom); >> or >> return LocalDate.of(year, month, (int) dom); >> you can do >> return new LocalDate(year, month, (int) dom); >> >> (there are two occurrences) >> >> Stephen >> >> >> >> On 8 January 2016 at 10:56, nadeesh tv wrote: >> >> Hi all, >> >> Thanks Stephen for the comments >> Please see the updated webrev >> http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ >> Regards, >> Nadeesh >> >> On 1/7/2016 6:15 PM, Stephen Colebourne wrote: >> >> I updated the benchmark with this change and another one: >> >> https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java >> >> Marginally fastest is this pattern as it avoids one if statement: >> >> if (dom > 0) { >> if (dom <= 28) { // same month >> return ... >> } >> if (dom <= 59) { // 59th Jan is 28th Feb >> return ... >> } >> } >> >> Rather than doing: >> return LocalDate.of(year, month, (int) dom); >> we can also do >> return new LocalDate(year, month, (int) dom); >> >> This is safe, because we know that the year, month and day are valid. >> (I can't easily test the performance of this change, but it should be >> noticeable as it avoids a lot of unnecessary checks). >> >> I'd like a few more test cases. Addition around 27/28/29/30 from the >> first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of >> Feb. >> >> Stephen >> >> >> >> On 7 January 2016 at 09:20, nadeesh tv wrote: >> >> Hi , >> Please see the updated webrev >> http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ >> Thanks and regards, >> Nadeesh >> >> On 1/6/2016 12:11 AM, Roger Riggs wrote: >> >> Hi Nadeesh, >> >> LocalDate.java: +1374: >> For the most common case of dom > 0 && <= 28, I would have explicitly >> and >> immediately returned the new LocalDate. >> >> if (dom > 0 && dom <= 28) { >> return LocalDate.of(year, month, (int) dom); >> } >> ... >> >> >> TCKLocalDate.java: >> - Since the test_plusDays_normal is being replaced, its test case >> should be >> included in the DataProvider >> >> {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} >> >> Thanks, Roger >> >> On 1/5/2016 1:05 PM, nadeesh tv wrote: >> >> Hi all, >> >> Please review a fix for >> https://bugs.openjdk.java.net/browse/JDK-8068803 >> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >> >> Special thanks for Stephen for providing the source code patch >> >> >> >> -- >> Thanks and Regards, >> Nadeesh TV >> >> -- >> Thanks and Regards, >> Nadeesh TV >> >> >> >> -- >> Thanks and Regards, >> Nadeesh TV >> -- Thanks and Regards, Nadeesh TV From daniel.fuchs at oracle.com Sat Jan 9 15:40:27 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Sat, 9 Jan 2016 16:40:27 +0100 Subject: RFR: 8146665: Test jdk/test/java/util/logging/LogManager/Configuration/updateConfiguration/UpdateConfigurationTest.java fails - missing expected output In-Reply-To: <1829FDB2-5AA2-453A-8847-E416CBA05414@oracle.com> References: <568FD794.7030200@oracle.com> <1829FDB2-5AA2-453A-8847-E416CBA05414@oracle.com> Message-ID: <569129EB.7080204@oracle.com> Hi Mandy, On 1/8/16 8:51 PM, Mandy Chung wrote: >> On Jan 8, 2016, at 7:36 AM, Daniel Fuchs wrote: >> > >> >Hi, >> > >> >Please find below a tentative fix for: >> > >> >8146665: Test jdk/test/java/util/logging/LogManager/Configuration/updateConfiguration/UpdateConfigurationTest.java fails - missing expected output >> >https://bugs.openjdk.java.net/browse/JDK-8146665 >> > >> >Webrev: >> >http://cr.openjdk.java.net/~dfuchs/webrev_8146665/webrev.00/ >> > >> >The fix increases the numbers of gc performed (which may or may >> >not fix the issue), but more importantly fixes the finally { } clause >> >of the test(...) method, where a failure in releasing the second >> >logger reference may be hiding the original failure. > line 280 adding the suppressed == null check is okay. > Is the additional GC needed? It's probably better to call gc() at least once before calling poll(). > Are you able to reproduce the failure? No - I have tried to run the test with the same set of options on my local machine but never managed to make it fail. The suppressed == null check should make it possible to see the actual failure next time it fails, if it fails again. Maybe that will be more informative. best regards, -- daniel From mandy.chung at oracle.com Sat Jan 9 17:36:20 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Sat, 9 Jan 2016 09:36:20 -0800 Subject: RFR: 8146665: Test jdk/test/java/util/logging/LogManager/Configuration/updateConfiguration/UpdateConfigurationTest.java fails - missing expected output In-Reply-To: <569129EB.7080204@oracle.com> References: <568FD794.7030200@oracle.com> <1829FDB2-5AA2-453A-8847-E416CBA05414@oracle.com> <569129EB.7080204@oracle.com> Message-ID: <0ED826DB-E436-4016-B2E0-200008B6322E@oracle.com> > On Jan 9, 2016, at 7:40 AM, Daniel Fuchs wrote: > > > It's probably better to call gc() at least once before calling poll(). > >> Are you able to reproduce the failure? > > No - I have tried to run the test with the same set of options > on my local machine but never managed to make it fail. OK. > The suppressed == null check should make it possible to see the > actual failure next time it fails, if it fails again. > Maybe that will be more informative. Good to push this fix and see if the intermittent will still be reproduced. Mandy From joe.darcy at oracle.com Sat Jan 9 19:19:56 2016 From: joe.darcy at oracle.com (joe darcy) Date: Sat, 9 Jan 2016 11:19:56 -0800 Subject: JDK 9 RFR to remove jdk/internal/jimage/JImageReadTest.java from the problem list Message-ID: <56915D5C.9090503@oracle.com> Hello, The recent push of the fix for 8146712: jdk/internal/jimage/JImageReadTest.java fails on all platforms did not remove JImageReadTest.java from the problem list. Please review the patch below to restore the test to the set of tests that get run. Thanks, -Joe --- a/test/ProblemList.txt Sat Jan 09 11:19:32 2016 -0400 +++ b/test/ProblemList.txt Sat Jan 09 11:14:50 2016 -0800 @@ -384,13 +384,6 @@ ############################################################################ -# core_tools - -# 8146712 -jdk/internal/jimage/JImageReadTest.java generic-all - -############################################################################ - # svc_tools # 8031482 From james.laskey at oracle.com Sat Jan 9 19:35:31 2016 From: james.laskey at oracle.com (James Laskey) Date: Sat, 9 Jan 2016 15:35:31 -0400 Subject: JDK 9 RFR to remove jdk/internal/jimage/JImageReadTest.java from the problem list In-Reply-To: <56915D5C.9090503@oracle.com> References: <56915D5C.9090503@oracle.com> Message-ID: <20317CED-0F97-4596-9BFD-E34EB0F4F0CE@oracle.com> Alan questioned why this is showing up after the sjavac changes. There was concern there was some lurking issue. Sent from my iPhone > On Jan 9, 2016, at 3:19 PM, joe darcy wrote: > > Hello, > > The recent push of the fix for > > 8146712: jdk/internal/jimage/JImageReadTest.java fails on all platforms > > did not remove JImageReadTest.java from the problem list. > > Please review the patch below to restore the test to the set of tests that get run. > > Thanks, > > -Joe > > --- a/test/ProblemList.txt Sat Jan 09 11:19:32 2016 -0400 > +++ b/test/ProblemList.txt Sat Jan 09 11:14:50 2016 -0800 > @@ -384,13 +384,6 @@ > > ############################################################################ > > -# core_tools > - > -# 8146712 > -jdk/internal/jimage/JImageReadTest.java generic-all > - > -############################################################################ > - > # svc_tools > > # 8031482 > From joe.darcy at oracle.com Sat Jan 9 20:55:10 2016 From: joe.darcy at oracle.com (joe darcy) Date: Sat, 9 Jan 2016 12:55:10 -0800 Subject: JDK 9 RFR to remove jdk/internal/jimage/JImageReadTest.java from the problem list In-Reply-To: <20317CED-0F97-4596-9BFD-E34EB0F4F0CE@oracle.com> References: <56915D5C.9090503@oracle.com> <20317CED-0F97-4596-9BFD-E34EB0F4F0CE@oracle.com> Message-ID: <569173AE.6040009@oracle.com> Hi Jim, There may or may not be a lurking issue, but as it stand the test JImageReadTest.java is not being run in any of the official test runs since it is currently on the exclude list. -Joe On 1/9/2016 11:35 AM, James Laskey wrote: > Alan questioned why this is showing up after the sjavac changes. There was concern there was some lurking issue. > > Sent from my iPhone > >> On Jan 9, 2016, at 3:19 PM, joe darcy wrote: >> >> Hello, >> >> The recent push of the fix for >> >> 8146712: jdk/internal/jimage/JImageReadTest.java fails on all platforms >> >> did not remove JImageReadTest.java from the problem list. >> >> Please review the patch below to restore the test to the set of tests that get run. >> >> Thanks, >> >> -Joe >> >> --- a/test/ProblemList.txt Sat Jan 09 11:19:32 2016 -0400 >> +++ b/test/ProblemList.txt Sat Jan 09 11:14:50 2016 -0800 >> @@ -384,13 +384,6 @@ >> >> ############################################################################ >> >> -# core_tools >> - >> -# 8146712 >> -jdk/internal/jimage/JImageReadTest.java generic-all >> - >> -############################################################################ >> - >> # svc_tools >> >> # 8031482 >> From james.laskey at oracle.com Sat Jan 9 21:56:39 2016 From: james.laskey at oracle.com (James Laskey) Date: Sat, 9 Jan 2016 17:56:39 -0400 Subject: JDK 9 RFR to remove jdk/internal/jimage/JImageReadTest.java from the problem list In-Reply-To: <569173AE.6040009@oracle.com> References: <56915D5C.9090503@oracle.com> <20317CED-0F97-4596-9BFD-E34EB0F4F0CE@oracle.com> <569173AE.6040009@oracle.com> Message-ID: <3A3CEBBD-CE9E-4F4F-AAED-501E0B87D68C@oracle.com> I meant proceed but note there is a concern about why these resources are present now and not prior to the sjavac changes. Sent from my iPhone > On Jan 9, 2016, at 4:55 PM, joe darcy wrote: > > Hi Jim, > > There may or may not be a lurking issue, but as it stand the test JImageReadTest.java is not being run in any of the official test runs since it is currently on the exclude list. > > -Joe > >> On 1/9/2016 11:35 AM, James Laskey wrote: >> Alan questioned why this is showing up after the sjavac changes. There was concern there was some lurking issue. >> >> Sent from my iPhone >> >>> On Jan 9, 2016, at 3:19 PM, joe darcy wrote: >>> >>> Hello, >>> >>> The recent push of the fix for >>> >>> 8146712: jdk/internal/jimage/JImageReadTest.java fails on all platforms >>> >>> did not remove JImageReadTest.java from the problem list. >>> >>> Please review the patch below to restore the test to the set of tests that get run. >>> >>> Thanks, >>> >>> -Joe >>> >>> --- a/test/ProblemList.txt Sat Jan 09 11:19:32 2016 -0400 >>> +++ b/test/ProblemList.txt Sat Jan 09 11:14:50 2016 -0800 >>> @@ -384,13 +384,6 @@ >>> >>> ############################################################################ >>> >>> -# core_tools >>> - >>> -# 8146712 >>> -jdk/internal/jimage/JImageReadTest.java generic-all >>> - >>> -############################################################################ >>> - >>> # svc_tools >>> >>> # 8031482 > From scolebourne at joda.org Sat Jan 9 23:47:42 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Sat, 9 Jan 2016 23:47:42 +0000 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: <569100AF.7040306@oracle.com> References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> <568F95E7.9030304@oracle.com> <5690337C.8050905@Oracle.com> <5690D3D5.4080006@oracle.com> <569100AF.7040306@oracle.com> Message-ID: I'm happy with this now, thanks Stephen On 9 January 2016 at 12:44, nadeesh tv wrote: > Hi Stephen, > > It's already covered in > > public void test_plusDays_invalidTooLarge() and public void > test_minusDays_maximum(). > > Thanks and Regards, > Nadeesh > > > On 1/9/2016 4:58 PM, Stephen Colebourne wrote: >> >> Thanks for the update. You should have a test case for the exception >> thrown by the checkValidValue() >> Stephen >> >> On 9 January 2016 at 09:33, nadeesh tv wrote: >>> >>> Hi Stephen/Roger, >>> >>> Please see the updated the webrev >>> http://cr.openjdk.java.net/~ntv/8068803/webrev.03/ >>> >>> Explicit "YEAR.checkValidValue(year + 1);' added in 3rd case to handle >>> the >>> >>> invalidTooLarge year case (LocalDate.of(Year.MAX_VALUE, 12, >>> 31).plusDays(1)) >>> >>> Thanks and Regards, >>> Nadeesh >>> >>> On 1/9/2016 3:39 AM, Roger Riggs wrote: >>> >>> +1 >>> >>> (With Stephen's update below). >>> >>> Roger >>> >>> >>> On 1/8/2016 6:56 AM, Stephen Colebourne wrote: >>> >>> As I mentioned in my email: >>> >>> Rather than doing: >>> return withDayOfMonth((int) dom); >>> or >>> return LocalDate.of(year, month, (int) dom); >>> you can do >>> return new LocalDate(year, month, (int) dom); >>> >>> (there are two occurrences) >>> >>> Stephen >>> >>> >>> >>> On 8 January 2016 at 10:56, nadeesh tv wrote: >>> >>> Hi all, >>> >>> Thanks Stephen for the comments >>> Please see the updated webrev >>> http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ >>> Regards, >>> Nadeesh >>> >>> On 1/7/2016 6:15 PM, Stephen Colebourne wrote: >>> >>> I updated the benchmark with this change and another one: >>> >>> >>> https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java >>> >>> Marginally fastest is this pattern as it avoids one if statement: >>> >>> if (dom > 0) { >>> if (dom <= 28) { // same month >>> return ... >>> } >>> if (dom <= 59) { // 59th Jan is 28th Feb >>> return ... >>> } >>> } >>> >>> Rather than doing: >>> return LocalDate.of(year, month, (int) dom); >>> we can also do >>> return new LocalDate(year, month, (int) dom); >>> >>> This is safe, because we know that the year, month and day are valid. >>> (I can't easily test the performance of this change, but it should be >>> noticeable as it avoids a lot of unnecessary checks). >>> >>> I'd like a few more test cases. Addition around 27/28/29/30 from the >>> first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of >>> Feb. >>> >>> Stephen >>> >>> >>> >>> On 7 January 2016 at 09:20, nadeesh tv wrote: >>> >>> Hi , >>> Please see the updated webrev >>> http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ >>> Thanks and regards, >>> Nadeesh >>> >>> On 1/6/2016 12:11 AM, Roger Riggs wrote: >>> >>> Hi Nadeesh, >>> >>> LocalDate.java: +1374: >>> For the most common case of dom > 0 && <= 28, I would have >>> explicitly >>> and >>> immediately returned the new LocalDate. >>> >>> if (dom > 0 && dom <= 28) { >>> return LocalDate.of(year, month, (int) dom); >>> } >>> ... >>> >>> >>> TCKLocalDate.java: >>> - Since the test_plusDays_normal is being replaced, its test case >>> should be >>> included in the DataProvider >>> >>> {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} >>> >>> Thanks, Roger >>> >>> On 1/5/2016 1:05 PM, nadeesh tv wrote: >>> >>> Hi all, >>> >>> Please review a fix for >>> https://bugs.openjdk.java.net/browse/JDK-8068803 >>> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >>> >>> Special thanks for Stephen for providing the source code patch >>> >>> >>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> >>> >>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> > > -- > Thanks and Regards, > Nadeesh TV > From chris.hegarty at oracle.com Sun Jan 10 09:19:10 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 10 Jan 2016 09:19:10 +0000 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging Message-ID: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> In preparation for JEP 260, and as per the suggestion over on the core libs[1], sun.misc.PerformanceLogger should be moved to sun.util.logging. http://cr.openjdk.java.net/~chegar/8146735/ -Chris [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/037959.html From Alan.Bateman at oracle.com Sun Jan 10 09:23:28 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 10 Jan 2016 09:23:28 +0000 Subject: JDK 9 RFR to remove jdk/internal/jimage/JImageReadTest.java from the problem list In-Reply-To: <20317CED-0F97-4596-9BFD-E34EB0F4F0CE@oracle.com> References: <56915D5C.9090503@oracle.com> <20317CED-0F97-4596-9BFD-E34EB0F4F0CE@oracle.com> Message-ID: <56922310.600@oracle.com> On 09/01/2016 19:35, James Laskey wrote: > Alan questioned why this is showing up after the sjavac changes. There was concern there was some lurking issue. > No issue removing the test from the exclude list, that should have been done as part of JDK-8146712. I think we need to establish whether the META-INF/jvmci.providers configuration files are meant to be in the run-time image. I assume they are but they have not been present until the recent changes (looks like the sjavac change). Their presence confuses the test, a minor oversight in the test that is fixed now. The issue is of course nothing to do with the jimage code. If you look at a build from a few days ago then the META-INF/jvmci.providers aren't in the exploded build, pull in the sjavac change and they are in the exploded build. Maybe the changes to the exclude/include patterns causes this but needs digging into. -Alan From chris.hegarty at oracle.com Sun Jan 10 09:31:41 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 10 Jan 2016 09:31:41 +0000 Subject: RFR [9] 8146736: Move sun.misc performance counters to jdk.internal.perf Message-ID: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> In preparation for JEP 260, the performance counters can be moved out of sun.misc and located somewhere more appropriate, like jdk.internal.perf. http://cr.openjdk.java.net/~chegar/8146736/ Mandy asked whether this change should go through jdk9/hs-rt or jdk9/dev, I see no strong reason why is should go through hs-rt, so propose to go through jdk9/dev, unless there are objections. The hotspot changes are minor. Note: the use of cleaner in Perf will be replaced with the proposed CleanerFactory when it is added. -Chris. From Alan.Bateman at oracle.com Sun Jan 10 09:33:02 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 10 Jan 2016 09:33:02 +0000 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> Message-ID: <5692254E.90804@oracle.com> On 10/01/2016 09:19, Chris Hegarty wrote: > In preparation for JEP 260, and as per the suggestion over on the core libs[1], > sun.misc.PerformanceLogger should be moved to sun.util.logging. > > http://cr.openjdk.java.net/~chegar/8146735/ > > This looks okay to me. -Alan From Alan.Bateman at oracle.com Sun Jan 10 20:29:07 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 10 Jan 2016 20:29:07 +0000 Subject: RFR [9] 8146736: Move sun.misc performance counters to jdk.internal.perf In-Reply-To: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> References: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> Message-ID: <5692BF13.3030309@oracle.com> On 10/01/2016 09:31, Chris Hegarty wrote: > In preparation for JEP 260, the performance counters can be moved out of > sun.misc and located somewhere more appropriate, like jdk.internal.perf. > > http://cr.openjdk.java.net/~chegar/8146736/ > > Mandy asked whether this change should go through jdk9/hs-rt or jdk9/dev, > I see no strong reason why is should go through hs-rt, so propose to go through > jdk9/dev, unless there are objections. The hotspot changes are minor. > > Note: the use of cleaner in Perf will be replaced with the proposed CleanerFactory > when it is added. > > This looks okay to me. Going through jdk9/hs-rt means it could be several weeks before it gets to jdk9/dev so would be good to avoid the slow route if you can. -Alan. From david.holmes at oracle.com Mon Jan 11 01:46:00 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 11 Jan 2016 11:46:00 +1000 Subject: RFR [9] 8146736: Move sun.misc performance counters to jdk.internal.perf In-Reply-To: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> References: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> Message-ID: <56930958.8040906@oracle.com> Hi Chris, On 10/01/2016 7:31 PM, Chris Hegarty wrote: > In preparation for JEP 260, the performance counters can be moved out of > sun.misc and located somewhere more appropriate, like jdk.internal.perf. > > http://cr.openjdk.java.net/~chegar/8146736/ > > Mandy asked whether this change should go through jdk9/hs-rt or jdk9/dev, > I see no strong reason why is should go through hs-rt, so propose to go through > jdk9/dev, unless there are objections. The hotspot changes are minor. The changes are minor and I see no reason to not go through the dev forest, given the related changes. I've cc'd hotspot-runtime so others are aware. Thanks, David > Note: the use of cleaner in Perf will be replaced with the proposed CleanerFactory > when it is added. > > -Chris. > From sundararajan.athijegannathan at oracle.com Mon Jan 11 05:03:28 2016 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 11 Jan 2016 10:33:28 +0530 Subject: RFR 8068938: javax.script package description should specify use of ServiceLoader Message-ID: <569337A0.6070200@oracle.com> Please review http://cr.openjdk.java.net/~sundar/8068938/webrev.00/ for https://bugs.openjdk.java.net/browse/JDK-8068938 This is a doc-only change to mention that ScriptEngineFactory instances are loaded using java.util.ServiceLoader API and avoid wording on META-INF/services [ which is just one impl. mechanism used by ServiceLoader ]. This will allow the ScriptEngineFactory implementations to be deployed as modules without the service configuration file. Thanks, -Sundar From chris.hegarty at oracle.com Mon Jan 11 06:20:37 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 06:20:37 +0000 Subject: RFR [9] 8146736: Move sun.misc performance counters to jdk.internal.perf In-Reply-To: <56930958.8040906@oracle.com> References: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> <56930958.8040906@oracle.com> Message-ID: <21AF1592-9564-4722-B854-A8BC44E68FCE@oracle.com> On 11 Jan 2016, at 01:46, David Holmes wrote: > Hi Chris, > > On 10/01/2016 7:31 PM, Chris Hegarty wrote: >> In preparation for JEP 260, the performance counters can be moved out of >> sun.misc and located somewhere more appropriate, like jdk.internal.perf. >> >> http://cr.openjdk.java.net/~chegar/8146736/ >> >> Mandy asked whether this change should go through jdk9/hs-rt or jdk9/dev, >> I see no strong reason why is should go through hs-rt, so propose to go through >> jdk9/dev, unless there are objections. The hotspot changes are minor. > > The changes are minor and I see no reason to not go through the dev forest, given the related changes. I've cc'd hotspot-runtime so others are aware. Thanks David. -Chris. > Thanks, > David > >> Note: the use of cleaner in Perf will be replaced with the proposed CleanerFactory >> when it is added. >> >> -Chris. >> From chris.hegarty at oracle.com Mon Jan 11 06:32:51 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 06:32:51 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <568FF6A5.4080700@oracle.com> References: <568FF6A5.4080700@oracle.com> Message-ID: <0F72DFA3-81DE-4463-8B1E-14B50AB8057E@oracle.com> On 8 Jan 2016, at 17:49, Mark Sheppard wrote: > Hi > please oblige and review the following changes > http://cr.openjdk.java.net/~msheppar/8144144/webrev/ In SelectorImpl.clearDeferredRegistrations() there is no need to call close() on the Socket returned from channel.socket(), since you have already closed the channel. -Chris. > which addresses the issue > https://bugs.openjdk.java.net/browse/JDK-8144144 > > the changes ensure that an Acceptor and its associated ServerSocket/ServerSocketChannel > are closed. > > regards > Mark From paul.sandoz at oracle.com Mon Jan 11 08:21:19 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Jan 2016 09:21:19 +0100 Subject: RFR [9] 8146736: Move sun.misc performance counters to jdk.internal.perf In-Reply-To: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> References: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> Message-ID: <054D67D4-C8CE-4979-BCBB-B164114F4209@oracle.com> > On 10 Jan 2016, at 10:31, Chris Hegarty wrote: > > In preparation for JEP 260, the performance counters can be moved out of > sun.misc and located somewhere more appropriate, like jdk.internal.perf. > > http://cr.openjdk.java.net/~chegar/8146736/ > > Mandy asked whether this change should go through jdk9/hs-rt or jdk9/dev, > I see no strong reason why is should go through hs-rt, so propose to go through > jdk9/dev, unless there are objections. The hotspot changes are minor. > > Note: the use of cleaner in Perf will be replaced with the proposed CleanerFactory > when it is added. > +1, to go through jdk9/dev. Paul. From Alan.Bateman at oracle.com Mon Jan 11 08:56:20 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Jan 2016 08:56:20 +0000 Subject: RFR 8068938: javax.script package description should specify use of ServiceLoader In-Reply-To: <569337A0.6070200@oracle.com> References: <569337A0.6070200@oracle.com> Message-ID: <56936E34.7060200@oracle.com> On 11/01/2016 05:03, Sundararajan Athijegannathan wrote: > Please review http://cr.openjdk.java.net/~sundar/8068938/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8068938 > > This is a doc-only change to mention that ScriptEngineFactory > instances are loaded using java.util.ServiceLoader API and avoid > wording on META-INF/services [ which is just one impl. mechanism used > by ServiceLoader ]. This will allow the ScriptEngineFactory > implementations to be deployed as modules without the service > configuration file. The original wording has "current ClassLoader" which isn't very clear, does it mean the caller's class loader or the TCCL? The updated wording removes this but leaves it ambiguous as to how ScriptEngineFactory providers are located. Can it reference someone where in the javax.script javadoc where this is more clearly specified? ScriptEngineFactories, might be better as {@code ScriptEngineFactory} instances. I would be good to keep the line length somewhat consistent with the rest of the class description. Having have one >100 char line will also be annoying for future side-by-side diffs. I assume this can go into jdk9/dev, no need for it to go into jigsaw/jake. -Alan. From sean.coffey at oracle.com Mon Jan 11 08:58:29 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 11 Jan 2016 08:58:29 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <0F72DFA3-81DE-4463-8B1E-14B50AB8057E@oracle.com> References: <568FF6A5.4080700@oracle.com> <0F72DFA3-81DE-4463-8B1E-14B50AB8057E@oracle.com> Message-ID: <56936EB5.7000806@oracle.com> Looks good to me Mark. Chris's point will help clean up the new clearDeferredRegistrations method. Is this close operation applicable only to the inboundConnectionCaches ? I guess that one creates the ServerSocket for the acceptor. Does any such socket get created/left behind for the outboundConnectionCaches ? Regards, Sean. On 11/01/2016 06:32, Chris Hegarty wrote: > On 8 Jan 2016, at 17:49, Mark Sheppard wrote: > >> Hi >> please oblige and review the following changes >> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ > In SelectorImpl.clearDeferredRegistrations() there is no need to call close() > on the Socket returned from channel.socket(), since you have already > closed the channel. > > -Chris. > >> which addresses the issue >> https://bugs.openjdk.java.net/browse/JDK-8144144 >> >> the changes ensure that an Acceptor and its associated ServerSocket/ServerSocketChannel >> are closed. >> >> regards >> Mark From mark.sheppard at oracle.com Mon Jan 11 10:39:45 2016 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Mon, 11 Jan 2016 10:39:45 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <0F72DFA3-81DE-4463-8B1E-14B50AB8057E@oracle.com> References: <568FF6A5.4080700@oracle.com> <0F72DFA3-81DE-4463-8B1E-14B50AB8057E@oracle.com> Message-ID: <56938671.2020607@oracle.com> OK thanks Chris my observation was that if only channel.close() was invoked then the underlying serversocket was not being closed, it remained in the LISTEN state. This was on Windows 7. I'll check this again ... regards Mark On 11/01/2016 06:32, Chris Hegarty wrote: > On 8 Jan 2016, at 17:49, Mark Sheppard wrote: > >> Hi >> please oblige and review the following changes >> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ > In SelectorImpl.clearDeferredRegistrations() there is no need to call close() > on the Socket returned from channel.socket(), since you have already > closed the channel. > > -Chris. > >> which addresses the issue >> https://bugs.openjdk.java.net/browse/JDK-8144144 >> >> the changes ensure that an Acceptor and its associated ServerSocket/ServerSocketChannel >> are closed. >> >> regards >> Mark From chris.hegarty at oracle.com Mon Jan 11 11:09:42 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 11:09:42 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <56938671.2020607@oracle.com> References: <568FF6A5.4080700@oracle.com> <0F72DFA3-81DE-4463-8B1E-14B50AB8057E@oracle.com> <56938671.2020607@oracle.com> Message-ID: <56938D76.4000803@oracle.com> On 11/01/16 10:39, Mark Sheppard wrote: > OK thanks Chris > > my observation was that if only channel.close() was invoked then > the underlying serversocket was not being closed, it remained in the > LISTEN state. > This was on Windows 7. It is not necessary to call close() on the Socket adapter, after channel.close() has been called. Alan just reminded me that, on some platforms ( Windows ) the underlying socket may remain open, if registered with a selector, even after calling closed, until the the selector fires. This may help explain why you seen the above behavior. -Chris. > I'll check this again ... > > regards > Mark > > On 11/01/2016 06:32, Chris Hegarty wrote: >> On 8 Jan 2016, at 17:49, Mark Sheppard wrote: >> >>> Hi >>> please oblige and review the following changes >>> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ >> In SelectorImpl.clearDeferredRegistrations() there is no need to call >> close() >> on the Socket returned from channel.socket(), since you have already >> closed the channel. >> >> -Chris. >> >>> which addresses the issue >>> https://bugs.openjdk.java.net/browse/JDK-8144144 >>> >>> the changes ensure that an Acceptor and its associated >>> ServerSocket/ServerSocketChannel >>> are closed. >>> >>> regards >>> Mark > From miroslav.kos at oracle.com Mon Jan 11 11:12:59 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Mon, 11 Jan 2016 12:12:59 +0100 Subject: RFR [9]: 8146086 Publishing two webservices on same port fails with "java.net.BindException: Address already in use" In-Reply-To: <567AAC48.6010008@oracle.com> References: <567AAC48.6010008@oracle.com> Message-ID: <56938E3B.7010205@oracle.com> Ping ... anybody? On 23/12/15 15:14, Miroslav Kos wrote: > Hi everybody, > > I'd like to ask for review of . This simple patch allows > publishing more webeservices on same port. The fix has been submitted > by Kubota Yuji, it is already integrated in standalone JAX-WS repo, > regression test added. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8146086 > jaxws patch: http://cr.openjdk.java.net/~mkos/8146086/jaxws.00/ > test: http://cr.openjdk.java.net/~mkos/8146086/jdk.00/ > > Thanks > Miran > > > > From miroslav.kos at oracle.com Mon Jan 11 11:14:07 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Mon, 11 Jan 2016 12:14:07 +0100 Subject: RFR [9]: 8145104: NPE is thrown when JAXBContextFactory implementation is specified in system property, 8145112: newInstance(String, ClassLoader): java.lang.JAXBException should not be wrapped as expected according to spec In-Reply-To: <567953BF.5010206@oracle.com> References: <567953BF.5010206@oracle.com> Message-ID: <56938E7F.1010800@oracle.com> Ping ... anybody? On 22/12/15 14:44, Miroslav Kos wrote: > Hi everybody, > I'd like to ask for reviewing following patch. It fixes two issues: > > JBS: https://bugs.openjdk.java.net/browse/JDK-8145104 > NPE is thrown when JAXBContextFactory implementation is specified in > system property > > In short, it's a regression of recent changes in JAXB-AP. The old > version of API is using static methods to create JAXBContext instance, > where as the new configuration facility uses defined interface. The > methods are called via core reflection, so when invoking non-static > method of spi implemetation, instance of the spi must be created > first. This fixes NPE which occured in certain scenarios. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8145112 > newInstance(String, ClassLoader): java.lang.JAXBException should not > be wrapped as expected according to spec > > There are still some changes in javadoc being discussed, but anyway > this double-wrapping exception into JAXBException doesn't make any > sense. JAXBException thrown from JAXB spi provider is re-thrown as is. > > Fix: http://cr.openjdk.java.net/~mkos/8145104/jaxws.01/ > Tests: http://cr.openjdk.java.net/~mkos/8145104/jdk.02/ > > Added 2 new jtreg tests, the tests added also to standalone project. > > Thanks > Miran > From chris.hegarty at oracle.com Mon Jan 11 11:14:19 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 11:14:19 +0000 Subject: RFR [9] 8146771: Remove unused CEFormatException and CEStreamExhausted from sun.misc Message-ID: <56938E8B.9010000@oracle.com> These Exception types were used by the old character encoders/decoders in sun.misc, that have since been removed, see 8144479 [1]. There is no reason to keep these old Exception types. $ hg rm src/java.base/share/classes/sun/misc/CEFormatException.java $ hg rm src/java.base/share/classes/sun/misc/CEStreamExhausted.java -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8146771 From chris.hegarty at oracle.com Mon Jan 11 11:16:53 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 11:16:53 +0000 Subject: RFR [9] 8146774: Remove sun.misc.JarFilter Message-ID: <56938F25.8020001@oracle.com> JarFilter was used solely by implementation of the extension mechanism, which itself was removed by 8064956 [1]. JarFilter should probably have been removed along with the other extension implementation, it was probably just overlooked. It should be removed. $ hg rm src/java.base/share/classes/sun/misc/JarFilter.java -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8064956 From chris.hegarty at oracle.com Mon Jan 11 11:20:39 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 11:20:39 +0000 Subject: RFR [9]: 8146086 Publishing two webservices on same port fails with "java.net.BindException: Address already in use" In-Reply-To: <56938E3B.7010205@oracle.com> References: <567AAC48.6010008@oracle.com> <56938E3B.7010205@oracle.com> Message-ID: <56939007.7070305@oracle.com> Miran, On 11/01/16 11:12, Miroslav Kos wrote: > Ping ... anybody? > > On 23/12/15 15:14, Miroslav Kos wrote: >> Hi everybody, >> >> I'd like to ask for review of . This simple patch allows >> publishing more webeservices on same port. The fix has been submitted >> by Kubota Yuji, it is already integrated in standalone JAX-WS repo, >> regression test added. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8146086 >> jaxws patch: http://cr.openjdk.java.net/~mkos/8146086/jaxws.00/ The change ok. >> test: http://cr.openjdk.java.net/~mkos/8146086/jdk.00/ The test uses a hardcoded port number. Is there a pattern in this area for enable the test to start up on an ephemeral port, and use that for the subsequent request? -Chris. >> Thanks >> Miran From Alan.Bateman at oracle.com Mon Jan 11 11:26:48 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Jan 2016 11:26:48 +0000 Subject: RFR [9] 8146771: Remove unused CEFormatException and CEStreamExhausted from sun.misc In-Reply-To: <56938E8B.9010000@oracle.com> References: <56938E8B.9010000@oracle.com> Message-ID: <56939178.40405@oracle.com> On 11/01/2016 11:14, Chris Hegarty wrote: > These Exception types were used by the old character encoders/decoders > in sun.misc, that have since been removed, see 8144479 [1]. There is no > reason to keep these old Exception types. > > $ hg rm src/java.base/share/classes/sun/misc/CEFormatException.java > $ hg rm src/java.base/share/classes/sun/misc/CEStreamExhausted.java > This looks okay to me. -Alan From Alan.Bateman at oracle.com Mon Jan 11 11:27:20 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 11 Jan 2016 11:27:20 +0000 Subject: RFR [9] 8146774: Remove sun.misc.JarFilter In-Reply-To: <56938F25.8020001@oracle.com> References: <56938F25.8020001@oracle.com> Message-ID: <56939198.6090509@oracle.com> On 11/01/2016 11:16, Chris Hegarty wrote: > JarFilter was used solely by implementation of the extension mechanism, > which itself was removed by 8064956 [1]. JarFilter should probably have > been removed along with the other extension implementation, it was > probably just overlooked. It should be removed. > > $ hg rm src/java.base/share/classes/sun/misc/JarFilter.java This looks okay to me. -Alan From konstantin.shefov at oracle.com Mon Jan 11 13:34:10 2016 From: konstantin.shefov at oracle.com (Konstantin Shefov) Date: Mon, 11 Jan 2016 16:34:10 +0300 Subject: RFR [9] 8141615: Add new public methods to sun.reflect.ConstantPool In-Reply-To: <567271CE.3080604@oracle.com> References: <5645F9C6.2070802@oracle.com> <186AB3D9-5DAE-4558-AF8F-5043E22E5668@oracle.com> <1180432413.1310829.1447794240035.JavaMail.zimbra@u-pem.fr> <564C5CC4.5070003@oracle.com> <565478CB.2030005@oracle.com> <5654914E.1090603@oracle.com> <56588950.7050101@oracle.com> <566AB958.8010901@oracle.com> <9C46F7FF-BE6A-4E8C-99D0-54A71168455A@oracle.com> <566FD940.4090608@oracle.com> <61833FD9-418C-4B6E-9D31-05B43E770422@oracle.com> <56714765.5010009@oracle.com> <38C93A32-85EF-4B51-BADE-730A1C1CD656@oracle.com> <567271CE.3080604@oracle.com> Message-ID: <5693AF52.6090109@oracle.com> Kindly reminder. Already approved by C. Thalinger and I. Ignatyev. Thanks -Konstantin On 12/17/2015 11:26 AM, Konstantin Shefov wrote: > Hi Coleen > > You have previously reviewed this enhancement and made a few comments > I have resolved them, so could you look at the webrevs again, please? > > I have added tests, removed cp tags that are not in JVM spec (100 - > 105) and made some other changes. > > JDK: http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.04 > HOTSPOT: http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.02 > > Thanks > -Konstantin > > > On 12/16/2015 07:42 PM, Christian Thalinger wrote: >> Looks good. Thanks. >> >>> On Dec 16, 2015, at 1:13 AM, Konstantin Shefov >>> wrote: >>> >>> Christian >>> >>> I have fixed the enum so it uses "ENUMENTRY(int)" format now and >>> does linear search. >>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.04/ >>> >>> -Konstantin >>> >>> On 12/15/2015 08:36 PM, Christian Thalinger wrote: >>>>> On Dec 14, 2015, at 11:11 PM, Konstantin Shefov >>>>> >>>> > wrote: >>>>> >>>>> Hi Christian >>>>> >>>>> Thanks for reviewing, I have changed indents as you asked: >>>>> >>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.03 >>>>> >>>> Thanks. I?m still not comfortable with the enum. It would be >>>> great if we could get the values from the VM like in JVMCI: >>>> >>>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/c036c7f17e09/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotConstantPool.java#l101 >>>> >>>> >>>> but that would be overkill here. But I would like to see the enum >>>> entries take the integer values as arguments, like here: >>>> >>>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/c036c7f17e09/src/jdk.vm.ci/share/classes/jdk.vm.ci.sparc/src/jdk/vm/ci/sparc/SPARCKind.java#l27 >>>> >>>> >>>> and either do a simple linear search to find the entry or build up >>>> a table like the HotSpotConstantPool example above. >>>> >>>>> -Konstantin >>>>> >>>>> On 12/15/2015 06:23 AM, Christian Thalinger wrote: >>>>>>> On Dec 11, 2015, at 1:54 AM, Konstantin Shefov >>>>>>> >>>>>> > wrote: >>>>>>> >>>>>>> Hello >>>>>>> >>>>>>> Please review the new version on the patch. >>>>>>> >>>>>>> New webrev: >>>>>>> Webrev hotspot: >>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.02 >>>>>>> >>>>>>> Webrev jdk: >>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.02 >>>>>>> >>>>>> These newlines look ridiculous especially when it?s not even needed: >>>>>> >>>>>> + // Returns a class reference index for a method or a field. >>>>>> + public int getClassRefIndexAt >>>>>> + (int index) { return >>>>>> getClassRefIndexAt0 (constantPoolOop, index); } >>>>>> >>>>>> Either fix the indent or just add them like regular methods >>>>>> should look like. >>>>>> >>>>>>> What has been changed: >>>>>>> 1. Added tests for the new added methods. >>>>>>> 2. Removed CP tag codes 100 - 105 from being passed to java and >>>>>>> left only codes from the open JVM spec >>>>>>> (https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4-140). >>>>>>> >>>>>>> Thanks >>>>>>> -Konstantin >>>>>>> >>>>>>> On 11/27/2015 07:48 PM, Konstantin Shefov wrote: >>>>>>>> Coleen, >>>>>>>> Thanks for review >>>>>>>> >>>>>>>> On 11/24/2015 07:33 PM, Coleen Phillimore wrote: >>>>>>>>> I have a couple preliminary comments. First, there are no >>>>>>>>> tests added with all this new functionality. Tests should be >>>>>>>>> added with the functionality changeset, not promised afterward. >>>>>>>> I will add tests. >>>>>>>>> Secondly, I do not like that JDK code knows the implementation >>>>>>>>> details of the JVM's constant pool tags. These should be only >>>>>>>>> for internal use. >>>>>>>> The package "sun.reflect" is for internal use only, so it >>>>>>>> shouldn?t be a problem. >>>>>>>>> My third comment is that I haven't looked carefully at this >>>>>>>>> constant pool implementation but it seems very unsafe if the >>>>>>>>> class is redefined and relies on an implementation detail in >>>>>>>>> the JVM that can change. I will have more comments once I >>>>>>>>> look more at the jvmti specification. >>>>>>>>> >>>>>>>>> thanks, >>>>>>>>> Coleen >>>>>>>>> >>>>>>>>> On 11/24/15 9:48 AM, Konstantin Shefov wrote: >>>>>>>>>> Hello >>>>>>>>>> >>>>>>>>>> Please, review modified webrev. >>>>>>>>>> >>>>>>>>>> I have added methods >>>>>>>>>> getNameAndTypeRefIndexAt(int index) - to get name and type >>>>>>>>>> entry index from a method, field or indy entry index; >>>>>>>>>> getClassRefIndexAt(int index) - to get class entry index from >>>>>>>>>> a method or field entry index; >>>>>>>>>> >>>>>>>>>> I removed previously added method >>>>>>>>>> getInvokedynamicRefInfoAt(int index) - as it is no more >>>>>>>>>> needed now. >>>>>>>>>> >>>>>>>>>> New webrev: >>>>>>>>>> Webrev hotspot: >>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.01 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Webrev jdk: >>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.01 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> -Konstantin >>>>>>>>>> >>>>>>>>>> On 11/18/2015 02:11 PM, Konstantin Shefov wrote: >>>>>>>>>>> Remi, >>>>>>>>>>> >>>>>>>>>>> Thanks for reviewing. Your suggestion sounds sensibly. >>>>>>>>>>> May be it also has sense to make a method >>>>>>>>>>> "getMethodRefNameAndTypeIndex(int index)" to acquire >>>>>>>>>>> name-and-type entry index for methods also. >>>>>>>>>>> >>>>>>>>>>> -Konstantin >>>>>>>>>>> >>>>>>>>>>> On 11/18/2015 12:04 AM, Remi Forax wrote: >>>>>>>>>>>> Hi Konstantin, >>>>>>>>>>>> Technically, getInvokedynamicRefInfoAt should be >>>>>>>>>>>> getNameAndTypeOfInvokedynamicRefInfoAt because it only >>>>>>>>>>>> extract the nameAndType value of the InvokeDynamicRef. >>>>>>>>>>>> >>>>>>>>>>>> In term of API, I think it's better to decompose the API, >>>>>>>>>>>> i.e. to have a method >>>>>>>>>>>> int getInvokedynamicRefNameAndTypeIndex(int index) >>>>>>>>>>>> that returns the nameAndType index and to reuse >>>>>>>>>>>> getNameAndTypeRefInfoAt(index) to get the corresponding >>>>>>>>>>>> array of Strings. >>>>>>>>>>>> >>>>>>>>>>>> cheers, >>>>>>>>>>>> R?mi >>>>>>>>>>>> >>>>>>>>>>>> ----- Mail original ----- >>>>>>>>>>>>> De: "Christian Thalinger" >>>>>>>>>>>> > >>>>>>>>>>>>> ?: "Konstantin Shefov" >>>>>>>>>>>>> Cc: "hotspot-dev developers" >>>>>>>>>>>>> , >>>>>>>>>>>>> core-libs-dev at openjdk.java.net >>>>>>>>>>>>> >>>>>>>>>>>>> Envoy?: Lundi 16 Novembre 2015 23:41:45 >>>>>>>>>>>>> Objet: Re: RFR [9] 8141615: Add new public methods to >>>>>>>>>>>>> sun.reflect.ConstantPool >>>>>>>>>>>>> >>>>>>>>>>>>> [CC'ing core-libs-dev] >>>>>>>>>>>>> >>>>>>>>>>>>>> On Nov 13, 2015, at 4:55 AM, Konstantin Shefov >>>>>>>>>>>>>> >>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hello >>>>>>>>>>>>>> >>>>>>>>>>>>>> Please review an enhancement to add three new methods to >>>>>>>>>>>>>> sun.reflect.ConstantPool class. >>>>>>>>>>>>>> The following methods are suggested: >>>>>>>>>>>>>> >>>>>>>>>>>>>> public String[] getNameAndTypeRefInfoAt(int index) - >>>>>>>>>>>>>> returns string >>>>>>>>>>>>>> representation of name and type from a NameAndType >>>>>>>>>>>>>> constant pool entry >>>>>>>>>>>>>> with the specified index >>>>>>>>>>>>>> >>>>>>>>>>>>>> public String[] getInvokedynamicRefInfoAt(int index) - >>>>>>>>>>>>>> returns string >>>>>>>>>>>>>> representation of name and type from an InvokeDynamic >>>>>>>>>>>>>> constant pool entry >>>>>>>>>>>>>> with the specified index >>>>>>>>>>>>>> >>>>>>>>>>>>>> public Tag getTagAt(int index) - returns a Tag enum >>>>>>>>>>>>>> instance of a constant >>>>>>>>>>>>>> pool entry with the specified index >>>>>>>>>>>>>> >>>>>>>>>>>>>> These three methods could be used for testing API working >>>>>>>>>>>>>> with constant >>>>>>>>>>>>>> pool, e.g. JVMCI. >>>>>>>>>>>>>> >>>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8141615 >>>>>>>>>>>>>> Webrev hotspot: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.00 >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Webrev jdk: >>>>>>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.00 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> -Konstantin > From paul.sandoz at oracle.com Mon Jan 11 13:45:52 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Jan 2016 14:45:52 +0100 Subject: RFR [9] 8141615: Add new public methods to sun.reflect.ConstantPool In-Reply-To: <5693AF52.6090109@oracle.com> References: <5645F9C6.2070802@oracle.com> <186AB3D9-5DAE-4558-AF8F-5043E22E5668@oracle.com> <1180432413.1310829.1447794240035.JavaMail.zimbra@u-pem.fr> <564C5CC4.5070003@oracle.com> <565478CB.2030005@oracle.com> <5654914E.1090603@oracle.com> <56588950.7050101@oracle.com> <566AB958.8010901@oracle.com> <9C46F7FF-BE6A-4E8C-99D0-54A71168455A@oracle.com> <566FD940.4090608@oracle.com> <61833FD9-418C-4B6E-9D31-05B43E770422@oracle.com> <56714765.5010009@oracle.com> <38C93A32-85EF-4B51-BADE-730A1C1CD656@oracle.com> <567271CE.3080604@oracle.com> <5693AF52.6090109@oracle.com> Message-ID: <339358FB-A0EF-4475-A9D9-E5C2182B5F6B@oracle.com> > On 11 Jan 2016, at 14:34, Konstantin Shefov wrote: > > Kindly reminder. > > Already approved by C. Thalinger and I. Ignatyev. > JDK changes look ok to me. I would prefer if the JDK test ConstantPoolTest.java could be converted to TestNG using data providers, rather than rolling your own specific pattern with enums. Paul. From Roger.Riggs at Oracle.com Mon Jan 11 14:18:07 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 11 Jan 2016 09:18:07 -0500 Subject: RFR [9] 8146771: Remove unused CEFormatException and CEStreamExhausted from sun.misc In-Reply-To: <56938E8B.9010000@oracle.com> References: <56938E8B.9010000@oracle.com> Message-ID: <5693B99F.4060102@Oracle.com> +1 On 1/11/2016 6:14 AM, Chris Hegarty wrote: > These Exception types were used by the old character encoders/decoders > in sun.misc, that have since been removed, see 8144479 [1]. There is no > reason to keep these old Exception types. > > $ hg rm src/java.base/share/classes/sun/misc/CEFormatException.java > $ hg rm src/java.base/share/classes/sun/misc/CEStreamExhausted.java > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8146771 From michael.haupt at oracle.com Mon Jan 11 14:31:01 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Mon, 11 Jan 2016 15:31:01 +0100 Subject: RFR(S): 8146786: [TESTBUG] straighten out testability for several issues Message-ID: Dear all, please review this fix. Bug: https://bugs.openjdk.java.net/browse/JDK-8146786 Webrev: http://cr.openjdk.java.net/~mhaupt/8146786/webrev.00/ The fix adds some missing @bug annotations in tests and contributes two files that were accidentally not pushed earlier. Thanks, Michael -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG, Schiffbauergasse 14 | 14467 Potsdam, Germany Oracle is committed to developing practices and products that help protect the environment From miroslav.kos at oracle.com Mon Jan 11 14:41:41 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Mon, 11 Jan 2016 15:41:41 +0100 Subject: RFR [9]: 8146086 Publishing two webservices on same port fails with "java.net.BindException: Address already in use" In-Reply-To: <56939007.7070305@oracle.com> References: <567AAC48.6010008@oracle.com> <56938E3B.7010205@oracle.com> <56939007.7070305@oracle.com> Message-ID: <5693BF25.1070207@oracle.com> Hi Chris, good catch - thanks! Simple fix here: http://cr.openjdk.java.net/~mkos/8146086/jdk.01/ M. On 11/01/16 12:20, Chris Hegarty wrote: > Miran, > > On 11/01/16 11:12, Miroslav Kos wrote: >> Ping ... anybody? >> >> On 23/12/15 15:14, Miroslav Kos wrote: >>> Hi everybody, >>> >>> I'd like to ask for review of . This simple patch allows >>> publishing more webeservices on same port. The fix has been submitted >>> by Kubota Yuji, it is already integrated in standalone JAX-WS repo, >>> regression test added. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8146086 >>> jaxws patch: http://cr.openjdk.java.net/~mkos/8146086/jaxws.00/ > > The change ok. > >>> test: http://cr.openjdk.java.net/~mkos/8146086/jdk.00/ > > The test uses a hardcoded port number. Is there a pattern in this > area for enable the test to start up on an ephemeral port, and use > that for the subsequent request? > > -Chris. > >>> Thanks >>> Miran From chris.hegarty at oracle.com Mon Jan 11 14:46:43 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 14:46:43 +0000 Subject: RFR [9]: 8146086 Publishing two webservices on same port fails with "java.net.BindException: Address already in use" In-Reply-To: <5693BF25.1070207@oracle.com> References: <567AAC48.6010008@oracle.com> <56938E3B.7010205@oracle.com> <56939007.7070305@oracle.com> <5693BF25.1070207@oracle.com> Message-ID: <5693C053.7030807@oracle.com> On 11/01/16 14:41, Miroslav Kos wrote: > Hi Chris, > good catch - thanks! Simple fix here: > http://cr.openjdk.java.net/~mkos/8146086/jdk.01/ Thanks Miran, looks good to me. -Chris. > M. > > > On 11/01/16 12:20, Chris Hegarty wrote: >> Miran, >> >> On 11/01/16 11:12, Miroslav Kos wrote: >>> Ping ... anybody? >>> >>> On 23/12/15 15:14, Miroslav Kos wrote: >>>> Hi everybody, >>>> >>>> I'd like to ask for review of . This simple patch allows >>>> publishing more webeservices on same port. The fix has been submitted >>>> by Kubota Yuji, it is already integrated in standalone JAX-WS repo, >>>> regression test added. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8146086 >>>> jaxws patch: http://cr.openjdk.java.net/~mkos/8146086/jaxws.00/ >> >> The change ok. >> >>>> test: http://cr.openjdk.java.net/~mkos/8146086/jdk.00/ >> >> The test uses a hardcoded port number. Is there a pattern in this >> area for enable the test to start up on an ephemeral port, and use >> that for the subsequent request? >> >> -Chris. >> >>>> Thanks >>>> Miran > From sundararajan.athijegannathan at oracle.com Mon Jan 11 14:56:00 2016 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Mon, 11 Jan 2016 20:26:00 +0530 Subject: RFR 8068938: javax.script package description should specify use of ServiceLoader In-Reply-To: <56936E34.7060200@oracle.com> References: <569337A0.6070200@oracle.com> <56936E34.7060200@oracle.com> Message-ID: <5693C280.2030508@oracle.com> Incorporated the changes suggested. Updated webrev is @ http://cr.openjdk.java.net/~sundar/8068938/webrev.01/ Thanks, -Sundar On 1/11/2016 2:26 PM, Alan Bateman wrote: > > On 11/01/2016 05:03, Sundararajan Athijegannathan wrote: >> Please review http://cr.openjdk.java.net/~sundar/8068938/webrev.00/ >> for https://bugs.openjdk.java.net/browse/JDK-8068938 >> >> This is a doc-only change to mention that ScriptEngineFactory >> instances are loaded using java.util.ServiceLoader API and avoid >> wording on META-INF/services [ which is just one impl. mechanism used >> by ServiceLoader ]. This will allow the ScriptEngineFactory >> implementations to be deployed as modules without the service >> configuration file. > The original wording has "current ClassLoader" which isn't very clear, > does it mean the caller's class loader or the TCCL? The updated > wording removes this but leaves it ambiguous as to how > ScriptEngineFactory providers are located. Can it reference someone > where in the javax.script javadoc where this is more clearly specified? > > ScriptEngineFactories, might be better as {@code > ScriptEngineFactory} instances. > > I would be good to keep the line length somewhat consistent with the > rest of the class description. Having have one >100 char line will > also be annoying for future side-by-side diffs. > > I assume this can go into jdk9/dev, no need for it to go into > jigsaw/jake. > > -Alan. > > > From paul.sandoz at oracle.com Mon Jan 11 15:04:55 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Jan 2016 16:04:55 +0100 Subject: RFR 8146458 Improve exception reporting for Objects.checkIndex/checkFromToIndex/checkFromIndexSize Message-ID: <40431638-BC6A-44E2-8A5E-8F0B180AA924@oracle.com> Hi, When the new range check methods Object.check* were added the exception reporting was a little vague and lossy, but i got a CCC pass to revisit later on, and John nudged me to sort this out, so here is another more reflective approach: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/webrev/ http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/specdiff/overview-summary.html To preserve all information the out of bounds exception mapping function is now BiFunction>, where the first argument is the check kind (and when used with a check method conveniently corresponds to the method name) and the second argument is the list of out of bound integer values that failed the check (when used with a check method the list elements conveniently correspond to the method arguments, in order). A new factory method rangeCheckExceptionMapper has been added that creates an out of bounds exception mapping function from a message exception function, thus one can do: BiFunction> f = rangeCheckExceptionMapper(IndexOutOfBoundsException::new); checkIndex(index, length, f); checkFromToIndex(from, to length, f); checkFromIndexSize(from, size length, f); which is equivalent in behaviour (exception throwing and message) to: checkIndex(index, length); checkFromToIndex(from, to length); checkFromIndexSize(from, size length); In all three check* cases a specific exception message will be produced: case "checkIndex": return String.format("Index %d out of bounds for length %d", args.get(0), args.get(1)); case "checkFromToIndex": return String.format("Range [%d, %d) out of bounds for length %d", args.get(0), args.get(1), args.get(2)); case "checkFromIndexSize": return String.format("Range [%d, % arguments. I could still do that, but that does spread out the notion beyond that of Objects and i prefer that it be contained. Paul. From paul.sandoz at oracle.com Mon Jan 11 15:07:33 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 11 Jan 2016 16:07:33 +0100 Subject: RFR(S): 8146786: [TESTBUG] straighten out testability for several issues In-Reply-To: References: Message-ID: <0AB168E6-7EA5-448B-AE00-790C00E5A3BA@oracle.com> > On 11 Jan 2016, at 15:31, Michael Haupt wrote: > > Dear all, > > please review this fix. > Bug: https://bugs.openjdk.java.net/browse/JDK-8146786 > Webrev: http://cr.openjdk.java.net/~mhaupt/8146786/webrev.00/ > > The fix adds some missing @bug annotations in tests and contributes two files that were accidentally not pushed earlier. > +1 Paul. From Roger.Riggs at Oracle.com Mon Jan 11 16:41:39 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 11 Jan 2016 11:41:39 -0500 Subject: RFR: JDK-8068803:Performance of LocalDate.plusDays could be better In-Reply-To: References: <568C05DF.4090606@oracle.com> <568C0E60.2030008@Oracle.com> <568E2DD1.5010900@oracle.com> <568F95E7.9030304@oracle.com> <5690337C.8050905@Oracle.com> <5690D3D5.4080006@oracle.com> <569100AF.7040306@oracle.com> Message-ID: <5693DB43.2020006@Oracle.com> +1 I'll integrate this. Thanks, Roger On 1/9/2016 6:47 PM, Stephen Colebourne wrote: > I'm happy with this now, thanks > Stephen > > > On 9 January 2016 at 12:44, nadeesh tv wrote: >> Hi Stephen, >> >> It's already covered in >> >> public void test_plusDays_invalidTooLarge() and public void >> test_minusDays_maximum(). >> >> Thanks and Regards, >> Nadeesh >> >> >> On 1/9/2016 4:58 PM, Stephen Colebourne wrote: >>> Thanks for the update. You should have a test case for the exception >>> thrown by the checkValidValue() >>> Stephen >>> >>> On 9 January 2016 at 09:33, nadeesh tv wrote: >>>> Hi Stephen/Roger, >>>> >>>> Please see the updated the webrev >>>> http://cr.openjdk.java.net/~ntv/8068803/webrev.03/ >>>> >>>> Explicit "YEAR.checkValidValue(year + 1);' added in 3rd case to handle >>>> the >>>> >>>> invalidTooLarge year case (LocalDate.of(Year.MAX_VALUE, 12, >>>> 31).plusDays(1)) >>>> >>>> Thanks and Regards, >>>> Nadeesh >>>> >>>> On 1/9/2016 3:39 AM, Roger Riggs wrote: >>>> >>>> +1 >>>> >>>> (With Stephen's update below). >>>> >>>> Roger >>>> >>>> >>>> On 1/8/2016 6:56 AM, Stephen Colebourne wrote: >>>> >>>> As I mentioned in my email: >>>> >>>> Rather than doing: >>>> return withDayOfMonth((int) dom); >>>> or >>>> return LocalDate.of(year, month, (int) dom); >>>> you can do >>>> return new LocalDate(year, month, (int) dom); >>>> >>>> (there are two occurrences) >>>> >>>> Stephen >>>> >>>> >>>> >>>> On 8 January 2016 at 10:56, nadeesh tv wrote: >>>> >>>> Hi all, >>>> >>>> Thanks Stephen for the comments >>>> Please see the updated webrev >>>> http://cr.openjdk.java.net/~ntv/8068803/webrev.02/ >>>> Regards, >>>> Nadeesh >>>> >>>> On 1/7/2016 6:15 PM, Stephen Colebourne wrote: >>>> >>>> I updated the benchmark with this change and another one: >>>> >>>> >>>> https://github.com/ThreeTen/threeten-bench/blob/master/src/main/java/org/threeten/LocalDateBenchmark.java >>>> >>>> Marginally fastest is this pattern as it avoids one if statement: >>>> >>>> if (dom > 0) { >>>> if (dom <= 28) { // same month >>>> return ... >>>> } >>>> if (dom <= 59) { // 59th Jan is 28th Feb >>>> return ... >>>> } >>>> } >>>> >>>> Rather than doing: >>>> return LocalDate.of(year, month, (int) dom); >>>> we can also do >>>> return new LocalDate(year, month, (int) dom); >>>> >>>> This is safe, because we know that the year, month and day are valid. >>>> (I can't easily test the performance of this change, but it should be >>>> noticeable as it avoids a lot of unnecessary checks). >>>> >>>> I'd like a few more test cases. Addition around 27/28/29/30 from the >>>> first of Jan/Feb, and of 13/14/15/16 from the 15th of Jan and 15th of >>>> Feb. >>>> >>>> Stephen >>>> >>>> >>>> >>>> On 7 January 2016 at 09:20, nadeesh tv wrote: >>>> >>>> Hi , >>>> Please see the updated webrev >>>> http://cr.openjdk.java.net/~ntv/8068803/webrev.01/ >>>> Thanks and regards, >>>> Nadeesh >>>> >>>> On 1/6/2016 12:11 AM, Roger Riggs wrote: >>>> >>>> Hi Nadeesh, >>>> >>>> LocalDate.java: +1374: >>>> For the most common case of dom > 0 && <= 28, I would have >>>> explicitly >>>> and >>>> immediately returned the new LocalDate. >>>> >>>> if (dom > 0 && dom <= 28) { >>>> return LocalDate.of(year, month, (int) dom); >>>> } >>>> ... >>>> >>>> >>>> TCKLocalDate.java: >>>> - Since the test_plusDays_normal is being replaced, its test case >>>> should be >>>> included in the DataProvider >>>> >>>> {LocalDate.of(2007, 7, 15), 1, LocalDate.of(2007, 7, 16)} >>>> >>>> Thanks, Roger >>>> >>>> On 1/5/2016 1:05 PM, nadeesh tv wrote: >>>> >>>> Hi all, >>>> >>>> Please review a fix for >>>> https://bugs.openjdk.java.net/browse/JDK-8068803 >>>> web rev : http://cr.openjdk.java.net/~ntv/8068803/webrev.00/ >>>> >>>> Special thanks for Stephen for providing the source code patch >>>> >>>> >>>> >>>> -- >>>> Thanks and Regards, >>>> Nadeesh TV >>>> >>>> -- >>>> Thanks and Regards, >>>> Nadeesh TV >>>> >>>> >>>> >>>> -- >>>> Thanks and Regards, >>>> Nadeesh TV >>>> >> -- >> Thanks and Regards, >> Nadeesh TV >> From christian.thalinger at oracle.com Mon Jan 11 17:31:07 2016 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Mon, 11 Jan 2016 07:31:07 -1000 Subject: JDK 9 RFR to remove jdk/internal/jimage/JImageReadTest.java from the problem list In-Reply-To: <56922310.600@oracle.com> References: <56915D5C.9090503@oracle.com> <20317CED-0F97-4596-9BFD-E34EB0F4F0CE@oracle.com> <56922310.600@oracle.com> Message-ID: <21072A05-DDF8-4DA6-958B-70B51675C602@oracle.com> > On Jan 9, 2016, at 11:23 PM, Alan Bateman wrote: > > > > On 09/01/2016 19:35, James Laskey wrote: >> Alan questioned why this is showing up after the sjavac changes. There was concern there was some lurking issue. >> > No issue removing the test from the exclude list, that should have been done as part of JDK-8146712. > > I think we need to establish whether the META-INF/jvmci.providers configuration files are meant to be in the run-time image. I assume they are but they have not been present until the recent changes (looks like the sjavac change). Their presence confuses the test, a minor oversight in the test that is fixed now. The issue is of course nothing to do with the jimage code. If you look at a build from a few days ago then the META-INF/jvmci.providers aren't in the exploded build, pull in the sjavac change and they are in the exploded build. Maybe the changes to the exclude/include patterns causes this but needs digging into. The @ServiceProvider mechanism will be moved from the JVMCI to the JVMCI compiler to make the JVMCI simpler: http://hg.openjdk.java.net/graal/graal-jvmci-9/hotspot/rev/d1326c9f3cfb http://hg.openjdk.java.net/graal/graal-compiler/rev/56359eb3abfa It?s a pending change that hasn?t reached hs-comp yet but should in the next weeks. Do whatever you need to do to make the tests work. > > -Alan From mandy.chung at oracle.com Mon Jan 11 19:15:06 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 11 Jan 2016 11:15:06 -0800 Subject: RFR [9] 8146774: Remove sun.misc.JarFilter In-Reply-To: <56938F25.8020001@oracle.com> References: <56938F25.8020001@oracle.com> Message-ID: <21A84E46-E6FB-46D0-9D19-E19D60907B93@oracle.com> > On Jan 11, 2016, at 3:16 AM, Chris Hegarty wrote: > > JarFilter was used solely by implementation of the extension mechanism, > which itself was removed by 8064956 [1]. JarFilter should probably have > been removed along with the other extension implementation, it was > probably just overlooked. It should be removed. > > $ hg rm src/java.base/share/classes/sun/misc/JarFilter.java +1 Mandy From mandy.chung at oracle.com Mon Jan 11 21:04:46 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 11 Jan 2016 13:04:46 -0800 Subject: RFR [9] 8146736: Move sun.misc performance counters to jdk.internal.perf In-Reply-To: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> References: <11DB3ED6-8CC6-41F3-82E3-670D62BE906C@oracle.com> Message-ID: <2F3B46B2-F168-499B-85CE-F8842815A2D9@oracle.com> > On Jan 10, 2016, at 1:31 AM, Chris Hegarty wrote: > > In preparation for JEP 260, the performance counters can be moved out of > sun.misc and located somewhere more appropriate, like jdk.internal.perf. > > http://cr.openjdk.java.net/~chegar/8146736/ > Looks okay. > Mandy asked whether this change should go through jdk9/hs-rt or jdk9/dev, > I see no strong reason why is should go through hs-rt, so propose to go through > jdk9/dev, unless there are objections. The hotspot changes are minor. I?m happy (and happier) for this change to push to jdk9/dev rather than jdk9/hs-rt. I asked as it wasn?t clear to me what repo to be pushed this patch to. Mandy From kumar.x.srinivasan at oracle.com Mon Jan 11 21:10:12 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Mon, 11 Jan 2016 13:10:12 -0800 Subject: RFR 8124977 cmdline encoding challenges on Windows In-Reply-To: References: <558C017A.2020708@oracle.com> <5651E9AA.9060908@oracle.com> Message-ID: <56941A34.8090608@oracle.com> Hi, Was on vacation, I started to prepare the patch from webrev.04 for integration. Please note: made some adjustments to your patch to pass jcheck, ie. usage of tabs and space at line endings, and modifications to Copyright dates. Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE. I have attached a patch to for your reference. However, there is a regression test failure on Windows, jdk/test/tools/launcher/I18NTest.java ---Test info---- Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java ++++Test Output++++ javac: file not found: i18nH?lloWorld.java ----End test info----- Have you run all the launcher regression tests with this changeset ? Thanks Kumar > Hi Kumar, just wondering if there are any updates on processing this submission. > Thanks! > > -----Original Message----- > From: Vladimir Shcherbakov > Sent: Wednesday, November 25, 2015 2:38 PM > To: Kumar Srinivasan ; Martin Sawicki > Cc: Kirk Shoop ; core-libs-dev Libs > Subject: RE: RFR 8124977 cmdline encoding challenges on Windows > > Hi Kumar, > > Please find updated webreview here: > http://cr.openjdk.java.net/~kshoop/8124977/webrev.04/ > > Thanks, > Vladimir. > > -----Original Message----- > From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] > Sent: Sunday, November 22, 2015 8:14 AM > To: Martin Sawicki > Cc: Kirk Shoop ; Vladimir Shcherbakov ; core-libs-dev Libs > Subject: Re: RFR 8124977 cmdline encoding challenges on Windows > > > Hi Martin, et. al., > > Sorry for not getting back earlier, I am very busy right now with my other large commitments for JDK9. > > I will sponsor this "enhancement/bug fix" sometime in the new year, meanwhile, there is the changeset [1] which is likely to cause merge conflicts, and perhaps logic issues. > > Thanks > Kumar > > [1] https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.openjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7Mg4lt2P0sdzLS%2fT9Q%3d >> Hi all >> Here's an updated webrev attempting to take into account the various pieces of feedback we have received: >> >> Issue: >> >> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. >> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40micro >> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7c >> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E%3 >> d >> Webrev: >> >> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.openj >> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashch% >> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af9 >> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJnWY >> 0%3d >> >> (Vladimir Shcherbakov is now working on this from our side) >> >> Looking forward to any other feedback. >> Thanks >> >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of Kumar Srinivasan >> Sent: Thursday, June 25, 2015 6:26 AM >> To: Kirk Shoop (MS OPEN TECH) >> Cc: Valery Kopylov (Akvelon) ; core-libs-dev >> Libs >> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >> >> Hi Kirk, >> >> Thanks for proposing this change. >> >> If you notice all the posix calls are wrapped in JLI_* this gives us the ability to use "W" functions. I almost got it done, several years ago, but we upgraded to VS2010 and my work based on VS2003 keeled over, meanwhile my focus was "shifted" to something else. >> >> main.c: is really envisioned to be a stub compiled by the tool >> launchers, like java, javac, javah, jar etc. I prefer to see all the >> heavy logic in this file moved to the platform specific file >> windows/java_md.* >> >> For the reason specified above we need to move fprintf or any naked posix calls to JLI_* indirections. >> >> I don't see any tests ? The tests must be written in java and placed in jdk/test/tools/launcher, there is a helper framework TestHelper.java. >> >> There are other changes in nio, charsets etc, this will be reviewed by my colleague specializing in that area (Sherman) cc'ed. >> >> >> Thanks >> >> Kumar >> >> >> >> >> >> >> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote: >>> Hi, >>> >>> Issue: >>> >>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs >>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic >>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2 >>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq >>> 9E%3d >>> >>> Webrev: >>> >>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open >>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40microsof >>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7cd0 >>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M%3d >>> >>> This webrev intends to address interaction between Windows console and java apps. >>> >>> Two switches were added that change the behavior of the launcher. The defaults do not change the launcher behavior. >>> >>> -Dwindows.UnicodeConsole=true - switches on Unicode support in the Windows console. This optional switch causes the launcher to call GetCommandLineW() and parse the arguments in unicode. It also modifies how the codepage for console output is selected. >>> >>> -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset to use; If not specified, UTF-8 is used by default. Ignored when windows.UnicodeConsole is not set to true. When the first switch is used, this optional switch allows the codepage for console output to be controlled. >>> >>> I would like to get feedback on the approach here and any additional work that is required solve these particular Unicode issues on Windows. >>> >>> Kirk -------------- next part -------------- A non-text attachment was scrubbed... Name: ms-jdk.patch Type: text/x-patch Size: 13575 bytes Desc: not available URL: From mandy.chung at oracle.com Mon Jan 11 21:27:56 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 11 Jan 2016 13:27:56 -0800 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> Message-ID: <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> On Jan 10, 2016, at 1:19 AM, Chris Hegarty wrote: > > In preparation for JEP 260, and as per the suggestion over on the core libs[1], > sun.misc.PerformanceLogger should be moved to sun.util.logging. > > http://cr.openjdk.java.net/~chegar/8146735/ Why not moving it to java.desktop, perhaps sun.awt.util.PerformanceLogger? Are you thinking to keep in the base for the future java.base class to use? No one uses this class since it was added and it?s good indication that it may unlikely be used by others. In the future I think we should use System.Logger instead. Mandy From iris.clark at oracle.com Mon Jan 11 21:44:49 2016 From: iris.clark at oracle.com (Iris Clark) Date: Mon, 11 Jan 2016 13:44:49 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <92b9301d-2418-4618-b877-b13c88216520@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> Message-ID: <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Hi, Joe, Roger, Alan, Magnus, and Mandy. At the end of December (shortly before the Christmas/Winter break and my vacation), I provided responses to your messages and an updated webrev: http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ I didn't hear from anybody, so I'd like to optimistically assume that you were satisfied. Is that correct? For you convenience, here's a reference to the December and November threads: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036904.html I'd like to wrap up this work for the initial implementation of jdk.Version soon. Regards, iris -----Original Message----- From: Iris Clark Sent: Friday, December 18, 2015 1:55 PM To: Joe Darcy; Mandy Chung Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Joe. Thanks for the review comments. >>> http://cr.openjdk.java.net/~irisa/verona/8072379/webrev.1/ > Is the intention that downstream JDK distributions, such as IcedTea, > whether based on OpenJDK or otherwise, would provide their own > specialization of the jdk.Version class? No. Downstream users may provide their own specialization, but there is no requirement that they must do so. jdk.OracleVersion has been removed. The functionality it provided, access to the fourth element of the version number, is trivially accessed by existing methods in jdk.Version. I've filed the following bug to address the more general use case: 8145793: Provide vendor-specific interpretation of a JDK version string https://bugs.openjdk.java.net/browse/JDK-8145793 > The equals / hashCode behavior of Version and OracleVersion is bit > surprising and I think somewhat underspecified given the possibility > of defining subclasses. Given the above regarding downstream use, I've make jdk.Version final and the constructor is now private. Oracle or any other JDK distribution wishing to impose additional semantics to the version string interpretation will need to do so via encapsulation rather than inheritance. > How is this API supposed to behave if the component version strings > have a numerical value greater than Integer.MAX_VALUE? >From the specification of the only instance method, Version.parse(): * @throws NumberFormatException * If an element of the version number or the build number cannot * be represented as an {@link Integer} > Was using Longs to record numerical versions rather than Integers > considered? Yes. In an informal poll conducted during implementation, it was thought that 2^32 bits would be more than sufficient to represent the components of typical version numbers. An updated webrev is forthcoming. Thanks, iris PS: I believe that we are both out the week of 21 December, so I don't expect to hear back from you until the New Year. From chris.hegarty at oracle.com Mon Jan 11 22:44:41 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 11 Jan 2016 22:44:41 +0000 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> Message-ID: <21AB656E-274D-4720-8499-543631EB5F31@oracle.com> > On 11 Jan 2016, at 21:27, Mandy Chung wrote: > > On Jan 10, 2016, at 1:19 AM, Chris Hegarty wrote: >> >> In preparation for JEP 260, and as per the suggestion over on the core libs[1], >> sun.misc.PerformanceLogger should be moved to sun.util.logging. >> >> http://cr.openjdk.java.net/~chegar/8146735/ > > Why not moving it to java.desktop Alan had suggested sun.util.logging, and it is already exported to java.desktop. > perhaps sun.awt.util.PerformanceLogger? I have no strong preference either way. > Are you thinking to keep in the base for the future java.base class to use? No. > No one uses this class since it was added and it?s good indication that it may unlikely be used by others. Ok. > In the future I think we should use System.Logger instead. Right, maybe a future issue could address this. -Chris. From coleen.phillimore at oracle.com Mon Jan 11 23:38:09 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Mon, 11 Jan 2016 18:38:09 -0500 Subject: RFR [9] 8141615: Add new public methods to sun.reflect.ConstantPool In-Reply-To: <5693AF52.6090109@oracle.com> References: <5645F9C6.2070802@oracle.com> <186AB3D9-5DAE-4558-AF8F-5043E22E5668@oracle.com> <1180432413.1310829.1447794240035.JavaMail.zimbra@u-pem.fr> <564C5CC4.5070003@oracle.com> <565478CB.2030005@oracle.com> <5654914E.1090603@oracle.com> <56588950.7050101@oracle.com> <566AB958.8010901@oracle.com> <9C46F7FF-BE6A-4E8C-99D0-54A71168455A@oracle.com> <566FD940.4090608@oracle.com> <61833FD9-418C-4B6E-9D31-05B43E770422@oracle.com> <56714765.5010009@oracle.com> <38C93A32-85EF-4B51-BADE-730A1C1CD656@oracle.com> <567271CE.3080604@oracle.com> <5693AF52.6090109@oracle.com> Message-ID: <56943CE1.3000207@oracle.com> This seems fine. Only one minor comment (where I don't need to see another webrev) + constantPoolHandle cp = constantPoolHandle(THREAD, sun_reflect_ConstantPool::get_cp(JNIHandles::resolve_non_null(obj))); would be shorter as + constantPoolHandle cp(THREAD, sun_reflect_ConstantPool::get_cp(JNIHandles::resolve_non_null(obj))); and avoid implicit copy construction. Thanks for adding tests. Coleen On 1/11/16 8:34 AM, Konstantin Shefov wrote: > Kindly reminder. > > Already approved by C. Thalinger and I. Ignatyev. > > Thanks > -Konstantin > > On 12/17/2015 11:26 AM, Konstantin Shefov wrote: >> Hi Coleen >> >> You have previously reviewed this enhancement and made a few comments >> I have resolved them, so could you look at the webrevs again, please? >> >> I have added tests, removed cp tags that are not in JVM spec (100 - >> 105) and made some other changes. >> >> JDK: http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.04 >> HOTSPOT: http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.02 >> >> Thanks >> -Konstantin >> >> >> On 12/16/2015 07:42 PM, Christian Thalinger wrote: >>> Looks good. Thanks. >>> >>>> On Dec 16, 2015, at 1:13 AM, Konstantin Shefov >>>> wrote: >>>> >>>> Christian >>>> >>>> I have fixed the enum so it uses "ENUMENTRY(int)" format now and >>>> does linear search. >>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.04/ >>>> >>>> -Konstantin >>>> >>>> On 12/15/2015 08:36 PM, Christian Thalinger wrote: >>>>>> On Dec 14, 2015, at 11:11 PM, Konstantin Shefov >>>>>> >>>>> > wrote: >>>>>> >>>>>> Hi Christian >>>>>> >>>>>> Thanks for reviewing, I have changed indents as you asked: >>>>>> >>>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.03 >>>>>> >>>>> Thanks. I?m still not comfortable with the enum. It would be >>>>> great if we could get the values from the VM like in JVMCI: >>>>> >>>>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/c036c7f17e09/src/jdk.vm.ci/share/classes/jdk.vm.ci.hotspot/src/jdk/vm/ci/hotspot/HotSpotConstantPool.java#l101 >>>>> >>>>> >>>>> but that would be overkill here. But I would like to see the enum >>>>> entries take the integer values as arguments, like here: >>>>> >>>>> http://hg.openjdk.java.net/jdk9/hs-comp/hotspot/file/c036c7f17e09/src/jdk.vm.ci/share/classes/jdk.vm.ci.sparc/src/jdk/vm/ci/sparc/SPARCKind.java#l27 >>>>> >>>>> >>>>> and either do a simple linear search to find the entry or build up >>>>> a table like the HotSpotConstantPool example above. >>>>> >>>>>> -Konstantin >>>>>> >>>>>> On 12/15/2015 06:23 AM, Christian Thalinger wrote: >>>>>>>> On Dec 11, 2015, at 1:54 AM, Konstantin Shefov >>>>>>>> >>>>>>> > wrote: >>>>>>>> >>>>>>>> Hello >>>>>>>> >>>>>>>> Please review the new version on the patch. >>>>>>>> >>>>>>>> New webrev: >>>>>>>> Webrev hotspot: >>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.02 >>>>>>>> >>>>>>>> Webrev jdk: >>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.02 >>>>>>>> >>>>>>> These newlines look ridiculous especially when it?s not even >>>>>>> needed: >>>>>>> >>>>>>> + // Returns a class reference index for a method or a field. >>>>>>> + public int getClassRefIndexAt >>>>>>> + (int index) { return >>>>>>> getClassRefIndexAt0 (constantPoolOop, index); } >>>>>>> >>>>>>> Either fix the indent or just add them like regular methods >>>>>>> should look like. >>>>>>> >>>>>>>> What has been changed: >>>>>>>> 1. Added tests for the new added methods. >>>>>>>> 2. Removed CP tag codes 100 - 105 from being passed to java and >>>>>>>> left only codes from the open JVM spec >>>>>>>> (https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.4-140). >>>>>>>> >>>>>>>> Thanks >>>>>>>> -Konstantin >>>>>>>> >>>>>>>> On 11/27/2015 07:48 PM, Konstantin Shefov wrote: >>>>>>>>> Coleen, >>>>>>>>> Thanks for review >>>>>>>>> >>>>>>>>> On 11/24/2015 07:33 PM, Coleen Phillimore wrote: >>>>>>>>>> I have a couple preliminary comments. First, there are no >>>>>>>>>> tests added with all this new functionality. Tests should be >>>>>>>>>> added with the functionality changeset, not promised afterward. >>>>>>>>> I will add tests. >>>>>>>>>> Secondly, I do not like that JDK code knows the >>>>>>>>>> implementation details of the JVM's constant pool tags. >>>>>>>>>> These should be only for internal use. >>>>>>>>> The package "sun.reflect" is for internal use only, so it >>>>>>>>> shouldn?t be a problem. >>>>>>>>>> My third comment is that I haven't looked carefully at this >>>>>>>>>> constant pool implementation but it seems very unsafe if the >>>>>>>>>> class is redefined and relies on an implementation detail in >>>>>>>>>> the JVM that can change. I will have more comments once I >>>>>>>>>> look more at the jvmti specification. >>>>>>>>>> >>>>>>>>>> thanks, >>>>>>>>>> Coleen >>>>>>>>>> >>>>>>>>>> On 11/24/15 9:48 AM, Konstantin Shefov wrote: >>>>>>>>>>> Hello >>>>>>>>>>> >>>>>>>>>>> Please, review modified webrev. >>>>>>>>>>> >>>>>>>>>>> I have added methods >>>>>>>>>>> getNameAndTypeRefIndexAt(int index) - to get name and type >>>>>>>>>>> entry index from a method, field or indy entry index; >>>>>>>>>>> getClassRefIndexAt(int index) - to get class entry index >>>>>>>>>>> from a method or field entry index; >>>>>>>>>>> >>>>>>>>>>> I removed previously added method >>>>>>>>>>> getInvokedynamicRefInfoAt(int index) - as it is no more >>>>>>>>>>> needed now. >>>>>>>>>>> >>>>>>>>>>> New webrev: >>>>>>>>>>> Webrev hotspot: >>>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.01 >>>>>>>>>>> >>>>>>>>>>> Webrev jdk: >>>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.01 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks >>>>>>>>>>> -Konstantin >>>>>>>>>>> >>>>>>>>>>> On 11/18/2015 02:11 PM, Konstantin Shefov wrote: >>>>>>>>>>>> Remi, >>>>>>>>>>>> >>>>>>>>>>>> Thanks for reviewing. Your suggestion sounds sensibly. >>>>>>>>>>>> May be it also has sense to make a method >>>>>>>>>>>> "getMethodRefNameAndTypeIndex(int index)" to acquire >>>>>>>>>>>> name-and-type entry index for methods also. >>>>>>>>>>>> >>>>>>>>>>>> -Konstantin >>>>>>>>>>>> >>>>>>>>>>>> On 11/18/2015 12:04 AM, Remi Forax wrote: >>>>>>>>>>>>> Hi Konstantin, >>>>>>>>>>>>> Technically, getInvokedynamicRefInfoAt should be >>>>>>>>>>>>> getNameAndTypeOfInvokedynamicRefInfoAt because it only >>>>>>>>>>>>> extract the nameAndType value of the InvokeDynamicRef. >>>>>>>>>>>>> >>>>>>>>>>>>> In term of API, I think it's better to decompose the API, >>>>>>>>>>>>> i.e. to have a method >>>>>>>>>>>>> int getInvokedynamicRefNameAndTypeIndex(int index) >>>>>>>>>>>>> that returns the nameAndType index and to reuse >>>>>>>>>>>>> getNameAndTypeRefInfoAt(index) to get the corresponding >>>>>>>>>>>>> array of Strings. >>>>>>>>>>>>> >>>>>>>>>>>>> cheers, >>>>>>>>>>>>> R?mi >>>>>>>>>>>>> >>>>>>>>>>>>> ----- Mail original ----- >>>>>>>>>>>>>> De: "Christian Thalinger" >>>>>>>>>>>>> > >>>>>>>>>>>>>> ?: "Konstantin Shefov" >>>>>>>>>>>>>> Cc: "hotspot-dev developers" >>>>>>>>>>>>>> , >>>>>>>>>>>>>> core-libs-dev at openjdk.java.net >>>>>>>>>>>>>> >>>>>>>>>>>>>> Envoy?: Lundi 16 Novembre 2015 23:41:45 >>>>>>>>>>>>>> Objet: Re: RFR [9] 8141615: Add new public methods to >>>>>>>>>>>>>> sun.reflect.ConstantPool >>>>>>>>>>>>>> >>>>>>>>>>>>>> [CC'ing core-libs-dev] >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Nov 13, 2015, at 4:55 AM, Konstantin Shefov >>>>>>>>>>>>>>> >>>>>>>>>>>>>> > wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hello >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Please review an enhancement to add three new methods to >>>>>>>>>>>>>>> sun.reflect.ConstantPool class. >>>>>>>>>>>>>>> The following methods are suggested: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> public String[] getNameAndTypeRefInfoAt(int index) - >>>>>>>>>>>>>>> returns string >>>>>>>>>>>>>>> representation of name and type from a NameAndType >>>>>>>>>>>>>>> constant pool entry >>>>>>>>>>>>>>> with the specified index >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> public String[] getInvokedynamicRefInfoAt(int index) - >>>>>>>>>>>>>>> returns string >>>>>>>>>>>>>>> representation of name and type from an InvokeDynamic >>>>>>>>>>>>>>> constant pool entry >>>>>>>>>>>>>>> with the specified index >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> public Tag getTagAt(int index) - returns a Tag enum >>>>>>>>>>>>>>> instance of a constant >>>>>>>>>>>>>>> pool entry with the specified index >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> These three methods could be used for testing API >>>>>>>>>>>>>>> working with constant >>>>>>>>>>>>>>> pool, e.g. JVMCI. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8141615 >>>>>>>>>>>>>>> Webrev hotspot: >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/hotspot/webrev.00 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Webrev jdk: >>>>>>>>>>>>>>> http://cr.openjdk.java.net/~kshefov/8141615/jdk/webrev.00 >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>> -Konstantin >> > From Alan.Bateman at oracle.com Tue Jan 12 08:20:56 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Jan 2016 08:20:56 +0000 Subject: RFR 8068938: javax.script package description should specify use of ServiceLoader In-Reply-To: <5693C280.2030508@oracle.com> References: <569337A0.6070200@oracle.com> <56936E34.7060200@oracle.com> <5693C280.2030508@oracle.com> Message-ID: <5694B768.7070900@oracle.com> On 11/01/2016 14:56, Sundararajan Athijegannathan wrote: > Incorporated the changes suggested. Updated webrev is @ > http://cr.openjdk.java.net/~sundar/8068938/webrev.01/ > This looks okay to me. -Alan From amy.lu at oracle.com Tue Jan 12 10:09:28 2016 From: amy.lu at oracle.com (Amy Lu) Date: Tue, 12 Jan 2016 18:09:28 +0800 Subject: JDK 9 RFR of JDK-8146762: Move back java/util/concurrent/Phaser/Basic.java to tier1 Message-ID: <5694D0D8.6070207@oracle.com> java/util/concurrent/Phaser/Basic.java This test has been moved from tier1 to tier2 (JDK-8142370) due to JDK-8141031. JDK-8141031 has been resolved in b96, and after that, not see this test fail. This patch is to move it back to tier1 bug: https://bugs.openjdk.java.net/browse/JDK-8146762 webrev: http://cr.openjdk.java.net/~amlu/8146762/webrev.00/ Thanks, Amy --- old/test/TEST.groups 2016-01-11 16:27:05.000000000 +0800 +++ new/test/TEST.groups 2016-01-11 16:27:05.000000000 +0800 @@ -31,7 +31,6 @@ -java/util/zip/TestLocalTime.java \ :jdk_util \ -java/util/WeakHashMap/GCDuringIteration.java \ - -java/util/concurrent/Phaser/Basic.java \ -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java sun/nio/cs/ISO8859x.java \ java/nio/Buffer \ @@ -41,7 +40,6 @@ tier2 = \ java/lang/ProcessHandle/TreeTest.java \ java/util/zip/TestLocalTime.java \ - java/util/concurrent/Phaser/Basic.java \ java/util/WeakHashMap/GCDuringIteration.java \ java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ :jdk_io \ From merkel05 at gmail.com Tue Jan 12 11:44:00 2016 From: merkel05 at gmail.com (Sergey Ustimenko) Date: Tue, 12 Jan 2016 14:44:00 +0300 Subject: JDK9 RFR of JDK-8140633: [PATCH] Method.getAnnotatedReturnType() returns empty annotations if return type is an array Message-ID: Hello, I've recently was working on the bugfix for https://bugs.openjdk.java.net/browse/JDK-8140633 and as a result i got that small fix. Also I've adopted presented in bugtracker test and done jtreg test. Here is patch. This is my first contribution try, so im probably missing something. # HG changeset patch # User fdesu # Date 1452597274 -10800 # Tue Jan 12 14:14:34 2016 +0300 # Node ID c7a11642c1a9bb38d558e8b237afce223c586d14 # Parent d09282af3b521f751a4b4d5056134877f4cd1b0e 8140633: Bugfix for Method.getAnnotatedReturnType() diff --git a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java --- a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java +++ b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java @@ -90,7 +90,7 @@ public static LocationInfo nestingForType(Type type, LocationInfo addTo) { if (isArray(type)) - return addTo; + return addTo.pushArray(); if (type instanceof Class) { Class clz = (Class)type; if (clz.getEnclosingClass() == null) diff --git a/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java new file mode 100644 --- /dev/null +++ b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java @@ -0,0 +1,58 @@ +/* + * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +import java.lang.annotation.*; +import java.lang.reflect.AnnotatedType; +import java.lang.reflect.Method; +import java.util.List; + +/* + * @test + * @bug 8140633 + * @summary Ensure that Method.getAnnotatedReturnType() not returns empty annotations when + * type is an array + * @author Sergey Ustimenko + */ +public class MethodAnnotatedReturnTypeTest { + + @Target({ ElementType.TYPE_PARAMETER, ElementType.TYPE_USE }) + @Retention(RetentionPolicy.RUNTIME) + public @interface MyAnnotation { + String value(); + } + + public interface ToImplement { + @MyAnnotation("sayHello") String sayHello(); + @MyAnnotation("sayHello1") List sayHello1(); + @MyAnnotation("sayHello2") String[] sayHello2(); + } + + public static void main(String[] args) { + for (Method m : ToImplement.class.getMethods()) { + AnnotatedType returnType = m.getAnnotatedReturnType(); + if(returnType.getAnnotations().length == 0) + throw new RuntimeException("Method.getAnnotatedReturnType() returned an empty array!"); + } + } + +} \ No newline at end of file -------------- next part -------------- # HG changeset patch # User fdesu # Date 1452597274 -10800 # Tue Jan 12 14:14:34 2016 +0300 # Node ID c7a11642c1a9bb38d558e8b237afce223c586d14 # Parent d09282af3b521f751a4b4d5056134877f4cd1b0e 8140633: Bugfix for Method.getAnnotatedReturnType() diff --git a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java --- a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java +++ b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java @@ -90,7 +90,7 @@ public static LocationInfo nestingForType(Type type, LocationInfo addTo) { if (isArray(type)) - return addTo; + return addTo.pushArray(); if (type instanceof Class) { Class clz = (Class)type; if (clz.getEnclosingClass() == null) diff --git a/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java new file mode 100644 --- /dev/null +++ b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java @@ -0,0 +1,58 @@ +/* + * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +import java.lang.annotation.*; +import java.lang.reflect.AnnotatedType; +import java.lang.reflect.Method; +import java.util.List; + +/* + * @test + * @bug 8140633 + * @summary Ensure that Method.getAnnotatedReturnType() not returns empty annotations when + * type is an array + * @author Sergey Ustimenko + */ +public class MethodAnnotatedReturnTypeTest { + + @Target({ ElementType.TYPE_PARAMETER, ElementType.TYPE_USE }) + @Retention(RetentionPolicy.RUNTIME) + public @interface MyAnnotation { + String value(); + } + + public interface ToImplement { + @MyAnnotation("sayHello") String sayHello(); + @MyAnnotation("sayHello1") List sayHello1(); + @MyAnnotation("sayHello2") String[] sayHello2(); + } + + public static void main(String[] args) { + for (Method m : ToImplement.class.getMethods()) { + AnnotatedType returnType = m.getAnnotatedReturnType(); + if(returnType.getAnnotations().length == 0) + throw new RuntimeException("Method.getAnnotatedReturnType() returned an empty array!"); + } + } + +} \ No newline at end of file From merkel05 at gmail.com Tue Jan 12 14:11:28 2016 From: merkel05 at gmail.com (Sergey Ustimenko) Date: Tue, 12 Jan 2016 17:11:28 +0300 Subject: [PATCH] 8140633: Method.getAnnotatedReturnType() returns empty annotations if return type is an array Message-ID: Hello, I've recently was working on the bugfix for https://bugs.openjdk.java.net/browse/JDK-8140633 and as a result i got that small fix. Also I've adopted presented in bugtracker test and done jtreg test. Here is patch. This is my first contribution try, so im probably missing something. # HG changeset patch # User fdesu # Date 1452597274 -10800 # Tue Jan 12 14:14:34 2016 +0300 # Node ID c7a11642c1a9bb38d558e8b237afce223c586d14 # Parent d09282af3b521f751a4b4d5056134877f4cd1b0e 8140633: Bugfix for Method.getAnnotatedReturnType() diff --git a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java --- a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java +++ b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java @@ -90,7 +90,7 @@ public static LocationInfo nestingForType(Type type, LocationInfo addTo) { if (isArray(type)) - return addTo; + return addTo.pushArray(); if (type instanceof Class) { Class clz = (Class)type; if (clz.getEnclosingClass() == null) diff --git a/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java new file mode 100644 --- /dev/null +++ b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java @@ -0,0 +1,58 @@ +/* + * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +import java.lang.annotation.*; +import java.lang.reflect.AnnotatedType; +import java.lang.reflect.Method; +import java.util.List; + +/* + * @test + * @bug 8140633 + * @summary Ensure that Method.getAnnotatedReturnType() not returns empty annotations when + * type is an array + * @author Sergey Ustimenko + */ +public class MethodAnnotatedReturnTypeTest { + + @Target({ ElementType.TYPE_PARAMETER, ElementType.TYPE_USE }) + @Retention(RetentionPolicy.RUNTIME) + public @interface MyAnnotation { + String value(); + } + + public interface ToImplement { + @MyAnnotation("sayHello") String sayHello(); + @MyAnnotation("sayHello1") List sayHello1(); + @MyAnnotation("sayHello2") String[] sayHello2(); + } + + public static void main(String[] args) { + for (Method m : ToImplement.class.getMethods()) { + AnnotatedType returnType = m.getAnnotatedReturnType(); + if(returnType.getAnnotations().length == 0) + throw new RuntimeException("Method.getAnnotatedReturnType() returned an empty array!"); + } + } + +} \ No newline at end of file -------------- next part -------------- # HG changeset patch # User fdesu # Date 1452597274 -10800 # Tue Jan 12 14:14:34 2016 +0300 # Node ID c7a11642c1a9bb38d558e8b237afce223c586d14 # Parent d09282af3b521f751a4b4d5056134877f4cd1b0e 8140633: Bugfix for Method.getAnnotatedReturnType() diff --git a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java --- a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java +++ b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java @@ -90,7 +90,7 @@ public static LocationInfo nestingForType(Type type, LocationInfo addTo) { if (isArray(type)) - return addTo; + return addTo.pushArray(); if (type instanceof Class) { Class clz = (Class)type; if (clz.getEnclosingClass() == null) diff --git a/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java new file mode 100644 --- /dev/null +++ b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java @@ -0,0 +1,58 @@ +/* + * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. + * + * This code is free software; you can redistribute it and/or modify it + * under the terms of the GNU General Public License version 2 only, as + * published by the Free Software Foundation. + * + * This code is distributed in the hope that it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License + * version 2 for more details (a copy is included in the LICENSE file that + * accompanied this code). + * + * You should have received a copy of the GNU General Public License version + * 2 along with this work; if not, write to the Free Software Foundation, + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. + * + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA + * or visit www.oracle.com if you need additional information or have any + * questions. + */ + +import java.lang.annotation.*; +import java.lang.reflect.AnnotatedType; +import java.lang.reflect.Method; +import java.util.List; + +/* + * @test + * @bug 8140633 + * @summary Ensure that Method.getAnnotatedReturnType() not returns empty annotations when + * type is an array + * @author Sergey Ustimenko + */ +public class MethodAnnotatedReturnTypeTest { + + @Target({ ElementType.TYPE_PARAMETER, ElementType.TYPE_USE }) + @Retention(RetentionPolicy.RUNTIME) + public @interface MyAnnotation { + String value(); + } + + public interface ToImplement { + @MyAnnotation("sayHello") String sayHello(); + @MyAnnotation("sayHello1") List sayHello1(); + @MyAnnotation("sayHello2") String[] sayHello2(); + } + + public static void main(String[] args) { + for (Method m : ToImplement.class.getMethods()) { + AnnotatedType returnType = m.getAnnotatedReturnType(); + if(returnType.getAnnotations().length == 0) + throw new RuntimeException("Method.getAnnotatedReturnType() returned an empty array!"); + } + } + +} \ No newline at end of file From chris.hegarty at oracle.com Tue Jan 12 14:21:19 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 12 Jan 2016 14:21:19 +0000 Subject: JDK 9 RFR of JDK-8146762: Move back java/util/concurrent/Phaser/Basic.java to tier1 In-Reply-To: <5694D0D8.6070207@oracle.com> References: <5694D0D8.6070207@oracle.com> Message-ID: <93AC3951-9115-4362-A7CF-00EE21CD8FB9@oracle.com> Thanks for doing this Amy. Reviewed. -Chris. > On 12 Jan 2016, at 10:09, Amy Lu wrote: > > java/util/concurrent/Phaser/Basic.java > > This test has been moved from tier1 to tier2 (JDK-8142370) due to JDK-8141031. JDK-8141031 has been resolved in b96, and after that, not see this test fail. This patch is to move it back to tier1 > > bug: https://bugs.openjdk.java.net/browse/JDK-8146762 > webrev: http://cr.openjdk.java.net/~amlu/8146762/webrev.00/ > > Thanks, > Amy > > > --- old/test/TEST.groups 2016-01-11 16:27:05.000000000 +0800 > +++ new/test/TEST.groups 2016-01-11 16:27:05.000000000 +0800 > @@ -31,7 +31,6 @@ > -java/util/zip/TestLocalTime.java \ > :jdk_util \ > -java/util/WeakHashMap/GCDuringIteration.java \ > - -java/util/concurrent/Phaser/Basic.java \ > -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java > sun/nio/cs/ISO8859x.java \ > java/nio/Buffer \ > @@ -41,7 +40,6 @@ > tier2 = \ > java/lang/ProcessHandle/TreeTest.java \ > java/util/zip/TestLocalTime.java \ > - java/util/concurrent/Phaser/Basic.java \ > java/util/WeakHashMap/GCDuringIteration.java \ > java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ > :jdk_io \ > From Alan.Bateman at oracle.com Tue Jan 12 15:41:17 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 12 Jan 2016 15:41:17 +0000 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: <56951E9D.2040509@oracle.com> On 11/01/2016 21:44, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter > break and my vacation), I provided responses to your messages > and an updated webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume > that you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > I think this looks good but we'll to decide which module to put this in. It can't be java.base (see design principles in JEP 200). If it's going into java.base temporarily then the top-level modules.xml will need to be updated to export the "jdk" package. -Alan From kumar.x.srinivasan at oracle.com Tue Jan 12 18:21:09 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Tue, 12 Jan 2016 10:21:09 -0800 Subject: Review request 8145409: Better detect JRE that JLI will be using In-Reply-To: <56799164.9080903@oracle.com> References: <56799164.9080903@oracle.com> Message-ID: <56954415.5020106@oracle.com> Hi Dmitry, src/java.base/windows/native/libjli/java_md.c 1) use NULL - GetModuleFileName(0, buf, bufsize); + GetModuleFileName(NULL, buf, bufsize); 2) extra LF ? + * Removes the trailing file name and one sub-folder from a path. + * + * If buf is "c:\foo\bin\javac", then put "c:\foo" into buf. and + return TruncatePath(buf); +} + + +/* 3) Is this comment correct ? /* + * Retrieves the path to the executable file of the current process + * and then truncates the path + */ Shouldn't it say Retrieves the path the JRE home by locating the executable of the current process ........ 4) Could you please add some comments to GetApplicationHomeFromAddress ? Shouldn't the method be called GetApplicationHomeFromDll ? + GetModuleHandleEx(dwFlags, (LPCSTR)&GetJREPath, &hModule); shouldn't you be testing the return value for success ? Thanks Kumar > Hello, > > Please review the following patch > > https://bugs.openjdk.java.net/browse/JDK-8145409 > http://cr.openjdk.java.net/~dcherepanov/8145409/webrev.v0/ > > The patch improves GetJREPath() on Windows and implements a fallback > for applications with custom native launcher and private JRE. > > Thanks, > > Dmitry > From chris.hegarty at oracle.com Tue Jan 12 18:30:11 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 12 Jan 2016 18:30:11 +0000 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: <21AB656E-274D-4720-8499-543631EB5F31@oracle.com> References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> <21AB656E-274D-4720-8499-543631EB5F31@oracle.com> Message-ID: Webrev updated in-place, moved to sun.awt.utli ( desktop module ) http://cr.openjdk.java.net/~chegar/8146735/ -Chris. On 11 Jan 2016, at 22:44, Chris Hegarty wrote: > >> On 11 Jan 2016, at 21:27, Mandy Chung wrote: >> >> On Jan 10, 2016, at 1:19 AM, Chris Hegarty wrote: >>> >>> In preparation for JEP 260, and as per the suggestion over on the core libs[1], >>> sun.misc.PerformanceLogger should be moved to sun.util.logging. >>> >>> http://cr.openjdk.java.net/~chegar/8146735/ >> >> Why not moving it to java.desktop > > Alan had suggested sun.util.logging, and it is already exported > to java.desktop. > >> perhaps sun.awt.util.PerformanceLogger? > > I have no strong preference either way. > >> Are you thinking to keep in the base for the future java.base class to use? > > No. > >> No one uses this class since it was added and it?s good indication that it may unlikely be used by others. > > Ok. > >> In the future I think we should use System.Logger instead. > > Right, maybe a future issue could address this. > > -Chris. > From aleksey.shipilev at oracle.com Tue Jan 12 20:49:29 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 12 Jan 2016 23:49:29 +0300 Subject: use of Unsafe for ASCII detection In-Reply-To: References: Message-ID: <569566D9.4060105@oracle.com> On 01/07/2016 02:21 AM, Martin Buchholz wrote: > A stupid benchmark (sorry Aleksey - maybe you already have a jmh > version of this?) http://cr.openjdk.java.net/~martin/Utf8Bench.java Okay, I'll bite: http://cr.openjdk.java.net/~shade/scratch/UTF8Scan.java As Andrew already pointed out, the trick is to get the ByteBuffer byte order to native, and use JDK 9 with getLongUnaligned bells and whistles. As Paul suggested, VarHandles views could be used to always avoid ByteBuffer allocation, even though compilers are moderately successful in removing them. > It's hard to resist Unsafe when it gives 3x improvement in a > performance hotspot. Maybe C2 can optimize the simple loop even more > than it is today? It would seem that with JDK 9, you don't need Unsafe in this scenario. ByteBuffers provide a comparable performance, and VarHandles would push the envelope a bit further, since we are implicitly optimizing the internal APIs to boost them up. Thanks, -Aleksey From Roger.Riggs at Oracle.com Tue Jan 12 21:31:47 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 12 Jan 2016 16:31:47 -0500 Subject: RFR 9: 8146671 [TEST BUG] : java/lang/ref/CleanerTest.java required more memory for -UseCompressedOops runs Message-ID: <569570C3.50209@Oracle.com> Please review an update to a CleanerTest; the 4m limit is not needed and causes a problem when run with CompressedOops Webrev: http://cr.openjdk.java.net/~rriggs/webrev-cleanertest-8146671/ Issue: https://bugs.openjdk.java.net/browse/JDK-8146671 Thanks, Roger From mandy.chung at oracle.com Tue Jan 12 21:58:17 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 12 Jan 2016 13:58:17 -0800 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> <21AB656E-274D-4720-8499-543631EB5F31@oracle.com> Message-ID: > On Jan 12, 2016, at 10:30 AM, Chris Hegarty wrote: > > Webrev updated in-place, moved to sun.awt.utli ( desktop module ) > > http://cr.openjdk.java.net/~chegar/8146735/ This looks fine. Good to see it moved to java.desktop. Mandy From mandy.chung at oracle.com Tue Jan 12 22:25:53 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 12 Jan 2016 14:25:53 -0800 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: > On Jan 11, 2016, at 1:44 PM, Iris Clark wrote: > > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter > break and my vacation), I provided responses to your messages > and an updated webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ It?s good to see that jdk.OracleVersion has been removed and use jdk.Version to obtain the fourth element of the version number. This patch looks good in general. Minor comment: 273 current = parse(System.getProperty("java.version")); System.getProperty will do a permission check and it needs to be called within a doPrivileged block. 154 * @see JEP 223: New Version-String Scheme Does the javadoc have the essential information from this JEP? Wonder if this @see is necessary. Alan already comment on the ?jdk? package that needs to find the proper module to export it (that?s a future RFE) and modules.xml should be updated. Mandy From kumar.x.srinivasan at oracle.com Tue Jan 12 22:34:48 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Tue, 12 Jan 2016 14:34:48 -0800 Subject: RFR 8124977 cmdline encoding challenges on Windows In-Reply-To: <56941A34.8090608@oracle.com> References: <558C017A.2020708@oracle.com> <5651E9AA.9060908@oracle.com> <56941A34.8090608@oracle.com> Message-ID: <56957F88.6060708@oracle.com> Hi Martin, Vladimir, It was suggested that this patch be tested on localized Windows machines and/or trying with the various Windows native encodings, appreciate if you can verify this as well. Thanks Kumar On 1/11/2016 1:10 PM, Kumar Srinivasan wrote: > Hi, > > Was on vacation, I started to prepare the patch from webrev.04 > for integration. Please note: made some adjustments to your > patch to pass jcheck, ie. usage of tabs and space at line endings, > and modifications to Copyright dates. > > Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE. > I have attached a patch to for your reference. > > However, there is a regression test failure on Windows, > jdk/test/tools/launcher/I18NTest.java > > ---Test info---- > Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java > > ++++Test Output++++ > javac: file not found: i18nH?lloWorld.java > ----End test info----- > > Have you run all the launcher regression tests with this changeset ? > > Thanks > Kumar > >> Hi Kumar, just wondering if there are any updates on processing this >> submission. >> Thanks! >> >> -----Original Message----- >> From: Vladimir Shcherbakov >> Sent: Wednesday, November 25, 2015 2:38 PM >> To: Kumar Srinivasan ; Martin Sawicki >> >> Cc: Kirk Shoop ; core-libs-dev Libs >> >> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows >> >> Hi Kumar, >> >> Please find updated webreview here: >> http://cr.openjdk.java.net/~kshoop/8124977/webrev.04/ >> >> Thanks, >> Vladimir. >> >> -----Original Message----- >> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] >> Sent: Sunday, November 22, 2015 8:14 AM >> To: Martin Sawicki >> Cc: Kirk Shoop ; Vladimir Shcherbakov >> ; core-libs-dev Libs >> >> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >> >> >> Hi Martin, et. al., >> >> Sorry for not getting back earlier, I am very busy right now with my >> other large commitments for JDK9. >> >> I will sponsor this "enhancement/bug fix" sometime in the new year, >> meanwhile, there is the changeset [1] which is likely to cause merge >> conflicts, and perhaps logic issues. >> >> Thanks >> Kumar >> >> [1] >> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.openjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7Mg4lt2P0sdzLS%2fT9Q%3d >>> Hi all >>> Here's an updated webrev attempting to take into account the various >>> pieces of feedback we have received: >>> >>> Issue: >>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. >>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40micro >>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7c >>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E%3 >>> d >>> Webrev: >>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.openj >>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashch% >>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af9 >>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJnWY >>> 0%3d >>> >>> (Vladimir Shcherbakov is now working on this from our side) >>> >>> Looking forward to any other feedback. >>> Thanks >>> >>> -----Original Message----- >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >>> Behalf Of Kumar Srinivasan >>> Sent: Thursday, June 25, 2015 6:26 AM >>> To: Kirk Shoop (MS OPEN TECH) >>> Cc: Valery Kopylov (Akvelon) ; core-libs-dev >>> Libs >>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >>> >>> Hi Kirk, >>> >>> Thanks for proposing this change. >>> >>> If you notice all the posix calls are wrapped in JLI_* this gives us >>> the ability to use "W" functions. I almost got it done, several >>> years ago, but we upgraded to VS2010 and my work based on VS2003 >>> keeled over, meanwhile my focus was "shifted" to something else. >>> >>> main.c: is really envisioned to be a stub compiled by the tool >>> launchers, like java, javac, javah, jar etc. I prefer to see all the >>> heavy logic in this file moved to the platform specific file >>> windows/java_md.* >>> >>> For the reason specified above we need to move fprintf or any naked >>> posix calls to JLI_* indirections. >>> >>> I don't see any tests ? The tests must be written in java and placed >>> in jdk/test/tools/launcher, there is a helper framework >>> TestHelper.java. >>> >>> There are other changes in nio, charsets etc, this will be reviewed >>> by my colleague specializing in that area (Sherman) cc'ed. >>> >>> >>> Thanks >>> >>> Kumar >>> >>> >>> >>> >>> >>> >>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote: >>>> Hi, >>>> >>>> Issue: >>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs >>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic >>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2 >>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq >>>> 9E%3d >>>> >>>> Webrev: >>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open >>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40microsof >>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7cd0 >>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M%3d >>>> >>>> This webrev intends to address interaction between Windows console >>>> and java apps. >>>> >>>> Two switches were added that change the behavior of the launcher. >>>> The defaults do not change the launcher behavior. >>>> >>>> -Dwindows.UnicodeConsole=true - switches on Unicode support in >>>> the Windows console. This optional switch causes the launcher to >>>> call GetCommandLineW() and parse the arguments in unicode. It also >>>> modifies how the codepage for console output is selected. >>>> >>>> -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset >>>> to use; If not specified, UTF-8 is used by default. Ignored when >>>> windows.UnicodeConsole is not set to true. When the first switch is >>>> used, this optional switch allows the codepage for console output >>>> to be controlled. >>>> >>>> I would like to get feedback on the approach here and any >>>> additional work that is required solve these particular Unicode >>>> issues on Windows. >>>> >>>> Kirk > From kevin.rushforth at oracle.com Tue Jan 12 22:44:37 2016 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Tue, 12 Jan 2016 14:44:37 -0800 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> <21AB656E-274D-4720-8499-543631EB5F31@oracle.com> Message-ID: <569581D5.8050802@oracle.com> I note that JavaFX optionally uses sun.misc.PerformanceLogger this in one place (via reflection). We'll need to file a new issue to clean this up. -- Kevin Mandy Chung wrote: >> On Jan 12, 2016, at 10:30 AM, Chris Hegarty wrote: >> >> Webrev updated in-place, moved to sun.awt.utli ( desktop module ) >> >> http://cr.openjdk.java.net/~chegar/8146735/ >> > > This looks fine. Good to see it moved to java.desktop. > > Mandy > > From huizhe.wang at oracle.com Tue Jan 12 22:46:07 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 12 Jan 2016 14:46:07 -0800 Subject: RFR 9: 8146671 [TEST BUG] : java/lang/ref/CleanerTest.java required more memory for -UseCompressedOops runs In-Reply-To: <569570C3.50209@Oracle.com> References: <569570C3.50209@Oracle.com> Message-ID: <5695822F.3070202@oracle.com> Looks good, Roger. -Joe On 1/12/2016 1:31 PM, Roger Riggs wrote: > Please review an update to a CleanerTest; the 4m limit is not needed > and causes a problem when run with CompressedOops > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-cleanertest-8146671/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146671 > > Thanks, Roger > > From joe.darcy at oracle.com Wed Jan 13 01:06:37 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 12 Jan 2016 17:06:37 -0800 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: <5695A31D.1050405@oracle.com> Hi Iris, Catching up on old reviews, the regular expression for versions is given as ^[1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*$ Is this equivalent to ^[1-9][0-9]*(\.[0-9])* Trying to put this in words, "The version number starts with a sequence of digits where the leading digit is not "0". Afterward, there are zero or more period separated groups of digits with no restrictions on the digits. Note that this implies The version number ends with a digit group." I suggest changing the class-level javadoc discussing comparisons to defer to the various compare methods. It would be helpful to not which compare methods are consistent with equals and which are not. Thanks, -Joe On 1/11/2016 1:44 PM, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter > break and my vacation), I provided responses to your messages > and an updated webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume > that you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > > Regards, > iris > > -----Original Message----- > From: Iris Clark > Sent: Friday, December 18, 2015 1:55 PM > To: Joe Darcy; Mandy Chung > Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net > Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion > > Hi, Joe. > > Thanks for the review comments. > >>>> http://cr.openjdk.java.net/~irisa/verona/8072379/webrev.1/ >> Is the intention that downstream JDK distributions, such as IcedTea, >> whether based on OpenJDK or otherwise, would provide their own >> specialization of the jdk.Version class? > No. Downstream users may provide their own specialization, but there is no requirement that they must do so. > > jdk.OracleVersion has been removed. The functionality it provided, access to the fourth element of the version number, is trivially accessed by existing methods in jdk.Version. > > I've filed the following bug to address the more general use case: > > 8145793: Provide vendor-specific interpretation of a JDK version string > https://bugs.openjdk.java.net/browse/JDK-8145793 > >> The equals / hashCode behavior of Version and OracleVersion is bit >> surprising and I think somewhat underspecified given the possibility >> of defining subclasses. > Given the above regarding downstream use, I've make jdk.Version final and the constructor is now private. Oracle or any other JDK distribution wishing to impose additional semantics to the version string interpretation will need to do so via encapsulation rather than inheritance. > >> How is this API supposed to behave if the component version strings >> have a numerical value greater than Integer.MAX_VALUE? > From the specification of the only instance method, Version.parse(): > > * @throws NumberFormatException > * If an element of the version number or the build number cannot > * be represented as an {@link Integer} > >> Was using Longs to record numerical versions rather than Integers >> considered? > Yes. In an informal poll conducted during implementation, it was thought that 2^32 bits would be more than sufficient to represent the components of typical version numbers. > > An updated webrev is forthcoming. > > Thanks, > iris > > PS: I believe that we are both out the week of 21 December, so I don't expect to hear back from you until the New Year. From huizhe.wang at oracle.com Wed Jan 13 05:18:37 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 12 Jan 2016 21:18:37 -0800 Subject: RFR (JAXP): 8146606: Catalog.matchSystem() appends an extra '/' to the matched result Message-ID: <5695DE2D.2050300@oracle.com> Please review a fix to the match method of RewriteSystem and RewriteUri that resulted in an extra slash in the URI. Webrev: http://cr.openjdk.java.net/~joehw/jdk9/8146606/webrev/ Issue: https://bugs.openjdk.java.net/browse/JDK-8146606 Thanks, Joe From daniel.fuchs at oracle.com Wed Jan 13 09:01:13 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Wed, 13 Jan 2016 10:01:13 +0100 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <5695A31D.1050405@oracle.com> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <5695A31D.1050405@oracle.com> Message-ID: <56961259.7000503@oracle.com> Hi Joe, On 1/13/16 2:06 AM, Joseph D. Darcy wrote: > Hi Iris, > > Catching up on old reviews, the regular expression for versions is given as > > ^[1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*$ > > Is this equivalent to > > ^[1-9][0-9]*(\.[0-9])* Iris's regexp will disallow trailing .0 9.0.1 will match, but 9.0 and 9.0.0 will not - the version should be plain 9 in those latter cases as documented in the javadoc. I believe this is important because it ensures that the version string is always in canonical form, which makes it possible to simply compare the 'versions' list (List) within the various variations of equals... So the regexp matching will eliminate non canonical forms without having to perform further checks down the road. Took me a while to figure it, I had to look at the test to pinpoint it. Now that I understand, I find it quite elegant :-) best regards, -- daniel From aph at redhat.com Wed Jan 13 09:35:50 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 13 Jan 2016 09:35:50 +0000 Subject: use of Unsafe for ASCII detection In-Reply-To: <569566D9.4060105@oracle.com> References: <569566D9.4060105@oracle.com> Message-ID: <56961A76.4000707@redhat.com> On 12/01/16 20:49, Aleksey Shipilev wrote: > Okay, I'll bite: > http://cr.openjdk.java.net/~shade/scratch/UTF8Scan.java Great job, Aleksey. Far more thorough than I would have done! Andrew. From dmitry.cherepanov at oracle.com Wed Jan 13 12:06:13 2016 From: dmitry.cherepanov at oracle.com (Dmitry Cherepanov) Date: Wed, 13 Jan 2016 15:06:13 +0300 Subject: Review request 8145409: Better detect JRE that JLI will be using In-Reply-To: <56954415.5020106@oracle.com> References: <56799164.9080903@oracle.com> <56954415.5020106@oracle.com> Message-ID: <56963DB5.9040605@oracle.com> Hi Kumar, Thanks for the comments. On 1/12/16 9:21 PM, Kumar Srinivasan wrote: > > Hi Dmitry, > > src/java.base/windows/native/libjli/java_md.c > > 1) use NULL > > - GetModuleFileName(0, buf, bufsize); > > + GetModuleFileName(NULL, buf, bufsize); OK. > > > 2) extra LF ? > > + * Removes the trailing file name and one sub-folder from a path. > + * > + * If buf is "c:\foo\bin\javac", then put "c:\foo" into buf. > > and > > + return TruncatePath(buf); > +} > + > + > +/* Fixed. > > > 3) Is this comment correct ? > > /* > + * Retrieves the path to the executable file of the current process > + * and then truncates the path > + */ > > Shouldn't it say Retrieves the path the JRE home by locating the > executable of > the current process ........ Fixed too. > > > 4) Could you please add some comments to GetApplicationHomeFromAddress ? > Shouldn't the method be called GetApplicationHomeFromDll ? Good idea. I've updated the name and comments. > > + GetModuleHandleEx(dwFlags, (LPCSTR)&GetJREPath, &hModule); > > shouldn't you be testing the return value for success ? Fixed. Updated webrev: http://cr.openjdk.java.net/~dcherepanov/8145409/webrev.v1/ Thanks Dmitry From lance.andersen at oracle.com Wed Jan 13 13:30:14 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 13 Jan 2016 08:30:14 -0500 Subject: RFR (JAXP): 8146606: Catalog.matchSystem() appends an extra '/' to the matched result In-Reply-To: <5695DE2D.2050300@oracle.com> References: <5695DE2D.2050300@oracle.com> Message-ID: Looks OK Joe Best Lance On Jan 13, 2016, at 12:18 AM, huizhe wang wrote: > Please review a fix to the match method of RewriteSystem and RewriteUri that resulted in an extra slash in the URI. > > Webrev: > http://cr.openjdk.java.net/~joehw/jdk9/8146606/webrev/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8146606 > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From chris.hegarty at oracle.com Wed Jan 13 13:45:56 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 13 Jan 2016 13:45:56 +0000 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: <569581D5.8050802@oracle.com> References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> <21AB656E-274D-4720-8499-543631EB5F31@oracle.com> <569581D5.8050802@oracle.com> Message-ID: <56965514.9080301@oracle.com> On 12/01/16 22:44, Kevin Rushforth wrote: > I note that JavaFX optionally uses sun.misc.PerformanceLogger this in > one place (via reflection). We'll need to file a new issue to clean this up. Thanks Kevin. Can you file the FX bug? -Chris. > -- Kevin > > > Mandy Chung wrote: >>> On Jan 12, 2016, at 10:30 AM, Chris Hegarty wrote: >>> >>> Webrev updated in-place, moved to sun.awt.utli ( desktop module ) >>> >>> http://cr.openjdk.java.net/~chegar/8146735/ >>> >> >> This looks fine. Good to see it moved to java.desktop. >> >> Mandy >> >> From kubota.yuji at gmail.com Wed Jan 13 14:31:24 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Wed, 13 Jan 2016 23:31:24 +0900 Subject: [PING] Potential infinite waiting at JMXConnection#createConnection In-Reply-To: References: <568A86DF.2030609@oracle.com> Message-ID: Hi all, Can somebody please review and sponsor this fix ? Thanks, Yuji 2016-01-05 17:56 GMT+09:00 KUBOTA Yuji : > Hi Jaroslav and core-libs-dev, > > Thank Jaroslav for your kindness! > > For core-libs-dev members, links the information about this issue. > > * details of problem > http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002152.html > > * patch > http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch > > * testcase for reproduce > http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/testProgram > http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-December/018415.html > > Could you please review these reports? > Hope this patch helps to community. > > Thanks, > Yuji > > 2016-01-04 23:51 GMT+09:00 Jaroslav Bachorik : >> Hi Yuji, >> >> On 4.1.2016 15:14, KUBOTA Yuji wrote: >>> >>> Hi all, >>> >>> Could you please review this patch? >> >> >> Sorry for the long delay. Shanliang has not been present for some time and >> probably this slipped the attention of the others. >> >> However, core-libs mailing list might be more appropriate place to review >> this change since you are dealing with s.r.t.t.TCPChannel >> (http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch) >> >> Regards, >> >> -JB- From kumar.x.srinivasan at oracle.com Wed Jan 13 14:48:54 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Wed, 13 Jan 2016 06:48:54 -0800 Subject: Review request 8145409: Better detect JRE that JLI will be using In-Reply-To: <56963DB5.9040605@oracle.com> References: <56799164.9080903@oracle.com> <56954415.5020106@oracle.com> <56963DB5.9040605@oracle.com> Message-ID: <569663D6.7060103@oracle.com> Hi Dmitry, Approved. Please make sure all the core tests are run using jprt and integrate this into jdk9/dev, if you can't do so, let me know. Thanks Kumar On 1/13/2016 4:06 AM, Dmitry Cherepanov wrote: > Hi Kumar, > > Thanks for the comments. > > On 1/12/16 9:21 PM, Kumar Srinivasan wrote: >> >> Hi Dmitry, >> >> src/java.base/windows/native/libjli/java_md.c >> >> 1) use NULL >> >> - GetModuleFileName(0, buf, bufsize); >> >> + GetModuleFileName(NULL, buf, bufsize); > OK. >> >> >> 2) extra LF ? >> >> + * Removes the trailing file name and one sub-folder from a path. >> + * >> + * If buf is "c:\foo\bin\javac", then put "c:\foo" into buf. >> >> and >> >> + return TruncatePath(buf); >> +} >> + >> + >> +/* > Fixed. > >> >> >> 3) Is this comment correct ? >> >> /* >> + * Retrieves the path to the executable file of the current process >> + * and then truncates the path >> + */ >> >> Shouldn't it say Retrieves the path the JRE home by locating the >> executable of >> the current process ........ > Fixed too. >> >> >> 4) Could you please add some comments to GetApplicationHomeFromAddress ? >> Shouldn't the method be called GetApplicationHomeFromDll ? > Good idea. I've updated the name and comments. > >> >> + GetModuleHandleEx(dwFlags, (LPCSTR)&GetJREPath, &hModule); >> >> shouldn't you be testing the return value for success ? > Fixed. > > Updated webrev: > http://cr.openjdk.java.net/~dcherepanov/8145409/webrev.v1/ > > Thanks > Dmitry > From kevin.rushforth at oracle.com Wed Jan 13 16:31:11 2016 From: kevin.rushforth at oracle.com (Kevin Rushforth) Date: Wed, 13 Jan 2016 08:31:11 -0800 Subject: RFR [9] 8146735: Move sun.misc.PerformanceLogger to sun.util.logging In-Reply-To: <56965514.9080301@oracle.com> References: <72A8BC72-473F-452B-84F5-B427CB62A426@oracle.com> <221951B6-5D60-4E3E-82F4-BD1F98A5D1F2@oracle.com> <21AB656E-274D-4720-8499-543631EB5F31@oracle.com> <569581D5.8050802@oracle.com> <56965514.9080301@oracle.com> Message-ID: <56967BCF.2070405@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8147010 -- Kevin Chris Hegarty wrote: > On 12/01/16 22:44, Kevin Rushforth wrote: >> I note that JavaFX optionally uses sun.misc.PerformanceLogger this in >> one place (via reflection). We'll need to file a new issue to clean >> this up. > > Thanks Kevin. Can you file the FX bug? > > -Chris. > > >> -- Kevin >> >> >> Mandy Chung wrote: >>>> On Jan 12, 2016, at 10:30 AM, Chris >>>> Hegarty wrote: >>>> >>>> Webrev updated in-place, moved to sun.awt.utli ( desktop module ) >>>> >>>> http://cr.openjdk.java.net/~chegar/8146735/ >>>> >>> >>> This looks fine. Good to see it moved to java.desktop. >>> >>> Mandy >>> >>> From steve.drach at oracle.com Wed Jan 13 16:47:20 2016 From: steve.drach at oracle.com (Steve Drach) Date: Wed, 13 Jan 2016 08:47:20 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> Message-ID: Hi, CCC had some suggestions to improve the code that was previously approved during review. As a consequence, I had to make significant changes to the API, and I believe the code need further review. Please review the latest iteration of the webrev for runtime support of multi-release jar files. Issue: https://bugs.openjdk.java.net/browse/JDK-8132734 JEP 238: https://bugs.openjdk.java.net/browse/JDK-8047305 Webrev: http://cr.openjdk.java.net/~sdrach/8132734/webrev.02/ Here?s a list of things that were changed: 1. Removed setVersion, setRuntimeVersioned, and getRuntimeVersionedEntry. Also removed the configuration lock. 2. Added a Release enum to represent releases. 3. Added a new constructor that uses the enum as one of it?s arguments. This is the only way to make multi-release JarFile objects. 4. Added new code to provide runtime versioning for class loaders using the new constructor. 4. Made the ?version? int final, removing the volatile modifier. 5. Made the two remaining public methods, getVersion and isMultiRelease final. 6. Added implSpec to getEntry and getJarEntry so potential subclassers are aware of how things work. 7. Rewrote and added tests to deal with the new constructor and the Release enum. 8. Added a simple http server to test getting jar files from both a local files system and from the network. Thanks, Steve From iris.clark at oracle.com Wed Jan 13 21:54:44 2016 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 13 Jan 2016 13:54:44 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <56951E9D.2040509@oracle.com> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> Message-ID: Hi, Alan. Thanks for looking at this (hopefully) one last time. > It can't be java.base (see design principles in JEP 200). > If it's going into java.base temporarily then the top-level > modules.xml will need to be updated to export the "jdk" package. This diff has been applied to modules.xml: diff -r 6fefc5bce180 modules.xml --- a/modules.xml Wed Jan 13 13:56:19 2016 +0000 +++ b/modules.xml Wed Jan 13 13:46:56 2016 -0800 @@ -205,6 +205,9 @@ javax.security.cert + jdk + + jdk.net It essentially reverts your 8049422 change [0] to that file. I will not re-add jdk to the javadoc bundle for javac trees API since that is not an appropriate location. I filed the following bug to track publication of jdk.Version: 8144069: Determine correct publication for jdk.Version API https://bugs.openjdk.java.net/browse/JDK-8144069 When this came up earlier, I filed this bug to track finding a more appropriate module for jdk.Version: 8144062: Determine appropriate module for jdk.Version https://bugs.openjdk.java.net/browse/JDK-8144062 Thanks, iris [0] http://hg.openjdk.java.net/jdk9/dev/rev/1bee5efa73e3 -----Original Message----- From: Alan Bateman Sent: Tuesday, January 12, 2016 7:41 AM To: Iris Clark; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: Re: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion On 11/01/2016 21:44, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter break and > my vacation), I provided responses to your messages and an updated > webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume that > you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036 > 904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > I think this looks good but we'll to decide which module to put this in. It can't be java.base (see design principles in JEP 200). If it's going into java.base temporarily then the top-level modules.xml will need to be updated to export the "jdk" package. -Alan From hdthanh at tma.com.vn Thu Jan 14 03:53:22 2016 From: hdthanh at tma.com.vn (Thanh Hong Dai) Date: Thu, 14 Jan 2016 10:53:22 +0700 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> Message-ID: <04d501d14e7f$1ee19b50$5ca4d1f0$@tma.com.vn> Hi, Some comment on the regex (and also the JEP): ([1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*)(\-([a-zA-Z0-9]+))?((\+)(0|[1-9][0-9]*)?)?(-([\-a-zA-Z0-9\.]+))? - The outer most quantifier in (((\.0)*\.[1-9][0-9]*)*)* is redundant and a source of catastrophic backtracking. You only need ((\.0)*\.[1-9][0-9]*)*. - The outside - in PRE part (\-([a-zA-Z0-9]+))? doesn't need escaping. Simplify it to (-([a-zA-Z0-9]+))? - Do you want to allow only a plus without the number in the BUILD part? Why do you capture the +? ((\+)(0|[1-9][0-9]*)?)? - Dot loses meaning in the character class, and hyphen loses meaning at the beginning or at the end of the character class. You can simplify the OPT part to (-([-a-zA-Z0-9.]+))? - You might want to consider using named-capturing groups instead of numbered capturing groups. Also, consider using non-capturing groups for groups you don't need to extract. Best regards, Hong Dai Thanh. -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Iris Clark Sent: Thursday, 14 January, 2016 4:55 AM To: Alan Bateman ; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Alan. Thanks for looking at this (hopefully) one last time. > It can't be java.base (see design principles in JEP 200). > If it's going into java.base temporarily then the top-level > modules.xml will need to be updated to export the "jdk" package. This diff has been applied to modules.xml: diff -r 6fefc5bce180 modules.xml --- a/modules.xml Wed Jan 13 13:56:19 2016 +0000 +++ b/modules.xml Wed Jan 13 13:46:56 2016 -0800 @@ -205,6 +205,9 @@ javax.security.cert + jdk + + jdk.net It essentially reverts your 8049422 change [0] to that file. I will not re-add jdk to the javadoc bundle for javac trees API since that is not an appropriate location. I filed the following bug to track publication of jdk.Version: 8144069: Determine correct publication for jdk.Version API https://bugs.openjdk.java.net/browse/JDK-8144069 When this came up earlier, I filed this bug to track finding a more appropriate module for jdk.Version: 8144062: Determine appropriate module for jdk.Version https://bugs.openjdk.java.net/browse/JDK-8144062 Thanks, iris [0] http://hg.openjdk.java.net/jdk9/dev/rev/1bee5efa73e3 -----Original Message----- From: Alan Bateman Sent: Tuesday, January 12, 2016 7:41 AM To: Iris Clark; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: Re: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion On 11/01/2016 21:44, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter break and > my vacation), I provided responses to your messages and an updated > webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume that > you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037 > 062.html > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036 > 904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > I think this looks good but we'll to decide which module to put this in. It can't be java.base (see design principles in JEP 200). If it's going into java.base temporarily then the top-level modules.xml will need to be updated to export the "jdk" package. -Alan From iris.clark at oracle.com Thu Jan 14 05:27:04 2016 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 13 Jan 2016 21:27:04 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: Hi, Mandy. Thanks for looking at this webrev again. > 273 current = parse(System.getProperty("java.version")); > > System.getProperty will do a permission check and it needs to be called within a doPrivileged block. Nice catch! I've added the permission check and the associated @throws. > 154 * @see JEP 223: New Version-String Scheme > > Does the javadoc have the essential information from this JEP? Wonder if this @see is necessary. I thought it might be nice to include the JEP reference as it contains additional data about versioning beyond the JavaDoc for this method; however, it's not critical to the understanding or use of this class. For some reason I'd thought that there was precedence in including JEP numbers as @see references, but I find no evidence of that. I've removed the @see. These are the diffs to address both of your comments: $ diff Version.java_save Version.java 28a29,30 > import java.security.AccessController; > import java.security.PrivilegedAction; 154,155d155 < * @see JEP 223: New Version-String Scheme < * 268a269,274 > * @throws SecurityException > * If a security manager exists and its {@link > * SecurityManager#checkPropertyAccess(String) > * checkPropertyAccess} method does not allow access to the > * system property "java.version" > * 272,273c278,285 < if (current == null) < current = parse(System.getProperty("java.version")); --- > if (current == null) { > current = parse(AccessController.doPrivileged( > new PrivilegedAction<>() { > public String run() { > return System.getProperty("java.version"); > } > })); > } Regards, iris -----Original Message----- From: Mandy Chung Sent: Tuesday, January 12, 2016 2:26 PM To: Iris Clark Cc: Mandy Chung; Joe Darcy; Magnus Ihse Bursie; Roger Riggs; Alan Bateman; core-libs-dev; verona-dev at openjdk.java.net Subject: Re: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion > On Jan 11, 2016, at 1:44 PM, Iris Clark wrote: > > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter break and > my vacation), I provided responses to your messages and an updated > webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ It?s good to see that jdk.OracleVersion has been removed and use jdk.Version to obtain the fourth element of the version number. This patch looks good in general. Minor comment: 273 current = parse(System.getProperty("java.version")); System.getProperty will do a permission check and it needs to be called within a doPrivileged block. 154 * @see JEP 223: New Version-String Scheme Does the javadoc have the essential information from this JEP? Wonder if this @see is necessary. Alan already comment on the ?jdk? package that needs to find the proper module to export it (that?s a future RFE) and modules.xml should be updated. Mandy From huizhe.wang at oracle.com Thu Jan 14 05:38:38 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 13 Jan 2016 21:38:38 -0800 Subject: RFR (JAXP): 8147051: StaxEntityResolverWrapper should create StaxXMLInputSource with a resolver indicator Message-ID: <5697345E.7020408@oracle.com> Happen to notice that the indicator that the source was created by a resolver was not set in StaxEntityResolverWrapper. Also, it's "IsCreatedByResolver" rather than "hasResolver". JBS: https://bugs.openjdk.java.net/browse/JDK-8147051 webrev: http://cr.openjdk.java.net/~joehw/jdk9/8147051/webrev/ Thanks, Joe From felix.yang at oracle.com Thu Jan 14 06:07:22 2016 From: felix.yang at oracle.com (Felix Yang) Date: Thu, 14 Jan 2016 14:07:22 +0800 Subject: RFR 8065076/9, test/java/net/SocketPermission/SocketPermissionTest.java failed intermittently In-Reply-To: <568F74F7.70100@oracle.com> References: <568F74F7.70100@oracle.com> Message-ID: <56973B1A.30000@oracle.com> Hi all, please review the fix for test/java/net/SocketPermission/SocketPermissionTest.java, which fails frequently with "java.net.BindException: Address already in use". Bug: https://bugs.openjdk.java.net/browse/JDK-8065076 Webrev: http://cr.openjdk.java.net/~xiaofeya/8065076/webrev.00 The test uses getFreePort() test util library to verify permission on specific "host:port". This fix added retries to avoid possible binding failures: 1. A port may be still occupied for a short moment after socket close. 2. Another application is possible to occupy the same port during the short time window of open-close-reuse. This has been observed in experimental test runs. Thanks, Felix From cheleswer.sahu at oracle.com Thu Jan 14 06:53:13 2016 From: cheleswer.sahu at oracle.com (cheleswer sahu) Date: Thu, 14 Jan 2016 12:23:13 +0530 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: References: <56962632.7080404@oracle.com> Message-ID: <569745D9.1090007@oracle.com> Adding core-libs-dev and hotspot-runtime-dev team . On 1/14/2016 12:24 AM, Martin Buchholz wrote: > You should include core-libs-dev (and perhaps hotspot-runtime-dev) in > this thread. You're changing core library code. > > On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu > wrote: >> Hi, >> >> Please review the code changes for >> "https://bugs.openjdk.java.net/browse/JDK-8130425". >> >> Problem summary: When a large TLS (Thread Local Storage) size is set for >> threads, JVM is throwing StackOverflow exception. >> >> Problem Identified: >> As per investigation and a discussion we came to the conclusion that issue >> is not with the JVM but it lies in the way glibc has been implemented. When >> a TLS is declared , it steals the space from threads stack size. >> So if a thread is created with small stack size, and TLS is set to a large >> value, then it will result in StackOverflow. This is the exact case in this >> bug where reaper thread is allocated a very low stack size 32768. >> >> Discussion thread: >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037558.html >> >> Solution proposed: >> Its expected to get fix in glibc sometime , but for now I propose a >> workaround, a boolean system property "processReaperUseDefaultStackSize" >> using which we can set the stack size for reaper thread to default >> instead of fix 32768. This property can be set by the user using "-D" or >> "System.setProperty()". I have tested this fix, it works well with TLS size >> between 32k to 128k. >> >> Fix: >> diff -r 5c4530bb9ae6 >> src/java.base/share/classes/java/lang/ProcessHandleImpl.jav >> --- a/src/java.base/share/classes/java/lang/ProcessHandleImpl.java Fri Jan >> 08 13:06:29 2016 +0800 >> +++ b/src/java.base/share/classes/java/lang/ProcessHandleImpl.java Tue Jan >> 12 15:55:50 2016 +0530 >> @@ -83,9 +83,13 @@ >> ThreadGroup systemThreadGroup = tg; >> >> ThreadFactory threadFactory = grimReaper -> { >> - // Our thread stack requirement is quite modest. >> - Thread t = new Thread(systemThreadGroup, grimReaper, >> - "process reaper", 32768); >> + Thread t = null; >> + if >> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >> + t = new Thread(systemThreadGroup, grimReaper, >> "process reaper"); >> + } else { >> + // Our thread stack requirement is quite modest. >> + t = new Thread(systemThreadGroup, grimReaper, >> "process reaper", 32768); >> + } >> t.setDaemon(true); >> // A small attempt (probably futile) to avoid priority >> inversion >> t.setPriority(Thread.MAX_PRIORITY); >> >> >> Regards, >> Cheleswer From joe.darcy at oracle.com Thu Jan 14 06:56:21 2016 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 13 Jan 2016 22:56:21 -0800 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <56961259.7000503@oracle.com> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <5695A31D.1050405@oracle.com> <56961259.7000503@oracle.com> Message-ID: <56974695.7020800@oracle.com> Hi Daniel, On 1/13/2016 1:01 AM, Daniel Fuchs wrote: > Hi Joe, > > On 1/13/16 2:06 AM, Joseph D. Darcy wrote: >> Hi Iris, >> >> Catching up on old reviews, the regular expression for versions is >> given as >> >> ^[1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*$ >> >> Is this equivalent to >> >> ^[1-9][0-9]*(\.[0-9])* > > > Iris's regexp will disallow trailing .0 > > 9.0.1 will match, but 9.0 and 9.0.0 will not - the version should > be plain 9 in those latter cases as documented in the javadoc. > > I believe this is important because it ensures that the version string > is always in canonical form, which makes it possible to simply compare > the 'versions' list (List) within the various variations > of equals... > > So the regexp matching will eliminate non canonical forms without > having to perform further checks down the road. > Took me a while to figure it, I had to look at the test to > pinpoint it. Now that I understand, I find it quite elegant :-) Ah okay, I missed those points when reading over the regular expression. I suggest some discussion be added to explain and document that those aspects of the design are intentional. Thanks, -Joe From dmitry.samersoff at oracle.com Thu Jan 14 08:58:14 2016 From: dmitry.samersoff at oracle.com (Dmitry Samersoff) Date: Thu, 14 Jan 2016 11:58:14 +0300 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <50765a19-34e4-4115-a32d-cea84c25394e@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> Message-ID: <56976326.7040809@oracle.com> Iris, Did you consider to split version string into array of groups first (using String.split()), then validate each group separately? It may make the code better readable and more resilient to possible future changes. -Dmitry On 2015-11-25 04:54, Iris Clark wrote: > Hi. > > Please review the new classes jdk.Version and jdk.OracleVersion. These are > simple Java APIs to parse, validate, and compare version numbers. > > Bug > > 8072379: Implement jdk.Version and jdk.OracleVersion > https://bugs.openjdk.java.net/browse/JDK-8072379 > > Webrev > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.1/ > > JavaDoc > > http://cr.openjdk.java.net/~iris/verona/8072379/doc.1/jdk/Version.html > http://cr.openjdk.java.net/~iris/verona/8072379/doc.1/jdk/OracleVersion.html > > The .java files are predominantly javadoc. The code is relatively > straight-forward. > > jdk.Version is the representation of the JDK version string as described in > JEP 223 ([0], 8061493). The javadoc is largely taken from the description > section in the JEP. The API is described in the "API" section. > > jdk.OracleVersion extends jdk.Version and is the representation of the Oracle > JDK version string. Its only purpose is to interpret the fourth element of > the version number as a patch release. > > There are some minor discrepancies between the implementation and the JEP. > The JEP will need to be updated. The most notable is the name of the package > ("jdk" vs. the original "jdk.util"). The rename was recommended by Mark. > > Thanks, > iris > > [0] http://openjdk.java.net/jeps/223 > -- Dmitry Samersoff Oracle Java development team, Saint Petersburg, Russia * I would love to change the world, but they won't give me the sources. From victor2 at ukr.net Thu Jan 14 09:36:09 2016 From: victor2 at ukr.net (Victor Polischuk) Date: Thu, 14 Jan 2016 11:36:09 +0200 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: <1452763568.271873285.enbp1j4a@frv39.fwdcdn.com> Hi Iris, Do you consider an option to let community reuse JDK versioning style for their own purposes. Probably defining an interface with basic default methods which can be extended by various libraries to provide unified way to gather version information from MANIFEST.MF, ClassLoader's jars and simply to allow developers inventing one wheel lesser. ? /Victor --- Original message --- From: "Iris Clark" Date: 11 January 2016, 23:45:56 > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter > break and my vacation), I provided responses to your messages > and an updated webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume > that you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > > Regards, > iris From Alan.Bateman at oracle.com Thu Jan 14 09:54:40 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jan 2016 09:54:40 +0000 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> Message-ID: <56977060.1060001@oracle.com> On 13/01/2016 21:54, Iris Clark wrote: > : > This diff has been applied to modules.xml: This looks okay. > : > > When this came up earlier, I filed this bug to track finding > a more appropriate module for jdk.Version: > > 8144062: Determine appropriate module for jdk.Version > https://bugs.openjdk.java.net/browse/JDK-8144062 > Thanks, I've changed this to be a P2 as we will need to decide on this soon. -Alan. From chris.hegarty at oracle.com Thu Jan 14 10:05:16 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 14 Jan 2016 10:05:16 +0000 Subject: RFR [9] 7067728: Remove stopThread default java.policy Message-ID: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> The "stopThread? RuntimePermission is granted by default. The Thread.stop methods have been deprecated for more than 15 years. It seems reasonable, in a major release, to remove the default grant of stopThread. diff --git a/src/java.base/share/conf/security/java.policy b/src/java.base/share/conf/security/java.policy --- a/src/java.base/share/conf/security/java.policy +++ b/src/java.base/share/conf/security/java.policy @@ -94,17 +94,6 @@ // default permissions granted to all domains grant { - // Allows any thread to stop itself using the java.lang.Thread.stop() - // method that takes no argument. - // Note that this permission is granted by default only to remain - // backwards compatible. - // It is strongly recommended that you either remove this permission - // from this policy file or further restrict it to code sources - // that you specify, because Thread.stop() is potentially unsafe. - // See the API specification of java.lang.Thread.stop() for more - // information. - permission java.lang.RuntimePermission "stopThread"; - As a result of this change, untrusted code calling Thread.stop will throw a SecurityException, as it will not have the required permission. The solution, from the deployers perspective, is to add "stopThread? RuntimePermission to the policy file. -Chris. From Alan.Bateman at oracle.com Thu Jan 14 10:14:44 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jan 2016 10:14:44 +0000 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> Message-ID: <56977514.9080009@oracle.com> On 14/01/2016 10:05, Chris Hegarty wrote: > The "stopThread? RuntimePermission is granted by default. The Thread.stop > methods have been deprecated for more than 15 years. It seems reasonable, > in a major release, to remove the default grant of stopThread. This looks okay to me, we should have dropped granting this permission by default a long time ago. -Alan From david.holmes at oracle.com Thu Jan 14 10:37:34 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 14 Jan 2016 20:37:34 +1000 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> Message-ID: <56977A6E.6060202@oracle.com> Hi Chris, I would have expected some tests to need modifying here (or other places!). David On 14/01/2016 8:05 PM, Chris Hegarty wrote: > The "stopThread? RuntimePermission is granted by default. The Thread.stop > methods have been deprecated for more than 15 years. It seems reasonable, > in a major release, to remove the default grant of stopThread. > > diff --git a/src/java.base/share/conf/security/java.policy b/src/java.base/share/conf/security/java.policy > --- a/src/java.base/share/conf/security/java.policy > +++ b/src/java.base/share/conf/security/java.policy > @@ -94,17 +94,6 @@ > // default permissions granted to all domains > > grant { > - // Allows any thread to stop itself using the java.lang.Thread.stop() > - // method that takes no argument. > - // Note that this permission is granted by default only to remain > - // backwards compatible. > - // It is strongly recommended that you either remove this permission > - // from this policy file or further restrict it to code sources > - // that you specify, because Thread.stop() is potentially unsafe. > - // See the API specification of java.lang.Thread.stop() for more > - // information. > - permission java.lang.RuntimePermission "stopThread"; > - > > As a result of this change, untrusted code calling Thread.stop will throw a > SecurityException, as it will not have the required permission. The solution, > from the deployers perspective, is to add "stopThread? RuntimePermission > to the policy file. > > -Chris. > From chris.hegarty at oracle.com Thu Jan 14 10:40:15 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 14 Jan 2016 10:40:15 +0000 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <56977A6E.6060202@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> <56977A6E.6060202@oracle.com> Message-ID: On 14 Jan 2016, at 10:37, David Holmes wrote: > Hi Chris, > > I would have expected some tests to need modifying here (or other places!). I haven?t seen any test failures resulting from this change ( not sure if that is a good or a bad thing! ). Though, there were several implementation bugs that needed to be resolved before being able to remove default grant. -Chris. > David > > On 14/01/2016 8:05 PM, Chris Hegarty wrote: >> The "stopThread? RuntimePermission is granted by default. The Thread.stop >> methods have been deprecated for more than 15 years. It seems reasonable, >> in a major release, to remove the default grant of stopThread. >> >> diff --git a/src/java.base/share/conf/security/java.policy b/src/java.base/share/conf/security/java.policy >> --- a/src/java.base/share/conf/security/java.policy >> +++ b/src/java.base/share/conf/security/java.policy >> @@ -94,17 +94,6 @@ >> // default permissions granted to all domains >> >> grant { >> - // Allows any thread to stop itself using the java.lang.Thread.stop() >> - // method that takes no argument. >> - // Note that this permission is granted by default only to remain >> - // backwards compatible. >> - // It is strongly recommended that you either remove this permission >> - // from this policy file or further restrict it to code sources >> - // that you specify, because Thread.stop() is potentially unsafe. >> - // See the API specification of java.lang.Thread.stop() for more >> - // information. >> - permission java.lang.RuntimePermission "stopThread"; >> - >> >> As a result of this change, untrusted code calling Thread.stop will throw a >> SecurityException, as it will not have the required permission. The solution, >> from the deployers perspective, is to add "stopThread? RuntimePermission >> to the policy file. >> >> -Chris. >> From michael.haupt at oracle.com Thu Jan 14 11:22:32 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Thu, 14 Jan 2016 12:22:32 +0100 Subject: RFR(S): 8147078: MethodHandles.catchException does not enforce Throwable subtype Message-ID: <51F17B6F-FBD8-43E8-B440-6B34D881EA4E@oracle.com> Dear all, please review this fix. Bug: https://bugs.openjdk.java.net/browse/JDK-8147078 Webrev: http://cr.openjdk.java.net/~mhaupt/8147078/webrev.00 Unlike MethodHandles.throwException, MethodHandles.catchException does not enforce that the passed exception type is indeed a subtype of Throwable. The change adds the corresponding check, and a test. Thanks, Michael -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG, Schiffbauergasse 14 | 14467 Potsdam, Germany Oracle is committed to developing practices and products that help protect the environment From sundararajan.athijegannathan at oracle.com Thu Jan 14 11:35:26 2016 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Thu, 14 Jan 2016 17:05:26 +0530 Subject: RFR(S): 8147078: MethodHandles.catchException does not enforce Throwable subtype In-Reply-To: <51F17B6F-FBD8-43E8-B440-6B34D881EA4E@oracle.com> References: <51F17B6F-FBD8-43E8-B440-6B34D881EA4E@oracle.com> Message-ID: <569787FE.3040309@oracle.com> +1 -Sundar On 1/14/2016 4:52 PM, Michael Haupt wrote: > Dear all, > > please review this fix. > Bug: https://bugs.openjdk.java.net/browse/JDK-8147078 > Webrev: http://cr.openjdk.java.net/~mhaupt/8147078/webrev.00 > > Unlike MethodHandles.throwException, MethodHandles.catchException does not enforce that the passed exception type is indeed a subtype of Throwable. The change adds the corresponding check, and a test. > > Thanks, > > Michael > From magnus.ihse.bursie at oracle.com Thu Jan 14 11:38:17 2016 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 14 Jan 2016 12:38:17 +0100 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: <569788A9.3010509@oracle.com> On 2016-01-11 22:44, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter > break and my vacation), I provided responses to your messages > and an updated webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume > that you were satisfied. Is that correct? I'm basically happy. As other reviewers, I think the regexes are a bit tough to parse (regexes always are), and some additional comments on the effect would be appreciated. Thanh Hong Dai had some good suggestions as well on how to improve readability. I noted that your java source file had a lot of initial tabs instead of spaces. I'm not sure if we have any code guidelines about this, but I have not encountered that in the JDK source base before, so I'd recommend you turn them into spaces. /Magnus > > For you convenience, here's a reference to the December and November > threads: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > > Regards, > iris > > -----Original Message----- > From: Iris Clark > Sent: Friday, December 18, 2015 1:55 PM > To: Joe Darcy; Mandy Chung > Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net > Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion > > Hi, Joe. > > Thanks for the review comments. > >>>> http://cr.openjdk.java.net/~irisa/verona/8072379/webrev.1/ >> Is the intention that downstream JDK distributions, such as IcedTea, >> whether based on OpenJDK or otherwise, would provide their own >> specialization of the jdk.Version class? > No. Downstream users may provide their own specialization, but there is no requirement that they must do so. > > jdk.OracleVersion has been removed. The functionality it provided, access to the fourth element of the version number, is trivially accessed by existing methods in jdk.Version. > > I've filed the following bug to address the more general use case: > > 8145793: Provide vendor-specific interpretation of a JDK version string > https://bugs.openjdk.java.net/browse/JDK-8145793 > >> The equals / hashCode behavior of Version and OracleVersion is bit >> surprising and I think somewhat underspecified given the possibility >> of defining subclasses. > Given the above regarding downstream use, I've make jdk.Version final and the constructor is now private. Oracle or any other JDK distribution wishing to impose additional semantics to the version string interpretation will need to do so via encapsulation rather than inheritance. > >> How is this API supposed to behave if the component version strings >> have a numerical value greater than Integer.MAX_VALUE? > From the specification of the only instance method, Version.parse(): > > * @throws NumberFormatException > * If an element of the version number or the build number cannot > * be represented as an {@link Integer} > >> Was using Longs to record numerical versions rather than Integers >> considered? > Yes. In an informal poll conducted during implementation, it was thought that 2^32 bits would be more than sufficient to represent the components of typical version numbers. > > An updated webrev is forthcoming. > > Thanks, > iris > > PS: I believe that we are both out the week of 21 December, so I don't expect to hear back from you until the New Year. From ramanand.patil at oracle.com Thu Jan 14 12:21:51 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Thu, 14 Jan 2016 04:21:51 -0800 (PST) Subject: RFR: JDK-8144988: Unexpected timezone returned after parsing a date Message-ID: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> Hi all, Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8144988 Webrev: http://cr.openjdk.java.net/~rpatil/8141243/webrev.00 This is basically a backport of JDK9 bug: https://bugs.openjdk.java.net/browse/JDK-8141243 JDK9 changeset(for reference): http://hg.openjdk.java.net/jdk9/dev/jdk/rev/a1aa2671f281 Reason for the review request is because of: i) Changes present in ResourceBundleGenerator.java file. ii) The patch from JDK9 does not automatically apply as is after using unshuffle_patch script. Few paths are adjusted as per the jdk8. Since CLDR became the default locale data in JDK9 leading incompatible behavior with prior releases, the relevant code in ResourceBundleGenerator is also backported in this patch. Even though JDK8 has CLDR locale provider disabled by default, the code change is done to be on safer side for cases where CLDR may be supporting "UTC" ID in the future. Regards, Ramanand. From paul.sandoz at oracle.com Thu Jan 14 12:44:36 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 14 Jan 2016 13:44:36 +0100 Subject: RFR(S): 8147078: MethodHandles.catchException does not enforce Throwable subtype In-Reply-To: <51F17B6F-FBD8-43E8-B440-6B34D881EA4E@oracle.com> References: <51F17B6F-FBD8-43E8-B440-6B34D881EA4E@oracle.com> Message-ID: <993827AA-EF77-40C6-A3C6-A7917E2963D4@oracle.com> > On 14 Jan 2016, at 12:22, Michael Haupt wrote: > > Dear all, > > please review this fix. > Bug: https://bugs.openjdk.java.net/browse/JDK-8147078 > Webrev: http://cr.openjdk.java.net/~mhaupt/8147078/webrev.00 > > Unlike MethodHandles.throwException, MethodHandles.catchException does not enforce that the passed exception type is indeed a subtype of Throwable. The change adds the corresponding check, and a test. > Erasure bites! +1 Paul. From sean.mullan at oracle.com Thu Jan 14 13:49:19 2016 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 14 Jan 2016 08:49:19 -0500 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> Message-ID: <5697A75F.60109@oracle.com> Looks good to me. --Sean On 01/14/2016 05:05 AM, Chris Hegarty wrote: > The "stopThread? RuntimePermission is granted by default. The Thread.stop > methods have been deprecated for more than 15 years. It seems reasonable, > in a major release, to remove the default grant of stopThread. > > diff --git a/src/java.base/share/conf/security/java.policy b/src/java.base/share/conf/security/java.policy > --- a/src/java.base/share/conf/security/java.policy > +++ b/src/java.base/share/conf/security/java.policy > @@ -94,17 +94,6 @@ > // default permissions granted to all domains > > grant { > - // Allows any thread to stop itself using the java.lang.Thread.stop() > - // method that takes no argument. > - // Note that this permission is granted by default only to remain > - // backwards compatible. > - // It is strongly recommended that you either remove this permission > - // from this policy file or further restrict it to code sources > - // that you specify, because Thread.stop() is potentially unsafe. > - // See the API specification of java.lang.Thread.stop() for more > - // information. > - permission java.lang.RuntimePermission "stopThread"; > - > > As a result of this change, untrusted code calling Thread.stop will throw a > SecurityException, as it will not have the required permission. The solution, > from the deployers perspective, is to add "stopThread? RuntimePermission > to the policy file. > > -Chris. > From magnus.ihse.bursie at oracle.com Thu Jan 14 15:00:31 2016 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 14 Jan 2016 16:00:31 +0100 Subject: RFR: JDK-8145549 Add support for Visual Studio 2015 Community edition In-Reply-To: <57E1C404-21A6-41F7-87EF-881EDD3830C1@oracle.com> References: <7B657FB8-B581-4AB2-81C1-877B4B1BD658@oracle.com> <57E1C404-21A6-41F7-87EF-881EDD3830C1@oracle.com> Message-ID: <5697B80F.4050905@oracle.com> On 2015-12-18 15:11, Wang Weijun wrote: > Hi Vinnie > > I take a look and it includes a change for src/jdk.crypto.mscapi/windows/native/libsunmscapi/security.cpp in the Java_sun_security_mscapi_KeyStore_getKeyLength() function. > > It seems there is no sun.security.mscapi.KeyStore#getKeyLength on the java side. Is the function useless now? Max, Is your intention here that you think the patch should remove the entire Java_sun_security_mscapi_KeyStore_getKeyLength function? /Magnus > > Thanks > Max > >> On Dec 16, 2015, at 9:50 PM, Magnus Ihse Bursie wrote: >> >> There is an interest from the community to build OpenJDK using Visual Studio 2015 Community edition. >> >> This patch is provided by Timo Kinnunen . I am sponsoring this patch. >> >> The changes to the source code files are mostly casts to uintptr_t, but there are some other changes as well. >> >> I'm not quite sure who's the owner of all these files. If I'm missing some group, please help me and forward the mail to them. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8145549 >> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8145549-vs2015-community-edition/webrev.01 >> >> /Magnus From magnus.ihse.bursie at oracle.com Thu Jan 14 15:05:11 2016 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Thu, 14 Jan 2016 16:05:11 +0100 Subject: RFR: JDK-8145549 Add support for Visual Studio 2015 Community edition In-Reply-To: <98B07F13-1D90-4DEC-AAED-D791F4666BF7@oracle.com> References: <7B657FB8-B581-4AB2-81C1-877B4B1BD658@oracle.com> <98B07F13-1D90-4DEC-AAED-D791F4666BF7@oracle.com> Message-ID: <5697B927.6090008@oracle.com> On 2016-01-05 03:25, Kim Barrett wrote: > On Dec 18, 2015, at 7:41 PM, Kim Barrett wrote: >> On Dec 16, 2015, at 8:50 AM, Magnus Ihse Bursie wrote: >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8145549 >>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8145549-vs2015-community-edition/webrev.01 >>> >>> /Magnus >> I only looked at hotspot files. >> >> [?] >> There are a couple of "TBD"s below that I need to spend more time on. >> > Back from holiday, and here?s my comments on those two TBDs Kim, Thank you for your feedback. Since I'm only the sponsor of this patch, not the developer, I'll let Timo answer your questions and discuss his choices. /Magnus > > ------------------------------------------------------------------------------ > hotspot/agent/src/share/native/sadis.c > 96 return (int)strlen(buf); > > The only call to the (file-scoped) getLastErrorString function is > Java_sun_jvm_hotspot_asm_Disassembler_load_1library, which ignores the > result. It would be better to change the return type of > getLastErrorString to void and update the return statements > accordingly. > > ------------------------------------------------------------------------------ > hotspot/src/share/vm/adlc/arena.hpp > 74 // Usual (non-placement) deallocation function to allow placement delete use size_t > 75 // See 3.7.4.2 [basic.stc.dynamic.deallocation] paragraph 2. > 76 void operator delete(void* p); > > and associated code in the .cpp file. > > I think this is another C++11 change: > http://open-std.org/JTC1/SC22/WG21/docs/cwg_defects.html#429 > > I think the proposed code change is OK, although the comment is > problematic: [basic.stc.dynamic.deallocation] is C++03 3.7.3.2 and > C++11 3.7.4.2. Also, the standard change that leads to this code > change is in C++11 5.3.4 [expr.new] paragraph 20 (C++02 p 19). > > Also, I *think* with the addition of the one-arg operator delete we > don't actually use the two-arg form. If so, then I suggest removing > it and the proposed new comment for the one-arg form. > > ------------------------------------------------------------------------------ > From iris.clark at oracle.com Thu Jan 14 15:48:49 2016 From: iris.clark at oracle.com (Iris Clark) Date: Thu, 14 Jan 2016 07:48:49 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <569788A9.3010509@oracle.com> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <569788A9.3010509@oracle.com> Message-ID: Hi, Magnus. Thanks for taking the time to look at this again. > I noted that your java source file had a lot of initial tabs instead of spaces. > I'm not sure if we have any code guidelines about this, jcheck prevents tabs in comments and source. I remove them, one of my editors adds them when I make changes. It's annoying. They won't be in the checked in source. Regards, iris -----Original Message----- From: Magnus Ihse Bursie Sent: Thursday, January 14, 2016 3:38 AM To: Iris Clark; Joe Darcy; Mandy Chung; Roger Riggs; Alan Bateman Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: Re: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion On 2016-01-11 22:44, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter break and > my vacation), I provided responses to your messages and an updated > webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume that > you were satisfied. Is that correct? I'm basically happy. As other reviewers, I think the regexes are a bit tough to parse (regexes always are), and some additional comments on the effect would be appreciated. Thanh Hong Dai had some good suggestions as well on how to improve readability. I noted that your java source file had a lot of initial tabs instead of spaces. I'm not sure if we have any code guidelines about this, but I have not encountered that in the JDK source base before, so I'd recommend you turn them into spaces. /Magnus > > For you convenience, here's a reference to the December and November > threads: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036 > 904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > > Regards, > iris > > -----Original Message----- > From: Iris Clark > Sent: Friday, December 18, 2015 1:55 PM > To: Joe Darcy; Mandy Chung > Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net > Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion > > Hi, Joe. > > Thanks for the review comments. > >>>> http://cr.openjdk.java.net/~irisa/verona/8072379/webrev.1/ >> Is the intention that downstream JDK distributions, such as IcedTea, >> whether based on OpenJDK or otherwise, would provide their own >> specialization of the jdk.Version class? > No. Downstream users may provide their own specialization, but there is no requirement that they must do so. > > jdk.OracleVersion has been removed. The functionality it provided, access to the fourth element of the version number, is trivially accessed by existing methods in jdk.Version. > > I've filed the following bug to address the more general use case: > > 8145793: Provide vendor-specific interpretation of a JDK version string > https://bugs.openjdk.java.net/browse/JDK-8145793 > >> The equals / hashCode behavior of Version and OracleVersion is bit >> surprising and I think somewhat underspecified given the possibility >> of defining subclasses. > Given the above regarding downstream use, I've make jdk.Version final and the constructor is now private. Oracle or any other JDK distribution wishing to impose additional semantics to the version string interpretation will need to do so via encapsulation rather than inheritance. > >> How is this API supposed to behave if the component version strings >> have a numerical value greater than Integer.MAX_VALUE? > From the specification of the only instance method, Version.parse(): > > * @throws NumberFormatException > * If an element of the version number or the build number cannot > * be represented as an {@link Integer} > >> Was using Longs to record numerical versions rather than Integers >> considered? > Yes. In an informal poll conducted during implementation, it was thought that 2^32 bits would be more than sufficient to represent the components of typical version numbers. > > An updated webrev is forthcoming. > > Thanks, > iris > > PS: I believe that we are both out the week of 21 December, so I don't expect to hear back from you until the New Year. From mandy.chung at oracle.com Thu Jan 14 15:59:07 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 14 Jan 2016 07:59:07 -0800 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: <03FABAC5-9DF0-4A33-ADCA-C0F8EDFC3B85@oracle.com> > On Jan 13, 2016, at 9:27 PM, Iris Clark wrote: > > These are the diffs to address both of your comments: > > $ diff Version.java_save Version.java > 28a29,30 >> import java.security.AccessController; >> import java.security.PrivilegedAction; > 154,155d155 > < * @see JEP 223: New Version-String Scheme > < * > 268a269,274 >> * @throws SecurityException >> * If a security manager exists and its {@link >> * SecurityManager#checkPropertyAccess(String) >> * checkPropertyAccess} method does not allow access to the >> * system property "java.version" >> * > 272,273c278,285 > < if (current == null) > < current = parse(System.getProperty("java.version")); > --- >> if (current == null) { >> current = parse(AccessController.doPrivileged( >> new PrivilegedAction<>() { >> public String run() { >> return System.getProperty("java.version"); >> } >> })); >> } Looks fine. Thanks Mandy From mandy.chung at oracle.com Thu Jan 14 16:52:54 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 14 Jan 2016 08:52:54 -0800 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> Message-ID: <0616B189-6042-44D4-A8B1-8D19A0860D97@oracle.com> > On Jan 14, 2016, at 2:05 AM, Chris Hegarty wrote: > > The "stopThread? RuntimePermission is granted by default. The Thread.stop > methods have been deprecated for more than 15 years. It seems reasonable, > in a major release, to remove the default grant of stopThread. +1 to remove "stopThread? RuntimePermission from java.policy. There are existing tests whose grants this "stopThread? RuntimePermission that may not be needed for the test. The test policy likely copies that from the default system java.policy. We should update these test policy as well. >> >> I would have expected some tests to need modifying here (or other places!). > > I haven?t seen any test failures resulting from this change ( not sure > if that is a good or a bad thing! ). Though, there were several implementation > bugs that needed to be resolved before being able to remove default grant. jtreg policy tag overrides the system default security policy with the specified file. Tests that call Thread::stop and tested with security manager must have "stopThread? RuntimePermission set in the test policy. jtreg was enhanced to add a new java.security.policy tag to extend the system security policy [1]. Only tests using java.security.policy tag and calling Thread::stop will need to be modified. Mandy [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7900898 From chris.hegarty at oracle.com Thu Jan 14 17:19:59 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 14 Jan 2016 17:19:59 +0000 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <0616B189-6042-44D4-A8B1-8D19A0860D97@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> <0616B189-6042-44D4-A8B1-8D19A0860D97@oracle.com> Message-ID: <05B81539-8555-452C-AD7E-41746C8CE5D0@oracle.com> On 14 Jan 2016, at 16:52, Mandy Chung wrote: >> On Jan 14, 2016, at 2:05 AM, Chris Hegarty wrote: >> >> The "stopThread? RuntimePermission is granted by default. The Thread.stop >> methods have been deprecated for more than 15 years. It seems reasonable, >> in a major release, to remove the default grant of stopThread. > > +1 to remove "stopThread? RuntimePermission from java.policy. Thanks for the review Mandy. > There are existing tests whose grants this "stopThread? RuntimePermission that may not be needed for the test. The test policy likely copies that from the default system java.policy. We should update these test policy as well. I do see a few of these, and some will need discussion. Ok if I file a separate bug on these, they are not directly related to this change, and do still pass, just that the permission is superfluous. >>> I would have expected some tests to need modifying here (or other places!). >> >> I haven?t seen any test failures resulting from this change ( not sure >> if that is a good or a bad thing! ). Though, there were several implementation >> bugs that needed to be resolved before being able to remove default grant. > > jtreg policy tag overrides the system default security policy with the specified file. Tests that call Thread::stop and tested with security manager must have "stopThread? RuntimePermission set in the test policy. jtreg was enhanced to add a new java.security.policy tag to extend the system security policy [1]. Thanks for this explanation. I always get confused with how jtreg supports this. > Only tests using java.security.policy tag and calling Thread::stop will need to be modified. I can find no such tests. -Chris. > Mandy > [1] https://bugs.openjdk.java.net/browse/CODETOOLS-7900898 From mandy.chung at oracle.com Thu Jan 14 17:29:54 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 14 Jan 2016 09:29:54 -0800 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <05B81539-8555-452C-AD7E-41746C8CE5D0@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> <0616B189-6042-44D4-A8B1-8D19A0860D97@oracle.com> <05B81539-8555-452C-AD7E-41746C8CE5D0@oracle.com> Message-ID: <71226EC2-EF9B-41F7-996E-B14D07B91C3F@oracle.com> > On Jan 14, 2016, at 9:19 AM, Chris Hegarty wrote: > >> There are existing tests whose grants this "stopThread? RuntimePermission that may not be needed for the test. The test policy likely copies that from the default system java.policy. We should update these test policy as well. > > I do see a few of these, and some will need discussion. Ok if I file a separate > bug on these, they are not directly related to this change, and do still pass, just > that the permission is superfluous. > Taking it out from the test policy file should be non-controversial and trivial to verify. I can see why you prefer to separate the test update from this change and I?m okay. >>>> I would have expected some tests to need modifying here (or other places!). >>> >>> I haven?t seen any test failures resulting from this change ( not sure >>> if that is a good or a bad thing! ). Though, there were several implementation >>> bugs that needed to be resolved before being able to remove default grant. >> >> jtreg policy tag overrides the system default security policy with the specified file. Tests that call Thread::stop and tested with security manager must have "stopThread? RuntimePermission set in the test policy. jtreg was enhanced to add a new java.security.policy tag to extend the system security policy [1]. > > Thanks for this explanation. I always get confused with how jtreg supports > this. > >> Only tests using java.security.policy tag and calling Thread::stop will need to be modified. > > I can find no such tests. That matches what I expect since most of the tests using the new java.security.policy tag are related to deprivileging work and new tests only. Mandy From chris.hegarty at oracle.com Thu Jan 14 17:32:09 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 14 Jan 2016 17:32:09 +0000 Subject: RFR [9] 7067728: Remove stopThread default java.policy In-Reply-To: <71226EC2-EF9B-41F7-996E-B14D07B91C3F@oracle.com> References: <80854855-CF11-4C11-8AD2-71C34436E309@oracle.com> <0616B189-6042-44D4-A8B1-8D19A0860D97@oracle.com> <05B81539-8555-452C-AD7E-41746C8CE5D0@oracle.com> <71226EC2-EF9B-41F7-996E-B14D07B91C3F@oracle.com> Message-ID: <098CD8EF-9BF0-4253-AEC0-164045E5643A@oracle.com> On 14 Jan 2016, at 17:29, Mandy Chung wrote: > >> On Jan 14, 2016, at 9:19 AM, Chris Hegarty wrote: >> >>> There are existing tests whose grants this "stopThread? RuntimePermission that may not be needed for the test. The test policy likely copies that from the default system java.policy. We should update these test policy as well. >> >> I do see a few of these, and some will need discussion. Ok if I file a separate >> bug on these, they are not directly related to this change, and do still pass, just >> that the permission is superfluous. >> > > Taking it out from the test policy file should be non-controversial and trivial to verify. Right. I was thinking that maybe these tests should be updated to use the new jtreg machanism, java.security.policy, rather than just removing stopThread? > I can see why you prefer to separate the test update from this change and I?m okay. Thanks. I?ll file a bug on it. > >>>>> I would have expected some tests to need modifying here (or other places!). >>>> >>>> I haven?t seen any test failures resulting from this change ( not sure >>>> if that is a good or a bad thing! ). Though, there were several implementation >>>> bugs that needed to be resolved before being able to remove default grant. >>> >>> jtreg policy tag overrides the system default security policy with the specified file. Tests that call Thread::stop and tested with security manager must have "stopThread? RuntimePermission set in the test policy. jtreg was enhanced to add a new java.security.policy tag to extend the system security policy [1]. >> >> Thanks for this explanation. I always get confused with how jtreg supports >> this. >> >>> Only tests using java.security.policy tag and calling Thread::stop will need to be modified. >> >> I can find no such tests. > > That matches what I expect since most of the tests using the new java.security.policy tag are related to deprivileging work and new tests only. Great. -Chris. From naoto.sato at oracle.com Thu Jan 14 18:13:34 2016 From: naoto.sato at oracle.com (Naoto Sato) Date: Thu, 14 Jan 2016 10:13:34 -0800 Subject: RFR: JDK-8144988: Unexpected timezone returned after parsing a date In-Reply-To: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> Message-ID: <5697E54E.7030605@oracle.com> Looks good to me. Naoto On 1/14/16 4:21 AM, Ramanand Patil wrote: > Hi all, > > Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8144988 > > Webrev: http://cr.openjdk.java.net/~rpatil/8141243/webrev.00 > > This is basically a backport of JDK9 bug: https://bugs.openjdk.java.net/browse/JDK-8141243 > > JDK9 changeset(for reference): http://hg.openjdk.java.net/jdk9/dev/jdk/rev/a1aa2671f281 > > Reason for the review request is because of: > i) Changes present in ResourceBundleGenerator.java file. > ii) The patch from JDK9 does not automatically apply as is after using unshuffle_patch script. Few paths are adjusted as per the jdk8. > > Since CLDR became the default locale data in JDK9 leading incompatible behavior with prior releases, the relevant code in ResourceBundleGenerator is also backported in this patch. > Even though JDK8 has CLDR locale provider disabled by default, the code change is done to be on safer side for cases where CLDR may be supporting "UTC" ID in the future. > > > Regards, > Ramanand. > From martinrb at google.com Thu Jan 14 18:59:26 2016 From: martinrb at google.com (Martin Buchholz) Date: Thu, 14 Jan 2016 10:59:26 -0800 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569745D9.1090007@oracle.com> References: <56962632.7080404@oracle.com> <569745D9.1090007@oracle.com> Message-ID: The process grim reaper ends up being the first point of failure since it tries not to waste the user's memory and it's in a core library, but in principle it's not special. I think a more general workaround would be to add a hotspot flag that would add a memory safety zone to all threads. If it's known that TLS is stealing 32k from every thread's stack, then the flag should ensure that every thread stack is 32k larger. More generally, I was hoping that hotspot would ensure that the -Xss size was available for actual java stack frames and OS overhead was added on automatically, but that is hard to implement, so the best alternative workaround is for users to be able to specify that additional stack size padding. Maybe -XX:StackSizeOverhead=32768 ? From marcins at microsoft.com Thu Jan 14 19:33:47 2016 From: marcins at microsoft.com (Martin Sawicki) Date: Thu, 14 Jan 2016 19:33:47 +0000 Subject: RFR 8124977 cmdline encoding challenges on Windows In-Reply-To: <56957F88.6060708@oracle.com> References: <558C017A.2020708@oracle.com> <5651E9AA.9060908@oracle.com> <56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com> Message-ID: Thanks for the feedback. Investigating the regression failure. We'll get back as soon as we figure this out. (and yes, we'll run this through some localized Windows VMs) Cheers -----Original Message----- From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] Sent: Tuesday, January 12, 2016 2:35 PM To: Martin Sawicki ; Vladimir Shcherbakov Cc: core-libs-dev Libs ; Naoto Sato Subject: Re: RFR 8124977 cmdline encoding challenges on Windows Hi Martin, Vladimir, It was suggested that this patch be tested on localized Windows machines and/or trying with the various Windows native encodings, appreciate if you can verify this as well. Thanks Kumar On 1/11/2016 1:10 PM, Kumar Srinivasan wrote: > Hi, > > Was on vacation, I started to prepare the patch from webrev.04 > for integration. Please note: made some adjustments to your > patch to pass jcheck, ie. usage of tabs and space at line endings, > and modifications to Copyright dates. > > Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE. > I have attached a patch to for your reference. > > However, there is a regression test failure on Windows, > jdk/test/tools/launcher/I18NTest.java > > ---Test info---- > Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java > > ++++Test Output++++ > javac: file not found: i18nH?lloWorld.java > ----End test info----- > > Have you run all the launcher regression tests with this changeset ? > > Thanks > Kumar > >> Hi Kumar, just wondering if there are any updates on processing this >> submission. >> Thanks! >> >> -----Original Message----- >> From: Vladimir Shcherbakov >> Sent: Wednesday, November 25, 2015 2:38 PM >> To: Kumar Srinivasan ; Martin Sawicki >> >> Cc: Kirk Shoop ; core-libs-dev Libs >> >> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows >> >> Hi Kumar, >> >> Please find updated webreview here: >> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.openjdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarcins%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNfwnbIw%3d >> >> Thanks, >> Vladimir. >> >> -----Original Message----- >> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] >> Sent: Sunday, November 22, 2015 8:14 AM >> To: Martin Sawicki >> Cc: Kirk Shoop ; Vladimir Shcherbakov >> ; core-libs-dev Libs >> >> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >> >> >> Hi Martin, et. al., >> >> Sorry for not getting back earlier, I am very busy right now with my >> other large commitments for JDK9. >> >> I will sponsor this "enhancement/bug fix" sometime in the new year, >> meanwhile, there is the changeset [1] which is likely to cause merge >> conflicts, and perhaps logic issues. >> >> Thanks >> Kumar >> >> [1] >> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.openjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7Mg4lt2P0sdzLS%2fT9Q%3d >>> Hi all >>> Here's an updated webrev attempting to take into account the various >>> pieces of feedback we have received: >>> >>> Issue: >>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. >>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40micro >>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d7c >>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E%3 >>> d >>> Webrev: >>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.openj >>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashch% >>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af9 >>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJnWY >>> 0%3d >>> >>> (Vladimir Shcherbakov is now working on this from our side) >>> >>> Looking forward to any other feedback. >>> Thanks >>> >>> -----Original Message----- >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >>> Behalf Of Kumar Srinivasan >>> Sent: Thursday, June 25, 2015 6:26 AM >>> To: Kirk Shoop (MS OPEN TECH) >>> Cc: Valery Kopylov (Akvelon) ; core-libs-dev >>> Libs >>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >>> >>> Hi Kirk, >>> >>> Thanks for proposing this change. >>> >>> If you notice all the posix calls are wrapped in JLI_* this gives us >>> the ability to use "W" functions. I almost got it done, several >>> years ago, but we upgraded to VS2010 and my work based on VS2003 >>> keeled over, meanwhile my focus was "shifted" to something else. >>> >>> main.c: is really envisioned to be a stub compiled by the tool >>> launchers, like java, javac, javah, jar etc. I prefer to see all the >>> heavy logic in this file moved to the platform specific file >>> windows/java_md.* >>> >>> For the reason specified above we need to move fprintf or any naked >>> posix calls to JLI_* indirections. >>> >>> I don't see any tests ? The tests must be written in java and placed >>> in jdk/test/tools/launcher, there is a helper framework >>> TestHelper.java. >>> >>> There are other changes in nio, charsets etc, this will be reviewed >>> by my colleague specializing in that area (Sherman) cc'ed. >>> >>> >>> Thanks >>> >>> Kumar >>> >>> >>> >>> >>> >>> >>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote: >>>> Hi, >>>> >>>> Issue: >>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs >>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic >>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2 >>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq >>>> 9E%3d >>>> >>>> Webrev: >>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open >>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40microsof >>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7cd0 >>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M%3d >>>> >>>> This webrev intends to address interaction between Windows console >>>> and java apps. >>>> >>>> Two switches were added that change the behavior of the launcher. >>>> The defaults do not change the launcher behavior. >>>> >>>> -Dwindows.UnicodeConsole=true - switches on Unicode support in >>>> the Windows console. This optional switch causes the launcher to >>>> call GetCommandLineW() and parse the arguments in unicode. It also >>>> modifies how the codepage for console output is selected. >>>> >>>> -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset >>>> to use; If not specified, UTF-8 is used by default. Ignored when >>>> windows.UnicodeConsole is not set to true. When the first switch is >>>> used, this optional switch allows the codepage for console output >>>> to be controlled. >>>> >>>> I would like to get feedback on the approach here and any >>>> additional work that is required solve these particular Unicode >>>> issues on Windows. >>>> >>>> Kirk > From chris.hegarty at oracle.com Thu Jan 14 19:53:48 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 14 Jan 2016 19:53:48 +0000 Subject: RFR [9]: Remove sun.misc.ClassFileTransformer. Message-ID: sun.misc.ClassFileTransformer has not been used since its hooks were removed in JDK 8, see 8009645 [1]. It is now time to remove the class itself. hg rm src/java.base/share/classes/sun/misc/ClassFileTransformer.java -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8009645 From mandy.chung at oracle.com Thu Jan 14 20:00:10 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 14 Jan 2016 12:00:10 -0800 Subject: RFR [9]: Remove sun.misc.ClassFileTransformer. In-Reply-To: References: Message-ID: <78A4DA87-71BC-4B91-9982-238E35A6461C@oracle.com> > On Jan 14, 2016, at 11:53 AM, Chris Hegarty wrote: > > sun.misc.ClassFileTransformer has not been used since its hooks were > removed in JDK 8, see 8009645 [1]. It is now time to remove the class > itself. > > hg rm src/java.base/share/classes/sun/misc/ClassFileTransformer.java +1 Mandy From Alan.Bateman at oracle.com Thu Jan 14 20:06:19 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 14 Jan 2016 20:06:19 +0000 Subject: RFR [9]: Remove sun.misc.ClassFileTransformer. In-Reply-To: References: Message-ID: <5697FFBB.5040803@oracle.com> On 14/01/2016 19:53, Chris Hegarty wrote: > sun.misc.ClassFileTransformer has not been used since its hooks were > removed in JDK 8, see 8009645 [1]. It is now time to remove the class > itself. > > hg rm src/java.base/share/classes/sun/misc/ClassFileTransformer.java > > Got for it. If you look at the history then you'll see it was completely nobbled in JDK 8 with a view to removing. -Alan From weijun.wang at oracle.com Fri Jan 15 00:39:30 2016 From: weijun.wang at oracle.com (Wang Weijun) Date: Fri, 15 Jan 2016 08:39:30 +0800 Subject: RFR: JDK-8145549 Add support for Visual Studio 2015 Community edition In-Reply-To: <5697B80F.4050905@oracle.com> References: <7B657FB8-B581-4AB2-81C1-877B4B1BD658@oracle.com> <57E1C404-21A6-41F7-87EF-881EDD3830C1@oracle.com> <5697B80F.4050905@oracle.com> Message-ID: <1411F578-C3A7-473D-9407-F10E018DF75E@oracle.com> > On Jan 14, 2016, at 11:00 PM, Magnus Ihse Bursie wrote: > > On 2015-12-18 15:11, Wang Weijun wrote: >> Hi Vinnie >> >> I take a look and it includes a change for src/jdk.crypto.mscapi/windows/native/libsunmscapi/security.cpp in the Java_sun_security_mscapi_KeyStore_getKeyLength() function. >> >> It seems there is no sun.security.mscapi.KeyStore#getKeyLength on the java side. Is the function useless now? > > Max, > > Is your intention here that you think the patch should remove the entire Java_sun_security_mscapi_KeyStore_getKeyLength function? Yes, I hope so. --Max > > /Magnus > >> >> Thanks >> Max >> >>> On Dec 16, 2015, at 9:50 PM, Magnus Ihse Bursie wrote: >>> >>> There is an interest from the community to build OpenJDK using Visual Studio 2015 Community edition. >>> >>> This patch is provided by Timo Kinnunen . I am sponsoring this patch. >>> >>> The changes to the source code files are mostly casts to uintptr_t, but there are some other changes as well. >>> >>> I'm not quite sure who's the owner of all these files. If I'm missing some group, please help me and forward the mail to them. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8145549 >>> WebRev: http://cr.openjdk.java.net/~ihse/JDK-8145549-vs2015-community-edition/webrev.01 >>> >>> /Magnus > From david.holmes at oracle.com Fri Jan 15 02:32:22 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 15 Jan 2016 12:32:22 +1000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: References: <56962632.7080404@oracle.com> <569745D9.1090007@oracle.com> Message-ID: <56985A36.7070905@oracle.com> On 15/01/2016 4:59 AM, Martin Buchholz wrote: > The process grim reaper ends up being the first point of failure since > it tries not to waste the user's memory and it's in a core library, > but in principle it's not special. I think a more general workaround > would be to add a hotspot flag that would add a memory safety zone to > all threads. If it's known that TLS is stealing 32k from every > thread's stack, then the flag should ensure that every thread stack is > 32k larger. I think we need a point fix for the process reaper case in the immediate term. We can then consider how to better address the general case in the medium to long term. > More generally, I was hoping that hotspot would ensure that the -Xss > size was available for actual java stack frames and OS overhead was > added on automatically, but that is hard to implement, so the best > alternative workaround is for users to be able to specify that > additional stack size padding. Maybe -XX:StackSizeOverhead=32768 ? Even this is still a band-aid - the user has to experience the problem, recognize it, and then figure out the right adjustment to add. Plus if any third-party library uses native TLS the requirement could change dynamically. So I'd prefer a new RFE to look at this general issue. Thanks, David From masayoshi.okutsu at oracle.com Fri Jan 15 02:47:41 2016 From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu) Date: Fri, 15 Jan 2016 11:47:41 +0900 Subject: RFR: JDK-8144988: Unexpected timezone returned after parsing a date In-Reply-To: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> Message-ID: <56985DCD.4070706@oracle.com> Hi Ramanand, test/java/text/Format/DateFormat/Bug8141243.java: 28 * @run main Bug8141243 29 * @run main/othervm -Djava.locale.providers=COMPAT Bug8141243 "COMPAT" is a new name of "JRE" in JDK 9. It's not supported in 8u. I think COMPAT is slightly ignored and that it becomes the same thing as line 28. Line 29 should be removed. Otherwise, the fix looks OK to me. Masayoshi On 1/14/2016 9:21 PM, Ramanand Patil wrote: > Hi all, > > Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8144988 > > Webrev: http://cr.openjdk.java.net/~rpatil/8141243/webrev.00 > > This is basically a backport of JDK9 bug: https://bugs.openjdk.java.net/browse/JDK-8141243 > > JDK9 changeset(for reference): http://hg.openjdk.java.net/jdk9/dev/jdk/rev/a1aa2671f281 > > Reason for the review request is because of: > i) Changes present in ResourceBundleGenerator.java file. > ii) The patch from JDK9 does not automatically apply as is after using unshuffle_patch script. Few paths are adjusted as per the jdk8. > > Since CLDR became the default locale data in JDK9 leading incompatible behavior with prior releases, the relevant code in ResourceBundleGenerator is also backported in this patch. > Even though JDK8 has CLDR locale provider disabled by default, the code change is done to be on safer side for cases where CLDR may be supporting "UTC" ID in the future. > > > Regards, > Ramanand. From amaembo at gmail.com Fri Jan 15 04:35:43 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 15 Jan 2016 10:35:43 +0600 Subject: Clarification of BaseStream.onClose() behavior Message-ID: <1666489469.20160115103543@gmail.com> Hello! Current documentation for BaseStream.onClose() does not specify explicitly how the stream would behave if additional close handlers are registered after the stream is consumed or even closed. The implementation actually allows registering the additional close handlers after consumption or closure and close() method is idempotent: newly registered handlers are perfectly called: Stream s = Stream.of("content"); s = s.onClose(() -> System.out.println("A")); s.forEach(System.out::println); s = s.onClose(() -> System.out.println("B")); s.close(); // prints A and B s = s.onClose(() -> System.out.println("C")); s.close(); // prints C (removing "s =" produces the same result). I think if such behavior is intended, it should be explicitly noted in the specification, as third-party implementations of BaseStream interface should provide consistent behavior. Or at least it should be noted that some BaseStream implementations may have idempotent close() method, so the derived AutoCloseable objects (which own the BaseStream object) should be aware about this behavior and provide idempotent close() method as well. Without knowing this one may write: class MyObject implements AutoCloseable { private BaseStream s; public MyObject(BaseStream s) { this.s = s; } @Override public void close() throws Exception { if(s != null) { s.close(); s = null; } } } Such code closes the stream only once, so newly registered handlers will not be called if MyObject::close is called the second time. However, to my opinion the current behavior is weird and should be changed in order not to allow registering new close handles (throwing IllegalStateException) when the stream is already closed, or even better when the stream is linked/consumed. As far as I know currently in JDK close handlers are registered only for non-consumed stream, so such change would not break existing JDK code (though may break some strange third party code). It should be noted that AutoCloseable interface discourages idempotent close() methods (though not forbids them). What do you think? With best regards, Tagir Valeev. From scolebourne at joda.org Fri Jan 15 13:05:51 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 15 Jan 2016 13:05:51 +0000 Subject: Map.Entry methods for streams Message-ID: I've had a morning of discussion about streaming Map this morning. While there is clearly no appetite for a JDK MapStream right now, it does seem that two additional methods on Map.Entry could help. Two of the common cases when streaming over Map.Entry are to transform the keys and to transform the values. Currently, the code is as follows: hashMap.entrySet().stream() .map(e -> new AbstractMap.SimpleImmutableEntry(e.getKey(), e.getValue().transformed())) ... This isn't really very pleasant. Adding a default method to Map.Entry would help: hashMap.entrySet().stream() .map(e -> e.mapValue(v -> v.transformed())) ... Implementation of the mapValue method is left to the reader, but it isn't hard. A mapKey() method would also be needed. Note that both are designed to work without mutating the map (returning a new entry). Finally, the Collectors class could do with a new method entriesToMap() that collects a stream of Map.Entry back into a Map. While these proposals are not as powerful as a MapStream, they would smooth some rough edges in the API when working with maps. Stephen From forax at univ-mlv.fr Fri Jan 15 13:37:52 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 15 Jan 2016 14:37:52 +0100 (CET) Subject: Map.Entry methods for streams In-Reply-To: References: Message-ID: <1061989918.2155972.1452865072808.JavaMail.zimbra@u-pem.fr> Hi Stephan, mixing methods that create a new entry (your mapValue) with one that mutate the current entry (setValue) is still a bad idea. I think it's better to introduce a static method in Map.Entry like this: public static Function, Map.Entry> mapValue(Function valueMapper) { return entry -> Map.entry(entry.getKey(), valueMapper.apply(entry.getValue())); } and use it like this: map.entrySet().stream().map(mapValue(v -> v.transformed)) Also with the introduction of Map.entry() in 9, your "not really pleasant example" is a little better map.entrySet().stream().map(e -> entry(e.getKey(), e.getValue.transformed())) regards, R?mi PS: in 9, you can use Map.entry() instead of new AbstractMap.SimpleImmutableEntry<>(). ----- Mail original ----- > De: "Stephen Colebourne" > ?: "core-libs-dev" > Envoy?: Vendredi 15 Janvier 2016 14:05:51 > Objet: Map.Entry methods for streams > > I've had a morning of discussion about streaming Map this morning. While > there is clearly no appetite for a JDK MapStream right now, it does seem > that two additional methods on Map.Entry could help. > > Two of the common cases when streaming over Map.Entry are to transform the > keys and to transform the values. Currently, the code is as follows: > > hashMap.entrySet().stream() > .map(e -> new AbstractMap.SimpleImmutableEntry(e.getKey(), > e.getValue().transformed())) > ... > > This isn't really very pleasant. > > Adding a default method to Map.Entry would help: > > hashMap.entrySet().stream() > .map(e -> e.mapValue(v -> v.transformed())) > ... > > Implementation of the mapValue method is left to the reader, but it isn't > hard. A mapKey() method would also be needed. Note that both are designed > to work without mutating the map (returning a new entry). > > Finally, the Collectors class could do with a new method entriesToMap() > that collects a stream of Map.Entry back into a Map. > > While these proposals are not as powerful as a MapStream, they would smooth > some rough edges in the API when working with maps. > > Stephen > From ramanand.patil at oracle.com Fri Jan 15 15:31:56 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Fri, 15 Jan 2016 07:31:56 -0800 (PST) Subject: RFR: JDK-8144988: Unexpected timezone returned after parsing a date In-Reply-To: <56985DCD.4070706@oracle.com> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> <56985DCD.4070706@oracle.com> Message-ID: <144a4088-0d5e-4c7c-b249-eb62198f0a01@default> Hi Masayoshi, Thank you for pointing that out. I have removed line 29 from the test. Please review the updated Webrev: http://cr.openjdk.java.net/~rpatil/8141243/webrev.01/ Regards, Ramanand. -----Original Message----- From: Masayoshi Okutsu Sent: Friday, January 15, 2016 8:18 AM To: Ramanand Patil; i18n-dev at openjdk.java.net Cc: core-libs-dev at openjdk.java.net Subject: Re: RFR: JDK-8144988: Unexpected timezone returned after parsing a date Hi Ramanand, test/java/text/Format/DateFormat/Bug8141243.java: 28 * @run main Bug8141243 29 * @run main/othervm -Djava.locale.providers=COMPAT Bug8141243 "COMPAT" is a new name of "JRE" in JDK 9. It's not supported in 8u. I think COMPAT is slightly ignored and that it becomes the same thing as line 28. Line 29 should be removed. Otherwise, the fix looks OK to me. Masayoshi On 1/14/2016 9:21 PM, Ramanand Patil wrote: > Hi all, > > Please review the fix for bug: > https://bugs.openjdk.java.net/browse/JDK-8144988 > > Webrev: http://cr.openjdk.java.net/~rpatil/8141243/webrev.00 > > This is basically a backport of JDK9 bug: > https://bugs.openjdk.java.net/browse/JDK-8141243 > > JDK9 changeset(for reference): > http://hg.openjdk.java.net/jdk9/dev/jdk/rev/a1aa2671f281 > > Reason for the review request is because of: > i) Changes present in ResourceBundleGenerator.java file. > ii) The patch from JDK9 does not automatically apply as is after using unshuffle_patch script. Few paths are adjusted as per the jdk8. > > Since CLDR became the default locale data in JDK9 leading incompatible behavior with prior releases, the relevant code in ResourceBundleGenerator is also backported in this patch. > Even though JDK8 has CLDR locale provider disabled by default, the code change is done to be on safer side for cases where CLDR may be supporting "UTC" ID in the future. > > > Regards, > Ramanand. From peter.levart at gmail.com Fri Jan 15 15:44:14 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 15 Jan 2016 16:44:14 +0100 Subject: Map.Entry methods for streams In-Reply-To: <1061989918.2155972.1452865072808.JavaMail.zimbra@u-pem.fr> References: <1061989918.2155972.1452865072808.JavaMail.zimbra@u-pem.fr> Message-ID: <569913CE.8080602@gmail.com> Hi, Another possibility would be to introduce the following default methods to Map.Entry: public interface Map.Entry { ... default Map.Entry withKey(L key) { ... } default Map.Entry withValue(W value) { ... } } Usage would then look like: map.entrySet().stream().map(e -> e.withValue(e.getValue().transformed())) i.e. I don't think there's a need for function-taking methods like mapKey/mapValue, because Map.Entry returning methods are not common in APIs (Iterator.next() is a rare exception)... Optional returning methods OTOH are common in APIs, so function-taking methods on Optional are useful. Not so with Map.Entry, I think. Regards, Peter On 01/15/2016 02:37 PM, Remi Forax wrote: > Hi Stephan, > mixing methods that create a new entry (your mapValue) with one that mutate the current entry (setValue) is still a bad idea. > > I think it's better to introduce a static method in Map.Entry like this: > public static Function, Map.Entry> mapValue(Function valueMapper) { > return entry -> Map.entry(entry.getKey(), valueMapper.apply(entry.getValue())); > } > > and use it like this: > map.entrySet().stream().map(mapValue(v -> v.transformed)) > > Also with the introduction of Map.entry() in 9, your "not really pleasant example" is a little better > map.entrySet().stream().map(e -> entry(e.getKey(), e.getValue.transformed())) > > regards, > R?mi > > PS: in 9, you can use Map.entry() instead of new AbstractMap.SimpleImmutableEntry<>(). > > ----- Mail original ----- >> De: "Stephen Colebourne" >> ?: "core-libs-dev" >> Envoy?: Vendredi 15 Janvier 2016 14:05:51 >> Objet: Map.Entry methods for streams >> >> I've had a morning of discussion about streaming Map this morning. While >> there is clearly no appetite for a JDK MapStream right now, it does seem >> that two additional methods on Map.Entry could help. >> >> Two of the common cases when streaming over Map.Entry are to transform the >> keys and to transform the values. Currently, the code is as follows: >> >> hashMap.entrySet().stream() >> .map(e -> new AbstractMap.SimpleImmutableEntry(e.getKey(), >> e.getValue().transformed())) >> ... >> >> This isn't really very pleasant. >> >> Adding a default method to Map.Entry would help: >> >> hashMap.entrySet().stream() >> .map(e -> e.mapValue(v -> v.transformed())) >> ... >> >> Implementation of the mapValue method is left to the reader, but it isn't >> hard. A mapKey() method would also be needed. Note that both are designed >> to work without mutating the map (returning a new entry). >> >> Finally, the Collectors class could do with a new method entriesToMap() >> that collects a stream of Map.Entry back into a Map. >> >> While these proposals are not as powerful as a MapStream, they would smooth >> some rough edges in the API when working with maps. >> >> Stephen >> From amaembo at gmail.com Fri Jan 15 15:51:02 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 15 Jan 2016 21:51:02 +0600 Subject: Map.Entry methods for streams In-Reply-To: References: Message-ID: <1149065449.20160115215102@gmail.com> Hello! SC> Finally, the Collectors class could do with a new method entriesToMap() SC> that collects a stream of Map.Entry back into a Map. I was thinking about adding such collector into my library and checked StackOverflow to understand the useful scenarios. Seems that having entriesToMap() collector without arguments (even along with proposed mapValues()/mapKeys()) only would be too limited and solve only some subset of interesting problems. First, you would need to add versions with a) custom merging policy b) custom Map factory (probably could be omitted as toMap() does not have such overload) c) both Answers which need this: http://stackoverflow.com/a/22132422/ http://stackoverflow.com/a/33724926/ http://stackoverflow.com/a/30084289/ And many more Second, to be consistent with the current API you may need to add entriesToConcurrentMap() Third, sometimes you need to know both key and value to produce the new key. In this case proposed mapValues()/mapKeys() will not work: http://stackoverflow.com/a/34766036/ Fourth, very often entries from the stream should be grouped instead, so probably something like groupingEntries() should also be added (along with Map factory argument, downstream collector argument and concurrent version!) The most common scenario is the following: groupingBy(Entry::getKey, mapping(Entry::getValue, toList())) http://stackoverflow.com/a/25246138/ http://stackoverflow.com/a/31488612/ http://stackoverflow.com/a/30877403/ And many more To my opinion, creating such dedicated collector is even more important than toMap() as people understand toMap(Entry::getKey, Entry::getValue) much better than cascaded combination of groupingBy, mapping and toList. See Stuart Marks explaining: http://stackoverflow.com/a/28599035/ Sometimes it works without additional mapping: groupingBy(Map.Entry::getKey, summingLong(Map.Entry::getValue)) http://stackoverflow.com/a/34325686/ http://stackoverflow.com/a/31751786/ Also sometimes you need to swap key and value like here: groupingBy(e->e.getValue().get(0), mapping(Map.Entry::getKey, toList())) http://stackoverflow.com/a/30652372/ http://stackoverflow.com/a/33900949/ So probably Entry swap() method would also be useful! In general it's not very evident where to draw the line between API simplicity and covered use cases. Hopefully my examples will help to take good decision. By the way in my library MapStream is called as EntryStream. Like IntStream is stream of ints, the EntryStream is stream of entries. MapStream would be the stream of maps which is not true. Also in context of the Stream API map often means transformation which would add the confusion. With best regards, Tagir Valeev. From paul.sandoz at oracle.com Fri Jan 15 17:18:18 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jan 2016 18:18:18 +0100 Subject: Clarification of BaseStream.onClose() behavior In-Reply-To: <1666489469.20160115103543@gmail.com> References: <1666489469.20160115103543@gmail.com> Message-ID: <68E50067-D3FB-414F-80AD-40F188DAE5FC@oracle.com> Hi Tagir, AutoCloseable.close states: * However, implementers of this interface are strongly encouraged * to make their {@code close} methods idempotent. It?s possible we just did not consider this aspect too much in BaseStream.close and we just lent on the "strongly encouraged?, which seems fine to me to continue to do so. We did not bother to throw ISEs from parallel/sequence/onClose primarily because they return ?this? and they are kind of harmless, but the undefined behaviour you point out could be shut down without much concern for backwards compatibility. We can modify them to check whether they have been linked or consumed (while not updating the previous stage). Wanna log an issue? Thanks, Paul. > On 15 Jan 2016, at 05:35, Tagir F. Valeev wrote: > > Hello! > > Current documentation for BaseStream.onClose() does not specify > explicitly how the stream would behave if additional close handlers > are registered after the stream is consumed or even closed. The > implementation actually allows registering the additional close > handlers after consumption or closure and close() method is > idempotent: newly registered handlers are perfectly called: > > Stream s = Stream.of("content"); > s = s.onClose(() -> System.out.println("A")); > s.forEach(System.out::println); > s = s.onClose(() -> System.out.println("B")); > s.close(); // prints A and B > s = s.onClose(() -> System.out.println("C")); > s.close(); // prints C > > (removing "s =" produces the same result). > > I think if such behavior is intended, it should be explicitly noted in > the specification, as third-party implementations of BaseStream > interface should provide consistent behavior. Or at least it should be > noted that some BaseStream implementations may have idempotent close() > method, so the derived AutoCloseable objects (which own the BaseStream > object) should be aware about this behavior and provide idempotent > close() method as well. Without knowing this one may write: > > class MyObject implements AutoCloseable { > private BaseStream s; > > public MyObject(BaseStream s) { > this.s = s; > } > > @Override > public void close() throws Exception { > if(s != null) { > s.close(); > s = null; > } > } > } > > Such code closes the stream only once, so newly registered handlers > will not be called if MyObject::close is called the second time. > > However, to my opinion the current behavior is weird and should be > changed in order not to allow registering new close handles (throwing > IllegalStateException) when the stream is already closed, or even > better when the stream is linked/consumed. As far as I know currently > in JDK close handlers are registered only for non-consumed stream, so > such change would not break existing JDK code (though may break some > strange third party code). It should be noted that AutoCloseable > interface discourages idempotent close() methods (though not forbids > them). > > What do you think? > > With best regards, > Tagir Valeev. > From paul.sandoz at oracle.com Fri Jan 15 17:27:01 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 15 Jan 2016 18:27:01 +0100 Subject: Map.Entry methods for streams In-Reply-To: <1149065449.20160115215102@gmail.com> References: <1149065449.20160115215102@gmail.com> Message-ID: Hi, We have had multiple attempts at a BiStream for references. Mike had quite a good one. I thought my last attempt was rather good too :-) it avoided boxing on critical paths, while avoiding much bifurcation of the Spliterator API, and supported some nice use-cases. However, it would likely stick out like a sore thumb when Value types and then Tuples arrive (when? ?when they are ready!?), and then we can do much better. Some small tweaks to Map.Entry might be handy, but i think beyond that and it?s likely to start pulling on a bigger set of use-cases. Paul. > On 15 Jan 2016, at 16:51, Tagir F. Valeev wrote: > > Hello! > > SC> Finally, the Collectors class could do with a new method entriesToMap() > SC> that collects a stream of Map.Entry back into a Map. > > I was thinking about adding such collector into my library and checked > StackOverflow to understand the useful scenarios. Seems that having > entriesToMap() collector without arguments (even along with proposed > mapValues()/mapKeys()) only would be too limited and solve only some > subset of interesting problems. > > First, you would need to add versions with > a) custom merging policy > b) custom Map factory (probably could be omitted as toMap() does not > have such overload) > c) both > Answers which need this: > http://stackoverflow.com/a/22132422/ > http://stackoverflow.com/a/33724926/ > http://stackoverflow.com/a/30084289/ > And many more > > Second, to be consistent with the current API you may need to add > entriesToConcurrentMap() > > Third, sometimes you need to know both key and value to produce the > new key. In this case proposed mapValues()/mapKeys() will not work: > http://stackoverflow.com/a/34766036/ > > Fourth, very often entries from the stream should be grouped instead, > so probably something like groupingEntries() should also be added > (along with Map factory argument, downstream collector argument and > concurrent version!) The most common scenario is the following: > > groupingBy(Entry::getKey, > mapping(Entry::getValue, toList())) > http://stackoverflow.com/a/25246138/ > http://stackoverflow.com/a/31488612/ > http://stackoverflow.com/a/30877403/ > And many more > > To my opinion, creating such dedicated collector is even more > important than toMap() as people understand toMap(Entry::getKey, > Entry::getValue) much better than cascaded combination of groupingBy, > mapping and toList. See Stuart Marks explaining: > http://stackoverflow.com/a/28599035/ > > Sometimes it works without additional mapping: > groupingBy(Map.Entry::getKey, summingLong(Map.Entry::getValue)) > http://stackoverflow.com/a/34325686/ > http://stackoverflow.com/a/31751786/ > > Also sometimes you need to swap key and value like here: > groupingBy(e->e.getValue().get(0), > mapping(Map.Entry::getKey, toList())) > http://stackoverflow.com/a/30652372/ > http://stackoverflow.com/a/33900949/ > So probably Entry swap() method would also be useful! > > In general it's not very evident where to draw the line between API > simplicity and covered use cases. Hopefully my examples will help to > take good decision. > > By the way in my library MapStream is called as EntryStream. Like > IntStream is stream of ints, the EntryStream is stream of entries. > MapStream would be the stream of maps which is not true. Also in > context of the Stream API map often means transformation which would > add the confusion. > > With best regards, > Tagir Valeev. > From joe.darcy at oracle.com Fri Jan 15 18:01:01 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 15 Jan 2016 10:01:01 -0800 Subject: JDK 9 RFR of JDK-8147480: Mark FJExceptionTableLeak.java as intermittently failing Message-ID: <569933DD.2010509@oracle.com> Hello, We've seen some more low-frequency failures of java/util/concurrent/forkjoin/FJExceptionTableLeak.java Please review the patch below which records the known-intermittentness in the source of the test and demotes the test from tier 1 to tier 2. When the issue is resolved (JDK-8144990), the the keyword can be removed and the test raised up to tier 1 again. Thanks, -Joe diff -r e00d22e96e6e test/TEST.groups --- a/test/TEST.groups Thu Jan 14 12:04:19 2016 -0800 +++ b/test/TEST.groups Fri Jan 15 10:00:18 2016 -0800 @@ -1,4 +1,4 @@ -# Copyright (c) 2013, 2015, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2013, 2016, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. # # This code is free software; you can redistribute it and/or modify it @@ -32,6 +32,7 @@ :jdk_util \ -java/util/WeakHashMap/GCDuringIteration.java \ -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java + -java/util/concurrent/forkjoin/FJExceptionTableLeak.java sun/nio/cs/ISO8859x.java \ java/nio/Buffer \ com/sun/crypto/provider/Cipher \ @@ -42,6 +43,7 @@ java/util/zip/TestLocalTime.java \ java/util/WeakHashMap/GCDuringIteration.java \ java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ + java/util/concurrent/forkjoin/FJExceptionTableLeak.java :jdk_io \ :jdk_nio \ -sun/nio/cs/ISO8859x.java \ diff -r e00d22e96e6e test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java --- a/test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java Thu Jan 14 12:04:19 2016 -0800 +++ b/test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java Fri Jan 15 10:00:18 2016 -0800 @@ -37,6 +37,7 @@ * @bug 8004138 * @summary Check if ForkJoinPool table leaks thrown exceptions. * @run main/othervm -Xmx2200k FJExceptionTableLeak + * @key intermittent */ import java.util.concurrent.ForkJoinPool; From roger.riggs at oracle.com Fri Jan 15 18:03:37 2016 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 15 Jan 2016 13:03:37 -0500 Subject: JDK 9 RFR of JDK-8147480: Mark FJExceptionTableLeak.java as intermittently failing In-Reply-To: <569933DD.2010509@oracle.com> References: <569933DD.2010509@oracle.com> Message-ID: <56993479.1080401@oracle.com> Looks fine Joe. On 1/15/16 1:01 PM, joe darcy wrote: > Hello, > > We've seen some more low-frequency failures of > > java/util/concurrent/forkjoin/FJExceptionTableLeak.java > > Please review the patch below which records the > known-intermittentness in the source of the test and demotes the test > from tier 1 to tier 2. When the issue is resolved (JDK-8144990), the > the keyword can be removed and the test raised up to tier 1 again. > > Thanks, > > -Joe > > diff -r e00d22e96e6e test/TEST.groups > --- a/test/TEST.groups Thu Jan 14 12:04:19 2016 -0800 > +++ b/test/TEST.groups Fri Jan 15 10:00:18 2016 -0800 > @@ -1,4 +1,4 @@ > -# Copyright (c) 2013, 2015, Oracle and/or its affiliates. All rights > reserved. > +# Copyright (c) 2013, 2016, Oracle and/or its affiliates. All rights > reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > # > # This code is free software; you can redistribute it and/or modify it > @@ -32,6 +32,7 @@ > :jdk_util \ > -java/util/WeakHashMap/GCDuringIteration.java \ > -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java > + -java/util/concurrent/forkjoin/FJExceptionTableLeak.java > sun/nio/cs/ISO8859x.java \ > java/nio/Buffer \ > com/sun/crypto/provider/Cipher \ > @@ -42,6 +43,7 @@ > java/util/zip/TestLocalTime.java \ > java/util/WeakHashMap/GCDuringIteration.java \ > java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ > + java/util/concurrent/forkjoin/FJExceptionTableLeak.java > :jdk_io \ > :jdk_nio \ > -sun/nio/cs/ISO8859x.java \ > diff -r e00d22e96e6e > test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java > --- a/test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java Thu > Jan 14 12:04:19 2016 -0800 > +++ b/test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java Fri > Jan 15 10:00:18 2016 -0800 > @@ -37,6 +37,7 @@ > * @bug 8004138 > * @summary Check if ForkJoinPool table leaks thrown exceptions. > * @run main/othervm -Xmx2200k FJExceptionTableLeak > + * @key intermittent > */ > > import java.util.concurrent.ForkJoinPool; > From lance.andersen at oracle.com Fri Jan 15 18:03:22 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 15 Jan 2016 13:03:22 -0500 Subject: JDK 9 RFR of JDK-8147480: Mark FJExceptionTableLeak.java as intermittently failing In-Reply-To: <569933DD.2010509@oracle.com> References: <569933DD.2010509@oracle.com> Message-ID: +1 On Jan 15, 2016, at 1:01 PM, joe darcy wrote: > Hello, > > We've seen some more low-frequency failures of > > java/util/concurrent/forkjoin/FJExceptionTableLeak.java > > Please review the patch below which records the known-intermittentness in the source of the test and demotes the test from tier 1 to tier 2. When the issue is resolved (JDK-8144990), the the keyword can be removed and the test raised up to tier 1 again. > > Thanks, > > -Joe > > diff -r e00d22e96e6e test/TEST.groups > --- a/test/TEST.groups Thu Jan 14 12:04:19 2016 -0800 > +++ b/test/TEST.groups Fri Jan 15 10:00:18 2016 -0800 > @@ -1,4 +1,4 @@ > -# Copyright (c) 2013, 2015, Oracle and/or its affiliates. All rights reserved. > +# Copyright (c) 2013, 2016, Oracle and/or its affiliates. All rights reserved. > # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > # > # This code is free software; you can redistribute it and/or modify it > @@ -32,6 +32,7 @@ > :jdk_util \ > -java/util/WeakHashMap/GCDuringIteration.java \ > -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java > + -java/util/concurrent/forkjoin/FJExceptionTableLeak.java > sun/nio/cs/ISO8859x.java \ > java/nio/Buffer \ > com/sun/crypto/provider/Cipher \ > @@ -42,6 +43,7 @@ > java/util/zip/TestLocalTime.java \ > java/util/WeakHashMap/GCDuringIteration.java \ > java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ > + java/util/concurrent/forkjoin/FJExceptionTableLeak.java > :jdk_io \ > :jdk_nio \ > -sun/nio/cs/ISO8859x.java \ > diff -r e00d22e96e6e test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java > --- a/test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java Thu Jan 14 12:04:19 2016 -0800 > +++ b/test/java/util/concurrent/forkjoin/FJExceptionTableLeak.java Fri Jan 15 10:00:18 2016 -0800 > @@ -37,6 +37,7 @@ > * @bug 8004138 > * @summary Check if ForkJoinPool table leaks thrown exceptions. > * @run main/othervm -Xmx2200k FJExceptionTableLeak > + * @key intermittent > */ > > import java.util.concurrent.ForkJoinPool; > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From timo.kinnunen at gmail.com Fri Jan 15 19:02:54 2016 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Fri, 15 Jan 2016 20:02:54 +0100 Subject: Map.Entry methods for streams In-Reply-To: <569913CE.8080602@gmail.com> References: <1061989918.2155972.1452865072808.JavaMail.zimbra@u-pem.fr> <569913CE.8080602@gmail.com> Message-ID: <56994266.8673c20a.08fa.79d1@mx.google.com> Hi, For completeness, having also the methods default Entry withKeyAsValue(NK newKey) { return new AbstractMap.SimpleEntry<>(newKey, getKey()); } public Entry withValueAsKey(NV newValue) { return new AbstractMap.SimpleEntry<>(getValue(), newValue); } would be useful when inverting a graph, for example: Map> dag = //. . . Map> gad = dag.entrySet() .stream() .flatMap(e -> e.getValue().stream().map(e::withKeyAsValue)) .collect(groupingBy(Entry::getKey, mapping(Entry::getValue, toList()))); -- Have a nice day, Timo Sent from Mail for Windows 10 From: Peter Levart Sent: Friday, January 15, 2016 16:44 To: Remi Forax; Stephen Colebourne Cc: core-libs-dev Subject: Re: Map.Entry methods for streams Hi, Another possibility would be to introduce the following default methods to Map.Entry: public interface Map.Entry { ... default Map.Entry withKey(L key) { ... } default Map.Entry withValue(W value) { ... } } Usage would then look like: map.entrySet().stream().map(e -> e.withValue(e.getValue().transformed())) i.e. I don't think there's a need for function-taking methods like mapKey/mapValue, because Map.Entry returning methods are not common in APIs (Iterator.next() is a rare exception)... Optional returning methods OTOH are common in APIs, so function-taking methods on Optional are useful. Not so with Map.Entry, I think. Regards, Peter On 01/15/2016 02:37 PM, Remi Forax wrote: > Hi Stephan, > mixing methods that create a new entry (your mapValue) with one that mutate the current entry (setValue) is still a bad idea. > > I think it's better to introduce a static method in Map.Entry like this: > public static Function, Map.Entry> mapValue(Function valueMapper) { > return entry -> Map.entry(entry.getKey(), valueMapper.apply(entry.getValue())); > } > > and use it like this: > map.entrySet().stream().map(mapValue(v -> v.transformed)) > > Also with the introduction of Map.entry() in 9, your "not really pleasant example" is a little better > map.entrySet().stream().map(e -> entry(e.getKey(), e.getValue.transformed())) > > regards, > R?mi > > PS: in 9, you can use Map.entry() instead of new AbstractMap.SimpleImmutableEntry<>(). > > ----- Mail original ----- >> De: "Stephen Colebourne" >> ?: "core-libs-dev" >> Envoy?: Vendredi 15 Janvier 2016 14:05:51 >> Objet: Map.Entry methods for streams >> >> I've had a morning of discussion about streaming Map this morning. While >> there is clearly no appetite for a JDK MapStream right now, it does seem >> that two additional methods on Map.Entry could help. >> >> Two of the common cases when streaming over Map.Entry are to transform the >> keys and to transform the values. Currently, the code is as follows: >> >> hashMap.entrySet().stream() >> .map(e -> new AbstractMap.SimpleImmutableEntry(e.getKey(), >> e.getValue().transformed())) >> ... >> >> This isn't really very pleasant. >> >> Adding a default method to Map.Entry would help: >> >> hashMap.entrySet().stream() >> .map(e -> e.mapValue(v -> v.transformed())) >> ... >> >> Implementation of the mapValue method is left to the reader, but it isn't >> hard. A mapKey() method would also be needed. Note that both are designed >> to work without mutating the map (returning a new entry). >> >> Finally, the Collectors class could do with a new method entriesToMap() >> that collects a stream of Map.Entry back into a Map. >> >> While these proposals are not as powerful as a MapStream, they would smooth >> some rough edges in the API when working with maps. >> >> Stephen >> From lance.andersen at oracle.com Fri Jan 15 19:22:27 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 15 Jan 2016 14:22:27 -0500 Subject: RFR (JAXP): 8147051: StaxEntityResolverWrapper should create StaxXMLInputSource with a resolver indicator In-Reply-To: <5697345E.7020408@oracle.com> References: <5697345E.7020408@oracle.com> Message-ID: <5439EAC4-2AD9-4C90-8331-C284761E5369@oracle.com> Hi Joe, This looks fine Best Lance On Jan 14, 2016, at 12:38 AM, huizhe wang wrote: > Happen to notice that the indicator that the source was created by a resolver was not set in StaxEntityResolverWrapper. Also, it's "IsCreatedByResolver" rather than "hasResolver". > > JBS: https://bugs.openjdk.java.net/browse/JDK-8147051 > > webrev: http://cr.openjdk.java.net/~joehw/jdk9/8147051/webrev/ > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From john.r.rose at oracle.com Fri Jan 15 21:24:37 2016 From: john.r.rose at oracle.com (John Rose) Date: Fri, 15 Jan 2016 13:24:37 -0800 Subject: RFR 8146458 Improve exception reporting for Objects.checkIndex/checkFromToIndex/checkFromIndexSize In-Reply-To: <40431638-BC6A-44E2-8A5E-8F0B180AA924@oracle.com> References: <40431638-BC6A-44E2-8A5E-8F0B180AA924@oracle.com> Message-ID: On Jan 11, 2016, at 7:04 AM, Paul Sandoz wrote: > > When the new range check methods Object.check* were added the exception reporting was a little vague and lossy, but i got a CCC pass to revisit later on, and John nudged me to sort this out, so here is another more reflective approach: > > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/webrev/ > http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/specdiff/overview-summary.html I like it. (No surprise there.) In the new API point the term "mapper", and "exception mapping function" in the existing API points, doesn't really convey much to me. The main cog. diss. is that, at the time the EMF is called, there is no exception created yet, although it is true that an exceptional condition has been identifier. But the exception thrown is the first to be created; it is not "mapped" from a previous exception. So I'd prefer "factory", personally: "factory", "exception factory [function]". A T-factory maps () to T, while a T-mapper maps (T) to T. > ... > > The two int arg constructors added to Array/String/IndexOutOfBoundsException have been removed. Yes; that pays for the new API point, IMO. Very nice. One nit: -+ // Switch to default if fewer arguments than required are supplied -+ switch ((args.size() < argSize) ? "" : checkKind) { ++ // Switch to default if wrong number of arguments is supplied ++ switch ((args.size() != argSize) ? "" : checkKind) { That is, excess values should show up, so they can be debugged. (Unless you have a use case in mind?) Would you mind including a test case which exercises, say, ArrayIndexOutOfBoundsException::new? Might as well make sure those wires connect properly; test the class of the thrown exception as well as the message. > I did ponder about adding a new constructors with String, List arguments. I could still do that, but that does spread out the notion beyond that of Objects and i prefer that it be contained. Nope. Keep this thing localized. Reviewed, if you need a reviewer. ? John From stuart.marks at oracle.com Fri Jan 15 22:43:46 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 15 Jan 2016 14:43:46 -0800 Subject: RFR 8146458 Improve exception reporting for Objects.checkIndex/checkFromToIndex/checkFromIndexSize In-Reply-To: References: <40431638-BC6A-44E2-8A5E-8F0B180AA924@oracle.com> Message-ID: <56997622.8000505@oracle.com> Hi Paul, I agree with John on a couple points, namely the removal of the two-arg constructors from the exception types, and the use of a term like "factory" instead of "mapper" for the exception-instance-producing functions. An alternative might be "supplier" (though that might cause confusion with the Supplier functional interface). I'm concerned about a few other things though. I spent some time going over the change this morning and it took a while to digest. My hunch is that for what this is trying to do, it could be simpler, though I haven't yet fully absorbed all of what's going on here to come up with concrete suggestions for how to do so. One area in particular is the message formatter functions. There are really three cases smashed into one: KIND ARGS checkIndex index, length checkFromToIndex from, to, length checkFromIndexSize from, size, length I think, in order to avoid introducing new functional interfaces, this information is funneled into a single varargs call, which has to use Integer boxes, because it then wraps its args into a List (thanks for using the new List factory!), which then has to unpack the list and check its length against the number of arguments for the particular message kind. I understand that this is the "slow" exception-throwing path, but my sense is still that this ought to be made simpler. Perhaps the scope of what it's trying to accomplished should be reduced, if that helps things. I'll continue to think about this further. I also have a naming bikeshed but that can be postponed. :-) s'marks On 1/15/16 1:24 PM, John Rose wrote: > On Jan 11, 2016, at 7:04 AM, Paul Sandoz wrote: >> >> When the new range check methods Object.check* were added the exception reporting was a little vague and lossy, but i got a CCC pass to revisit later on, and John nudged me to sort this out, so here is another more reflective approach: >> >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/webrev/ >> http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/specdiff/overview-summary.html > > I like it. (No surprise there.) > > In the new API point the term "mapper", and "exception mapping function" in the existing > API points, doesn't really convey much to me. > > The main cog. diss. is that, at the time the EMF is called, there is no exception created yet, > although it is true that an exceptional condition has been identifier. But the exception thrown > is the first to be created; it is not "mapped" from a previous exception. > > So I'd prefer "factory", personally: "factory", "exception factory [function]". > A T-factory maps () to T, while a T-mapper maps (T) to T. > >> ... >> >> The two int arg constructors added to Array/String/IndexOutOfBoundsException have been removed. > > Yes; that pays for the new API point, IMO. Very nice. > > One nit: > > -+ // Switch to default if fewer arguments than required are supplied > -+ switch ((args.size() < argSize) ? "" : checkKind) { > ++ // Switch to default if wrong number of arguments is supplied > ++ switch ((args.size() != argSize) ? "" : checkKind) { > > That is, excess values should show up, so they can be debugged. > (Unless you have a use case in mind?) > > Would you mind including a test case which exercises, say, > ArrayIndexOutOfBoundsException::new? Might as well > make sure those wires connect properly; test the class of > the thrown exception as well as the message. > >> I did ponder about adding a new constructors with String, List arguments. I could still do that, but that does spread out the notion beyond that of Objects and i prefer that it be contained. > > Nope. Keep this thing localized. > > Reviewed, if you need a reviewer. > > ? John > From scolebourne at joda.org Sat Jan 16 01:12:44 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Sat, 16 Jan 2016 01:12:44 +0000 Subject: Map.Entry methods for streams In-Reply-To: References: <1149065449.20160115215102@gmail.com> Message-ID: So, this proposal is essentially trying to add something small to JDK 9 in advance of the long wait for value types and a full solution to the problem space. A stream of Map.Entry is painful today, and my idea or Remi's would definitely assist without pushing the API too far (I'm not such a fan of Peter's suggestion). As it happens, we're also looking at a dedicated MapStream now: https://github.com/OpenGamma/Strata/commit/ee42c4dac90a1f08b43af6eec799d7c5e876ab4a#diff-b1758880cf04177f31c86fe8e1d5b012R36 Stephen On 15 January 2016 at 17:27, Paul Sandoz wrote: > Hi, > > We have had multiple attempts at a BiStream for references. > > Mike had quite a good one. I thought my last attempt was rather good too > :-) it avoided boxing on critical paths, while avoiding much bifurcation of > the Spliterator API, and supported some nice use-cases. However, it would > likely stick out like a sore thumb when Value types and then Tuples arrive > (when? ?when they are ready!?), and then we can do much better. > > Some small tweaks to Map.Entry might be handy, but i think beyond that and > it?s likely to start pulling on a bigger set of use-cases. > > Paul. > > > > On 15 Jan 2016, at 16:51, Tagir F. Valeev wrote: > > > > Hello! > > > > SC> Finally, the Collectors class could do with a new method > entriesToMap() > > SC> that collects a stream of Map.Entry back into a Map. > > > > I was thinking about adding such collector into my library and checked > > StackOverflow to understand the useful scenarios. Seems that having > > entriesToMap() collector without arguments (even along with proposed > > mapValues()/mapKeys()) only would be too limited and solve only some > > subset of interesting problems. > > > > First, you would need to add versions with > > a) custom merging policy > > b) custom Map factory (probably could be omitted as toMap() does not > > have such overload) > > c) both > > Answers which need this: > > http://stackoverflow.com/a/22132422/ > > http://stackoverflow.com/a/33724926/ > > http://stackoverflow.com/a/30084289/ > > And many more > > > > Second, to be consistent with the current API you may need to add > > entriesToConcurrentMap() > > > > Third, sometimes you need to know both key and value to produce the > > new key. In this case proposed mapValues()/mapKeys() will not work: > > http://stackoverflow.com/a/34766036/ > > > > Fourth, very often entries from the stream should be grouped instead, > > so probably something like groupingEntries() should also be added > > (along with Map factory argument, downstream collector argument and > > concurrent version!) The most common scenario is the following: > > > > groupingBy(Entry::getKey, > > mapping(Entry::getValue, toList())) > > http://stackoverflow.com/a/25246138/ > > http://stackoverflow.com/a/31488612/ > > http://stackoverflow.com/a/30877403/ > > And many more > > > > To my opinion, creating such dedicated collector is even more > > important than toMap() as people understand toMap(Entry::getKey, > > Entry::getValue) much better than cascaded combination of groupingBy, > > mapping and toList. See Stuart Marks explaining: > > http://stackoverflow.com/a/28599035/ > > > > Sometimes it works without additional mapping: > > groupingBy(Map.Entry::getKey, summingLong(Map.Entry::getValue)) > > http://stackoverflow.com/a/34325686/ > > http://stackoverflow.com/a/31751786/ > > > > Also sometimes you need to swap key and value like here: > > groupingBy(e->e.getValue().get(0), > > mapping(Map.Entry::getKey, toList())) > > http://stackoverflow.com/a/30652372/ > > http://stackoverflow.com/a/33900949/ > > So probably Entry swap() method would also be useful! > > > > In general it's not very evident where to draw the line between API > > simplicity and covered use cases. Hopefully my examples will help to > > take good decision. > > > > By the way in my library MapStream is called as EntryStream. Like > > IntStream is stream of ints, the EntryStream is stream of entries. > > MapStream would be the stream of maps which is not true. Also in > > context of the Stream API map often means transformation which would > > add the confusion. > > > > With best regards, > > Tagir Valeev. > > > > From amaembo at gmail.com Sat Jan 16 02:59:26 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sat, 16 Jan 2016 08:59:26 +0600 Subject: Map.Entry methods for streams In-Reply-To: <56994266.8673c20a.08fa.79d1@mx.google.com> References: <1061989918.2155972.1452865072808.JavaMail.zimbra@u-pem.fr> <569913CE.8080602@gmail.com> <56994266.8673c20a.08fa.79d1@mx.google.com> Message-ID: <1541458872.20160116085926@gmail.com> Hello! Note that having single swap() method which just swaps key and value in addition to withKey/withValue covers this use case: Map> gad = dag.entrySet() .stream() .flatMap(e -> e.getValue().stream().map(e.swap()::withKey)) .collect(groupingBy(Entry::getKey, mapping(Entry::getValue, toList()))); TK> For completeness, having also the methods TK> default Entry withKeyAsValue(NK newKey) { TK> return new AbstractMap.SimpleEntry<>(newKey, getKey()); TK> } TK> public Entry withValueAsKey(NV newValue) { TK> return new AbstractMap.SimpleEntry<>(getValue(), newValue); TK> } TK> would be useful when inverting a graph, for example: TK> Map> dag = //. . . TK> Map> gad = dag.entrySet() TK> .stream() TK> .flatMap(e -> TK> e.getValue().stream().map(e::withKeyAsValue)) TK> .collect(groupingBy(Entry::getKey, TK> mapping(Entry::getValue, toList()))); From amaembo at gmail.com Sat Jan 16 03:13:32 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sat, 16 Jan 2016 09:13:32 +0600 Subject: Clarification of BaseStream.onClose() behavior In-Reply-To: <68E50067-D3FB-414F-80AD-40F188DAE5FC@oracle.com> References: <1666489469.20160115103543@gmail.com> <68E50067-D3FB-414F-80AD-40F188DAE5FC@oracle.com> Message-ID: <1616962894.20160116091332@gmail.com> Hello! Seems that my original letter sounds confusing due to my bad English. Every occurrence of "idempotent" should be read as "not idempotent". Sorry for this! PS> We did not bother to throw ISEs from parallel/sequence/onClose PS> primarily because they return ?this? and they are kind of PS> harmless, but the undefined behaviour you point out could be shut PS> down without much concern for backwards compatibility. We can PS> modify them to check whether they have been linked or consumed PS> (while not updating the previous stage). PS> Wanna log an issue? Sure: https://bugs.openjdk.java.net/browse/JDK-8147505 Thanks, Tagir Valeev. PS> Thanks, PS> Paul. >> On 15 Jan 2016, at 05:35, Tagir F. Valeev wrote: >> >> Hello! >> >> Current documentation for BaseStream.onClose() does not specify >> explicitly how the stream would behave if additional close handlers >> are registered after the stream is consumed or even closed. The >> implementation actually allows registering the additional close >> handlers after consumption or closure and close() method is >> idempotent: newly registered handlers are perfectly called: >> >> Stream s = Stream.of("content"); >> s = s.onClose(() -> System.out.println("A")); >> s.forEach(System.out::println); >> s = s.onClose(() -> System.out.println("B")); >> s.close(); // prints A and B >> s = s.onClose(() -> System.out.println("C")); >> s.close(); // prints C >> >> (removing "s =" produces the same result). >> >> I think if such behavior is intended, it should be explicitly noted in >> the specification, as third-party implementations of BaseStream >> interface should provide consistent behavior. Or at least it should be >> noted that some BaseStream implementations may have idempotent close() >> method, so the derived AutoCloseable objects (which own the BaseStream >> object) should be aware about this behavior and provide idempotent >> close() method as well. Without knowing this one may write: >> >> class MyObject implements AutoCloseable { >> private BaseStream s; >> >> public MyObject(BaseStream s) { >> this.s = s; >> } >> >> @Override >> public void close() throws Exception { >> if(s != null) { >> s.close(); >> s = null; >> } >> } >> } >> >> Such code closes the stream only once, so newly registered handlers >> will not be called if MyObject::close is called the second time. >> >> However, to my opinion the current behavior is weird and should be >> changed in order not to allow registering new close handles (throwing >> IllegalStateException) when the stream is already closed, or even >> better when the stream is linked/consumed. As far as I know currently >> in JDK close handlers are registered only for non-consumed stream, so >> such change would not break existing JDK code (though may break some >> strange third party code). It should be noted that AutoCloseable >> interface discourages idempotent close() methods (though not forbids >> them). >> >> What do you think? >> >> With best regards, >> Tagir Valeev. >> From joe.darcy at oracle.com Sat Jan 16 07:21:42 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 15 Jan 2016 23:21:42 -0800 Subject: JDK 9 RFR of JDK-8147508: Correct fix for JDK-8147480 Message-ID: <5699EF86.1060501@oracle.com> Hello, Mea culpa, I omitted the line continuation characters in my recent fix for JDK-8147480. Please review the patch below to add the continuation characters and restore the full set of tests to tier 1 and tier 2 (also correcting a less serious typo from JDK-8144880 a few weeks back). Thanks, -Joe diff -r 16ae8f6c208c test/TEST.groups --- a/test/TEST.groups Fri Jan 15 10:30:56 2016 -0800 +++ b/test/TEST.groups Fri Jan 15 23:13:55 2016 -0800 @@ -31,8 +31,8 @@ -java/util/zip/TestLocalTime.java \ :jdk_util \ -java/util/WeakHashMap/GCDuringIteration.java \ - -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java - -java/util/concurrent/forkjoin/FJExceptionTableLeak.java + -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ + -java/util/concurrent/forkjoin/FJExceptionTableLeak.java \ sun/nio/cs/ISO8859x.java \ java/nio/Buffer \ com/sun/crypto/provider/Cipher \ @@ -43,7 +43,7 @@ java/util/zip/TestLocalTime.java \ java/util/WeakHashMap/GCDuringIteration.java \ java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ - java/util/concurrent/forkjoin/FJExceptionTableLeak.java + java/util/concurrent/forkjoin/FJExceptionTableLeak.java \ :jdk_io \ :jdk_nio \ -sun/nio/cs/ISO8859x.java \ From martinrb at google.com Sat Jan 16 07:36:55 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 15 Jan 2016 23:36:55 -0800 Subject: JDK 9 RFR of JDK-8147508: Correct fix for JDK-8147480 In-Reply-To: <5699EF86.1060501@oracle.com> References: <5699EF86.1060501@oracle.com> Message-ID: looks good! On Fri, Jan 15, 2016 at 11:21 PM, joe darcy wrote: > Hello, > > Mea culpa, I omitted the line continuation characters in my recent fix for > JDK-8147480. > > Please review the patch below to add the continuation characters and restore > the full set of tests to tier 1 and tier 2 (also correcting a less serious > typo from JDK-8144880 a few weeks back). > > Thanks, > > -Joe > > diff -r 16ae8f6c208c test/TEST.groups > --- a/test/TEST.groups Fri Jan 15 10:30:56 2016 -0800 > +++ b/test/TEST.groups Fri Jan 15 23:13:55 2016 -0800 > @@ -31,8 +31,8 @@ > -java/util/zip/TestLocalTime.java \ > :jdk_util \ > -java/util/WeakHashMap/GCDuringIteration.java \ > - -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java > - -java/util/concurrent/forkjoin/FJExceptionTableLeak.java > + -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ > + -java/util/concurrent/forkjoin/FJExceptionTableLeak.java \ > sun/nio/cs/ISO8859x.java \ > java/nio/Buffer \ > com/sun/crypto/provider/Cipher \ > @@ -43,7 +43,7 @@ > java/util/zip/TestLocalTime.java \ > java/util/WeakHashMap/GCDuringIteration.java \ > java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ > - java/util/concurrent/forkjoin/FJExceptionTableLeak.java > + java/util/concurrent/forkjoin/FJExceptionTableLeak.java \ > :jdk_io \ > :jdk_nio \ > -sun/nio/cs/ISO8859x.java \ > From john.r.rose at oracle.com Sat Jan 16 07:36:44 2016 From: john.r.rose at oracle.com (John Rose) Date: Fri, 15 Jan 2016 23:36:44 -0800 Subject: RFR 8146458 Improve exception reporting for Objects.checkIndex/checkFromToIndex/checkFromIndexSize In-Reply-To: <56997622.8000505@oracle.com> References: <40431638-BC6A-44E2-8A5E-8F0B180AA924@oracle.com> <56997622.8000505@oracle.com> Message-ID: <0B0A9076-D80E-49A9-AD8D-DB538901EE81@oracle.com> On Jan 15, 2016, at 2:43 PM, Stuart Marks wrote: > > > I think, in order to avoid introducing new functional interfaces, this information is funneled into a single varargs call, which has to use Integer boxes, because it then wraps its args into a List (thanks for using the new List factory!), which then has to unpack the list and check its length against the number of arguments for the particular message kind. And that's before String.format does a bunch more steps of the same kind, including boxing and varargs. And then the exception creation itself walks the entire thread stack. Micro-optimization is hopeless. But it doesn't matter because it's all on the slow path. > I understand that this is the "slow" exception-throwing path, but my sense is still that this ought to be made simpler. Perhaps the scope of what it's trying to accomplished should be reduced, if that helps things. Simpler would be good but I don't see it any simpler, yet. Do give it some thought! From amaembo at gmail.com Sat Jan 16 13:31:50 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sat, 16 Jan 2016 19:31:50 +0600 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> Message-ID: <1448206903.20160116193150@gmail.com> Hello! Thanks for review! Here's the updated patch: http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ SC> The docs say that if the end date is before the start date, the SC> stream is empty. While I can see just about why LongStream.range() SC> works that way, I don't think this API should. The minimum is an SC> exception, but it would be easy to support negative in the SC> days-only case or the months-only case. The problem is where there SC> are both months/years and days and those should be exceptions. Now datesUntil(endExclusive) throws an IllegalArgumentException if end date is before start date. datesUntil(endExclusive, step) supports negative periods. It throws IllegalArgumentException if: - step is zero - step.toTotalMonths() and step.getDays() both non-zero and have opposite sign - step is negative and end date is after start date - step is positive and end date is before start date Otherwise it works correctly: you can use LocalDate.of(2016, 1, 1) .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); Steps like Period.of(-1, -1, -1) are also supported. SC> The single-arg implementation uses plusDays() with an SC> incrementing number. When the performance patch goes in, the SC> proposed streaming implementation will be optimal for small SC> streams but less optimal for large ones. The fastest way to loop SC> over a list of dates would be to manually generate them by SC> incrementing the day until it exceeds the length of month, and so SC> on. However, this would be serial. As I understand, plusDays performance patch is already pushed. It's possible to write custom Spliterator which would use previous.plusDays(1) in tryAdvance() and jump (via LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is requested. However this adds at least one additional class and more complexity. I guess, such optimization can be considered later as separate issue when API is stabilized. SC> As such, I think the best way to write this, taking account of SC> how plusDays() is implemented, is as follows: SC> LongStream.range(start.toEpochDay(), SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); Ok, now it's done this way. SC> In the period-based method, it would be best to capture the case SC> where totalMonths == 0 and days > 0 and forward to another method SC> that ignores months. That private method can share implementation SC> with the public single-arg method (passing in 1). This case still more complex than one-day case as it requires division and multiplication. Thus I'd keep these case separately. However I simplified "months = 0" path using ofEpochDay, now it should be faster. SC> The docs for the period-based method should specify the strategy SC> that produces the results (in abstract terms). This needs to cover SC> that the result is equivalent to always adding to the start period SC> a multiple of the period. I added some clarifications, please check. SC> Some nits: SC> I prefer to avoid @link in the first sentence. Just using 'stream' would be sufficient. Done. SC> The first sentence should be a summary. In this case it probably has a bit too much detail. Done. SC> The @return has 'values' on a new line when it could be on the same line. I set now line length = 100 characters in my IDE. Is it ok? SC> If statements need braces. Done. With best regards, Tagir Valeev. From peter.levart at gmail.com Sun Jan 17 18:18:26 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 17 Jan 2016 19:18:26 +0100 Subject: Ephemerons Message-ID: <569BDAF2.2080404@gmail.com> Hi, Ephemerons are special kind of references described by Barry Hayes [1]. In the simple variant, they contain two "referents". One is called the key, which can be viewed as a "weak" referent in the style of Java reference types (SoftReference, WeakReference, PhantomReference), the other is called the value, which is also treated specially by GC. It's reachability is dependent on the reachability of the key. Ephemerons solve the problem seen for example in the java.util.WeakHashMap when a value in this map directly or indirectly refers to it's key. Such entry will never be purged as the value is always reachable from the WeakHashMap instance and through the value, it's key is reachable too. There are other places where Ephemerons could be used - for example in ClassValue API. Try googling for "java ephemeron" (without quotes) to find out other situations where Ephemerons would be beneficial. If one was to implement Ephemerons in Java, the main question he would be asking is how Ephemeron(s) were supposed to interact with existing Java reference types. Hayes only defines their behavior in isolation, but Java already has 4 "weak" reference types with different "strengths": SoftReference, WeakReference, FinalReference (for implementing finalization) and PhantomReference. In the prototype I choose to define Ephemerons as a new reference type with it's own "strength" for the key referent positioned between WeakReference and FinalReference. It would be possible to merge it with an existing reference type like WeakReference or position it between SoftReference and WeakReference or even entirely "on the top" of the strength list, but I think it would not be wise to position it below FinalReference or even PhantomReference. What's best is an open question. What is also not so obvious is how to define the reachability of the Ephemeron's value and it's interaction with existing reference types. I think I defined it soundly (see the reachability types in the javadoc of [4]). The reason for defining the reachability of the value to alternate between ephemeraly-reachable and weakly-reachable and not between ephemeraly-reachable and strongly-reachable, while theoretically possible, is the desire to have a separate phase of processing for each reachability strength, like it is done currently, and which doesn't affect the performance of processing of existing reference types. Having skimmed through the reference processing code in the VM for a couple of times, I thought it should not be too complicated to implement another type of "weak" reference. Encouraged by how little changes were needed to remove the sun.misc.Cleaner special type of reference [2], I began experimenting and came up with a prototype that seems to work [3]. Luckily most of the logic to process Reference objects in VM is encapsulated in the ReferenceProcessor class and this logic is invoked from various GC implementations of the hotspot. Changes needed are therefore mostly localized to this class. The webrev also contains recent changes from JDK-8143847 [2] that have not yet propagated to the jdk9-dev repo. When they do, I will prepare a rebased webrev without those changes. I'm publishing this prototype here to get the answer to the main question: Is there an interest to have Ephemeron reference type in Java? Given that there was interest, I would also like to initiate a discussion about: - What would be the most appropriate "strength" of reachabillity for Epehmeron referents (key and value) and the desired interactions with existing reachabilities. - The prototype itself. Since I'm new to this part of code (that's my 1st not so shallow dive into the VM), I surely must have missed places that should be changed. Although the prototype seems to work, I have not created extensive tests for the functionality and have not exposed it to all the various GC algorithms and situations yet. I could use some advise on how to exercise all GC algorithm combinations possible in the VM (what flags to use, for example). Regards, Peter [1] https://static.aminer.org/pdf/PDF/000/522/273/ephemerons_a_new_finalization_mechanism.pdf [2] https://bugs.openjdk.java.net/browse/JDK-8143847 [3] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/ [4] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.01/src/java.base/share/classes/java/lang/ref/Ephemeron.java.html P.S. To wet some appetite, with the above prototype (applied to current jdk9-dev), it is possible to run the following example: import java.lang.ref.Ephemeron; import java.util.ArrayList; import java.util.List; public class EphemeronTest { static class Key { final int i; Key(int i) { this.i = i; } @Override public String toString() { return "k" + i; } } static class Value { final Key key; Value(Key key) { this.key = key; } @Override public String toString() { return "v(" + key + ")"; } } static class Eph extends Ephemeron { public Eph(Key key, Value value) { super(key, value); } @Override public String toString() { return getKey() + "=" + getValue(); } } static void test(int size, boolean forwardChaining) throws Exception { System.out.println(); System.out.println((forwardChaining ? "Forward" : "Backward") + " chaining of value->key ..."); System.out.println(); List ephs = new ArrayList<>(size); Key k1 = new Key(1); { Key kp = k1; for (int i = 2; i <= size; i++) { Key ki = new Key(i); ephs.add( forwardChaining ? new Eph(kp, new Value(ki)) : new Eph(ki, new Value(kp)) ); kp = ki; } ephs.add( forwardChaining ? new Eph(kp, new Value(k1)) : new Eph(k1, new Value(kp)) ); kp = null; } System.gc(); System.out.println("1: " + ephs); k1.getClass(); // reachabilityFence k1 = null; System.gc(); System.out.println("2: " + ephs); } public static void main(String[] args) throws Exception { int size = (args.length < 1) ? 5 : Math.max(2, Integer.parseInt(args[0])); test(size, true); test(size, false); } } Which prints: Forward chaining of value->key ... 1: [k1=v(k2), k2=v(k3), k3=v(k4), k4=v(k5), k5=v(k1)] 2: [null=null, null=null, null=null, null=null, null=null] Backward chaining of value->key ... 1: [k2=v(k1), k3=v(k2), k4=v(k3), k5=v(k4), k1=v(k5)] 2: [null=null, null=null, null=null, null=null, null=null] If you compile the JDK with --enable-debug and use the following VM switches when running: -XX:+PrintGCDetails -XX:+TraceReferenceGC, you can also observe the inner workings of the reference processing, including Ephemerons. From masayoshi.okutsu at oracle.com Mon Jan 18 06:47:36 2016 From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu) Date: Mon, 18 Jan 2016 15:47:36 +0900 Subject: RFR: JDK-8144988: Unexpected timezone returned after parsing a date In-Reply-To: <144a4088-0d5e-4c7c-b249-eb62198f0a01@default> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> <56985DCD.4070706@oracle.com> <144a4088-0d5e-4c7c-b249-eb62198f0a01@default> Message-ID: <569C8A88.1070907@oracle.com> Looks good to me. Masayoshi On 1/16/2016 12:31 AM, Ramanand Patil wrote: > Hi Masayoshi, > > Thank you for pointing that out. I have removed line 29 from the test. > > Please review the updated Webrev: http://cr.openjdk.java.net/~rpatil/8141243/webrev.01/ > > > Regards, > Ramanand. > > -----Original Message----- > From: Masayoshi Okutsu > Sent: Friday, January 15, 2016 8:18 AM > To: Ramanand Patil; i18n-dev at openjdk.java.net > Cc: core-libs-dev at openjdk.java.net > Subject: Re: RFR: JDK-8144988: Unexpected timezone returned after parsing a date > > Hi Ramanand, > > test/java/text/Format/DateFormat/Bug8141243.java: > > 28 * @run main Bug8141243 > 29 * @run main/othervm -Djava.locale.providers=COMPAT Bug8141243 > > "COMPAT" is a new name of "JRE" in JDK 9. It's not supported in 8u. I think COMPAT is slightly ignored and that it becomes the same thing as line 28. Line 29 should be removed. > > Otherwise, the fix looks OK to me. > > Masayoshi > > On 1/14/2016 9:21 PM, Ramanand Patil wrote: >> Hi all, >> >> Please review the fix for bug: >> https://bugs.openjdk.java.net/browse/JDK-8144988 >> >> Webrev: http://cr.openjdk.java.net/~rpatil/8141243/webrev.00 >> >> This is basically a backport of JDK9 bug: >> https://bugs.openjdk.java.net/browse/JDK-8141243 >> >> JDK9 changeset(for reference): >> http://hg.openjdk.java.net/jdk9/dev/jdk/rev/a1aa2671f281 >> >> Reason for the review request is because of: >> i) Changes present in ResourceBundleGenerator.java file. >> ii) The patch from JDK9 does not automatically apply as is after using unshuffle_patch script. Few paths are adjusted as per the jdk8. >> >> Since CLDR became the default locale data in JDK9 leading incompatible behavior with prior releases, the relevant code in ResourceBundleGenerator is also backported in this patch. >> Even though JDK8 has CLDR locale provider disabled by default, the code change is done to be on safer side for cases where CLDR may be supporting "UTC" ID in the future. >> >> >> Regards, >> Ramanand. From miroslav.kos at oracle.com Mon Jan 18 10:24:42 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Mon, 18 Jan 2016 11:24:42 +0100 Subject: RFR [9]: 8146086 Publishing two webservices on same port fails with "java.net.BindException: Address already in use" In-Reply-To: <5693BF25.1070207@oracle.com> References: <567AAC48.6010008@oracle.com> <56938E3B.7010205@oracle.com> <56939007.7070305@oracle.com> <5693BF25.1070207@oracle.com> Message-ID: <569CBD6A.3080109@oracle.com> Hi Chris, are you ok with the change? Thanks Miran On 11/01/16 15:41, Miroslav Kos wrote: > Hi Chris, > good catch - thanks! Simple fix here: > http://cr.openjdk.java.net/~mkos/8146086/jdk.01/ > > M. > > > On 11/01/16 12:20, Chris Hegarty wrote: >> Miran, >> >> On 11/01/16 11:12, Miroslav Kos wrote: >>> Ping ... anybody? >>> >>> On 23/12/15 15:14, Miroslav Kos wrote: >>>> Hi everybody, >>>> >>>> I'd like to ask for review of . This simple patch allows >>>> publishing more webeservices on same port. The fix has been submitted >>>> by Kubota Yuji, it is already integrated in standalone JAX-WS repo, >>>> regression test added. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8146086 >>>> jaxws patch: http://cr.openjdk.java.net/~mkos/8146086/jaxws.00/ >> >> The change ok. >> >>>> test: http://cr.openjdk.java.net/~mkos/8146086/jdk.00/ >> >> The test uses a hardcoded port number. Is there a pattern in this >> area for enable the test to start up on an ephemeral port, and use >> that for the subsequent request? >> >> -Chris. >> >>>> Thanks >>>> Miran > From chris.hegarty at oracle.com Mon Jan 18 11:03:52 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 18 Jan 2016 11:03:52 +0000 Subject: RFR [9]: 8146086 Publishing two webservices on same port fails with "java.net.BindException: Address already in use" In-Reply-To: <569CBD6A.3080109@oracle.com> References: <567AAC48.6010008@oracle.com> <56938E3B.7010205@oracle.com> <56939007.7070305@oracle.com> <5693BF25.1070207@oracle.com> <569CBD6A.3080109@oracle.com> Message-ID: <569CC698.8030302@oracle.com> On 18/01/16 10:24, Miroslav Kos wrote: > Hi Chris, > are you ok with the change? Yes. Reviewed. -Chris. > Thanks > Miran > > > On 11/01/16 15:41, Miroslav Kos wrote: >> Hi Chris, >> good catch - thanks! Simple fix here: >> http://cr.openjdk.java.net/~mkos/8146086/jdk.01/ >> >> M. >> >> >> On 11/01/16 12:20, Chris Hegarty wrote: >>> Miran, >>> >>> On 11/01/16 11:12, Miroslav Kos wrote: >>>> Ping ... anybody? >>>> >>>> On 23/12/15 15:14, Miroslav Kos wrote: >>>>> Hi everybody, >>>>> >>>>> I'd like to ask for review of . This simple patch allows >>>>> publishing more webeservices on same port. The fix has been submitted >>>>> by Kubota Yuji, it is already integrated in standalone JAX-WS repo, >>>>> regression test added. >>>>> >>>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8146086 >>>>> jaxws patch: http://cr.openjdk.java.net/~mkos/8146086/jaxws.00/ >>> >>> The change ok. >>> >>>>> test: http://cr.openjdk.java.net/~mkos/8146086/jdk.00/ >>> >>> The test uses a hardcoded port number. Is there a pattern in this >>> area for enable the test to start up on an ephemeral port, and use >>> that for the subsequent request? >>> >>> -Chris. >>> >>>>> Thanks >>>>> Miran >> > From paul.sandoz at oracle.com Mon Jan 18 11:22:50 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 18 Jan 2016 12:22:50 +0100 Subject: RFR 8146458 Improve exception reporting for Objects.checkIndex/checkFromToIndex/checkFromIndexSize In-Reply-To: <0B0A9076-D80E-49A9-AD8D-DB538901EE81@oracle.com> References: <40431638-BC6A-44E2-8A5E-8F0B180AA924@oracle.com> <56997622.8000505@oracle.com> <0B0A9076-D80E-49A9-AD8D-DB538901EE81@oracle.com> Message-ID: Thanks for the feedback. Updated: http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/webrev/ http://cr.openjdk.java.net/~psandoz/jdk9/JDK-8146458-checkIndex-exception-reporting/specdiff/overview-summary.html Changes: - refer to ?exception factory function? - renamed rangeCheckExceptionMapper to outOfBoundsExceptionFactory. Tweaked the documentation and added an api note for clarity. - tests for String/ArrayIOOBE, and uniform checking of expected exception message in all cases > On 16 Jan 2016, at 08:36, John Rose wrote: > > On Jan 15, 2016, at 2:43 PM, Stuart Marks wrote: >> >> >> I think, in order to avoid introducing new functional interfaces, this information is funneled into a single varargs call, which has to use Integer boxes, because it then wraps its args into a List (thanks for using the new List factory!), which then has to unpack the list and check its length against the number of arguments for the particular message kind. > > And that's before String.format does a bunch more steps of the same kind, including boxing and varargs. And then the exception creation itself walks the entire thread stack. Micro-optimization is hopeless. > > But it doesn't matter because it's all on the slow path. > Yes. FWIW the implementation also avoids any varargs call for the main check methods to keep the method size small (we could use @ForceInline instead, but i would prefer to use that for cases where it is really needed). >> I understand that this is the "slow" exception-throwing path, but my sense is still that this ought to be made simpler. Perhaps the scope of what it's trying to accomplished should be reduced, if that helps things. > > Simpler would be good but I don't see it any simpler, yet. > Me neither, any simpler and the generality is lost. Arguably there is actually quite a simple rule underlying this: the arguments to the factory function take on a form that is very similar to those required for a reflective look up and invocation: Objects.class.getMethod(checkKind, List.nCopies(args.size(), int.class).toArray()). invoke(null, args.toArray()) Paul. > Do give it some thought! > From martinrb at google.com Mon Jan 18 16:52:15 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 18 Jan 2016 08:52:15 -0800 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <56962632.7080404@oracle.com> References: <56962632.7080404@oracle.com> Message-ID: Historical note - I never liked having a reaper thread for each subprocess, but no other reliable implementation is known. Given the potential for many reaper threads, I at least wanted to keep the memory waste low. On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu wrote: > + Thread t = null; > + if > (Boolean.getBoolean("processReaperUseDefaultStackSize")) { > + t = new Thread(systemThreadGroup, grimReaper, > "process reaper"); > + } else { > + // Our thread stack requirement is quite modest. > + t = new Thread(systemThreadGroup, grimReaper, > "process reaper", 32768); > + } If we do end up using this strategy, cleaner would be to use https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- stackSize - the desired stack size for the new thread, or zero to indicate that this parameter is to be ignored. From xueming.shen at oracle.com Tue Jan 19 07:52:21 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 18 Jan 2016 23:52:21 -0800 Subject: RFR JDK-8147531,To add named character construct \N{...} to support Unicode name property Message-ID: <569DEB35.7020006@oracle.com> Hi, Please help review the change to add \N support in regex. Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 webrev: http://cr.openjdk.java.net/~sherman/8147531/webrev This is one of the items we were planning to address via JEP111 http://openjdk.java.net/jeps/111 https://bugs.openjdk.java.net/browse/JDK-8046101 Some of the constructs had been added already in early release. I'm planning to address the rest as individual rfe separately. Thanks, Sherman From kevin.walls at oracle.com Tue Jan 19 11:53:01 2016 From: kevin.walls at oracle.com (Kevin Walls) Date: Tue, 19 Jan 2016 11:53:01 +0000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: References: <56962632.7080404@oracle.com> Message-ID: <569E239D.1020701@oracle.com> | Hi Cheleswer, I think Martin is suggesting something like: | // Use a modest stack size, unless requested otherwise: long stackSize = Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; Thread t = new Thread(systemThreadGroup, grimReaper, "process reaper", stackSize); ||| If that tests OK for you, it may be the way we can go ahead with the point fix for this. Regards Kevin | On 18/01/2016 16:52, Martin Buchholz wrote: > Historical note - I never liked having a reaper thread for each > subprocess, but no other reliable implementation is known. Given the > potential for many reaper threads, I at least wanted to keep the > memory waste low. > > On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu > wrote: > >> + Thread t = null; >> + if >> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >> + t = new Thread(systemThreadGroup, grimReaper, >> "process reaper"); >> + } else { >> + // Our thread stack requirement is quite modest. >> + t = new Thread(systemThreadGroup, grimReaper, >> "process reaper", 32768); >> + } > If we do end up using this strategy, cleaner would be to use > https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- > stackSize - the desired stack size for the new thread, or zero to > indicate that this parameter is to be ignored. From david.holmes at oracle.com Tue Jan 19 12:10:10 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 19 Jan 2016 22:10:10 +1000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569E239D.1020701@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> Message-ID: <569E27A2.4070207@oracle.com> On 19/01/2016 9:53 PM, Kevin Walls wrote: > | > Hi Cheleswer, I think Martin is suggesting something like: > | > > // Use a modest stack size, unless requested otherwise: > long stackSize = Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; Someone from core-libs needs to chime on what the appropriate namespace for such a property would be. David ----- > Thread t = new Thread(systemThreadGroup, grimReaper, "process reaper", stackSize); > > ||| > If that tests OK for you, it may be the way we can go ahead with the > point fix for this. > > Regards > Kevin > | > On 18/01/2016 16:52, Martin Buchholz wrote: >> Historical note - I never liked having a reaper thread for each >> subprocess, but no other reliable implementation is known. Given the >> potential for many reaper threads, I at least wanted to keep the >> memory waste low. >> >> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >> wrote: >> >>> + Thread t = null; >>> + if >>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>> + t = new Thread(systemThreadGroup, grimReaper, >>> "process reaper"); >>> + } else { >>> + // Our thread stack requirement is quite modest. >>> + t = new Thread(systemThreadGroup, grimReaper, >>> "process reaper", 32768); >>> + } >> If we do end up using this strategy, cleaner would be to use >> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >> stackSize - the desired stack size for the new thread, or zero to >> indicate that this parameter is to be ignored. > From roger.riggs at oracle.com Tue Jan 19 14:50:25 2016 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 19 Jan 2016 09:50:25 -0500 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569E27A2.4070207@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> Message-ID: <569E4D31.1010006@oracle.com> Hi, How about "jdk.process.reaperStackSize". It will need a CCC . Roger On 1/19/16 7:10 AM, David Holmes wrote: > On 19/01/2016 9:53 PM, Kevin Walls wrote: >> | >> Hi Cheleswer, I think Martin is suggesting something like: >> | >> >> // Use a modest stack size, unless requested otherwise: >> long stackSize = >> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; > > Someone from core-libs needs to chime on what the appropriate > namespace for such a property would be. > > David > ----- > >> Thread t = new Thread(systemThreadGroup, grimReaper, "process >> reaper", stackSize); >> >> ||| >> If that tests OK for you, it may be the way we can go ahead with the >> point fix for this. >> >> Regards >> Kevin >> | >> On 18/01/2016 16:52, Martin Buchholz wrote: >>> Historical note - I never liked having a reaper thread for each >>> subprocess, but no other reliable implementation is known. Given the >>> potential for many reaper threads, I at least wanted to keep the >>> memory waste low. >>> >>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>> wrote: >>> >>>> + Thread t = null; >>>> + if >>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>> + t = new Thread(systemThreadGroup, grimReaper, >>>> "process reaper"); >>>> + } else { >>>> + // Our thread stack requirement is quite >>>> modest. >>>> + t = new Thread(systemThreadGroup, grimReaper, >>>> "process reaper", 32768); >>>> + } >>> If we do end up using this strategy, cleaner would be to use >>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>> >>> stackSize - the desired stack size for the new thread, or zero to >>> indicate that this parameter is to be ignored. >> From martinrb at google.com Tue Jan 19 17:24:24 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 19 Jan 2016 09:24:24 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) Message-ID: Hi Stuart et al, Please review my fix for this corner case bug, including more importantly some performance improvements. https://bugs.openjdk.java.net/browse/JDK-8146568 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/ArrayList-grow/ From chris.hegarty at oracle.com Tue Jan 19 17:31:38 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 19 Jan 2016 17:31:38 +0000 Subject: GNU-style options for the Jar Tool In-Reply-To: <433503D4-FF76-4F41-BF4F-2853F57A692A@oracle.com> References: <433503D4-FF76-4F41-BF4F-2853F57A692A@oracle.com> Message-ID: Please reply-to core-libs-dev at openjdk.java.net, which is more appropriate for discussion on Jar Tool. -Chris. On 19 Jan 2016, at 16:54, Chris Hegarty wrote: > GNU-style options for Jar Tool > > Prompted by the impending addition of new options for both multi-release jars > and jigsaw, it became immediately obvious that we were running out of single > characters that could be used to represent these new options. I'd like to > propose an update to the Jar Tool CLI to accept GNU-style options ( all short > forms being compatible with the existing current options ), so that we can > evolve the CLI in a reasonable fashion going forward. > > For example: > # Create an archive called classes.jar with two class files: > jar --create --file classes.jar Foo.class Bar.class > # Create an archive using an existing manifest, with all the files in foo/: > jar --create --file classes.jar --manifest mymanifest -C foo/ . > > The complete output of '--help' can be seen below [*] > > Short forms are the same as existing current options. > > It was suggested over on jigsaw-dev that the help output could drop the > compatibility interface. Only printing it if '--help:compatility', or similar. > > http://cr.openjdk.java.net/~chegar/jartool.options/ > > -Chris > > P.S. jigsaw and multi-release jars will add the current list of exiting > options. > > [*] ( there may be some formatting issue in email ) > $ ./jdk/bin/jar --help > Usage: jar [OPTION...] [-C dir] files ... > jar creates an archive for classes and resources, and can manipulate or > restore individual classes or resources from an archive. > > Examples: > # Create an archive called classes.jar with two class files: > jar --create --file classes.jar Foo.class Bar.class > # Create an archive using an existing manifest, with all the files in foo/: > jar --create --file classes.jar --manifest mymanifest -C foo/ . > > > Main operation mode: > -c, --create Create the archive > -i, --generate-index=FILE Generate index information for the specified jar > archives > -t, --list List the table of contents for the archive > -u, --update Update an existing jar archive > -x, --extract Extract named (or all) files from the archive > > Operation modifiers valid in any mode: > > -C DIR, Change to the specified directory and include the > following file > -f, --file=FILE The archive file name > -v, --verbose Generate verbose output on standard output > > Operation modifiers valid only in create mode: > -n, --pack200 Perform Pack200 normalization after creating a > new archive > > Operation modifiers valid only in create and update mode: > -e, --main-class=CLASSNAME The application entry point for stand-alone > applications bundled into a modular, or executable, > jar archive > -m, --manifest=FILE Include the manifest information from the given > manifest file > -M, --no-manifest Do not create a manifest file for the entries > > Operation modifiers valid only in create, update, and generate-index mode: > -0, --no-compress Store only; use no ZIP compression > > Other options: > -?, --help give this help list > --version print program version > > Mandatory or optional arguments to long options are also mandatory or optional > for any corresponding short options. > > > Compatibility Interface: > > Usage: jar {ctxui}[vfmn0PMe] [jar-file] [manifest-file] [entry-point] [-C dir] files ... > Options: > -c create new archive > -t list table of contents for archive > -x extract named (or all) files from archive > -u update existing archive > -v generate verbose output on standard output > -f specify archive file name > -m include manifest information from specified manifest file > -n perform Pack200 normalization after creating a new archive > -e specify application entry point for stand-alone application > bundled into an executable jar file > -0 store only; use no ZIP compression > -P preserve leading '/' (absolute path) and ".." (parent directory) components from file names > -M do not create a manifest file for the entries > -i generate index information for the specified jar files > -C change to the specified directory and include the following file > If any file is a directory then it is processed recursively. > The manifest file name, the archive file name and the entry point name are > specified in the same order as the 'm', 'f' and 'e' flags. > > Example 1: to archive two class files into an archive called classes.jar: > jar cvf classes.jar Foo.class Bar.class > Example 2: use an existing manifest file 'mymanifest' and archive all the > files in the foo/ directory into 'classes.jar': > jar cvfm classes.jar mymanifest -C foo/ . > > From lance.andersen at oracle.com Tue Jan 19 19:32:47 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 19 Jan 2016 14:32:47 -0500 Subject: RFR 8146803: Update to the sharding API Message-ID: <2F31E1E6-E5CA-4693-9604-835A3A41C692@oracle.com> Hi, Please review the attached webrev which addresses some changes we needed to make to the sharding API. The CCC has been approved. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8146803/webrev.00/ Here is a brief description of the changes -ConnectionPooledDataSource needed to add a PooledConnectionBuilder as like DataSource and XADataSource this will create a physical connection and the ShardingKey will be needed - PooledConnectionBuilder - is used to create a PooledConnection ->physical connection so it needed access to the ShardingKey - CommonDataSource added createShardingKeyBuilder making it available to DataSource, XADataSource and ConnectionPoolDatasource. - DataSource/XADataSource removed createShardingKeyBuilder as it now is on CommonDataSource which they both extend - XAConnection removed the setShardingKey methods. Having the methods on Connection should be adequate as the ShardingKey is needed for the Connection which they already exist on. -XAConnectionBuilder had a minor typo in the javadoc Best Lance Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From martinrb at google.com Tue Jan 19 19:43:38 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 19 Jan 2016 11:43:38 -0800 Subject: RFR JDK-8147531,To add named character construct \N{...} to support Unicode name property In-Reply-To: <569DEB35.7020006@oracle.com> References: <569DEB35.7020006@oracle.com> Message-ID: Many years ago I considered implementing this cool feature. I thought that few would find it worth the cost - it would be hard to keep the cost low if this feature is used only rarely. You might want an expiring cache of character name mappings, and the JDK doesn't have such a thing yet. (I haven't actually reviewed the implementation) On Mon, Jan 18, 2016 at 11:52 PM, Xueming Shen wrote: > Hi, > > Please help review the change to add \N support in regex. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 > webrev: http://cr.openjdk.java.net/~sherman/8147531/webrev > > This is one of the items we were planning to address via JEP111 > http://openjdk.java.net/jeps/111 > https://bugs.openjdk.java.net/browse/JDK-8046101 > > Some of the constructs had been added already in early release. I'm > planning to address the rest as individual rfe separately. > > Thanks, > Sherman From martinrb at google.com Tue Jan 19 20:06:33 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 19 Jan 2016 12:06:33 -0800 Subject: GNU-style options for the Jar Tool In-Reply-To: <433503D4-FF76-4F41-BF4F-2853F57A692A@oracle.com> References: <433503D4-FF76-4F41-BF4F-2853F57A692A@oracle.com> Message-ID: A change such as this is reasonable and welcome. The compatibility interface will need to be maintained forever, and will continue to be the preferred interface for casual human use, the same way that humans stubbornly prefer the non-standard "head -1". So I would not drop the old usage text. Options such as -n are obscure and likely to only be used in scripts, so maybe having a short form for '-n' is a bad idea? On Tue, Jan 19, 2016 at 8:54 AM, Chris Hegarty wrote: > GNU-style options for Jar Tool > > Prompted by the impending addition of new options for both multi-release jars > and jigsaw, it became immediately obvious that we were running out of single > characters that could be used to represent these new options. I'd like to > propose an update to the Jar Tool CLI to accept GNU-style options ( all short > forms being compatible with the existing current options ), so that we can > evolve the CLI in a reasonable fashion going forward. > > For example: > # Create an archive called classes.jar with two class files: > jar --create --file classes.jar Foo.class Bar.class > # Create an archive using an existing manifest, with all the files in foo/: > jar --create --file classes.jar --manifest mymanifest -C foo/ . > > The complete output of '--help' can be seen below [*] > > Short forms are the same as existing current options. > > It was suggested over on jigsaw-dev that the help output could drop the > compatibility interface. Only printing it if '--help:compatility', or similar. > > http://cr.openjdk.java.net/~chegar/jartool.options/ > > -Chris > > P.S. jigsaw and multi-release jars will add the current list of exiting > options. > > [*] ( there may be some formatting issue in email ) > $ ./jdk/bin/jar --help > Usage: jar [OPTION...] [-C dir] files ... > jar creates an archive for classes and resources, and can manipulate or > restore individual classes or resources from an archive. > > Examples: > # Create an archive called classes.jar with two class files: > jar --create --file classes.jar Foo.class Bar.class > # Create an archive using an existing manifest, with all the files in foo/: > jar --create --file classes.jar --manifest mymanifest -C foo/ . > > > Main operation mode: > -c, --create Create the archive > -i, --generate-index=FILE Generate index information for the specified jar > archives > -t, --list List the table of contents for the archive > -u, --update Update an existing jar archive > -x, --extract Extract named (or all) files from the archive > > Operation modifiers valid in any mode: > > -C DIR, Change to the specified directory and include the > following file > -f, --file=FILE The archive file name > -v, --verbose Generate verbose output on standard output > > Operation modifiers valid only in create mode: > -n, --pack200 Perform Pack200 normalization after creating a > new archive > > Operation modifiers valid only in create and update mode: > -e, --main-class=CLASSNAME The application entry point for stand-alone > applications bundled into a modular, or executable, > jar archive > -m, --manifest=FILE Include the manifest information from the given > manifest file > -M, --no-manifest Do not create a manifest file for the entries > > Operation modifiers valid only in create, update, and generate-index mode: > -0, --no-compress Store only; use no ZIP compression > > Other options: > -?, --help give this help list > --version print program version > > Mandatory or optional arguments to long options are also mandatory or optional > for any corresponding short options. > > > Compatibility Interface: > > Usage: jar {ctxui}[vfmn0PMe] [jar-file] [manifest-file] [entry-point] [-C dir] files ... > Options: > -c create new archive > -t list table of contents for archive > -x extract named (or all) files from archive > -u update existing archive > -v generate verbose output on standard output > -f specify archive file name > -m include manifest information from specified manifest file > -n perform Pack200 normalization after creating a new archive > -e specify application entry point for stand-alone application > bundled into an executable jar file > -0 store only; use no ZIP compression > -P preserve leading '/' (absolute path) and ".." (parent directory) components from file names > -M do not create a manifest file for the entries > -i generate index information for the specified jar files > -C change to the specified directory and include the following file > If any file is a directory then it is processed recursively. > The manifest file name, the archive file name and the entry point name are > specified in the same order as the 'm', 'f' and 'e' flags. > > Example 1: to archive two class files into an archive called classes.jar: > jar cvf classes.jar Foo.class Bar.class > Example 2: use an existing manifest file 'mymanifest' and archive all the > files in the foo/ directory into 'classes.jar': > jar cvfm classes.jar mymanifest -C foo/ . > > From huizhe.wang at oracle.com Tue Jan 19 22:30:48 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 19 Jan 2016 14:30:48 -0800 Subject: RFR 8146803: Update to the sharding API In-Reply-To: <2F31E1E6-E5CA-4693-9604-835A3A41C692@oracle.com> References: <2F31E1E6-E5CA-4693-9604-835A3A41C692@oracle.com> Message-ID: <569EB918.1020603@oracle.com> Looks good overall. A few minor things and typos: CommonDataSource: Lic header: 2013 --> 2016; createShardingKeyBuilder: "@since 1.9" --> since 9 ConnectionPoolDataSource: Lic header: 2013 --> 2016; createShardingKeyBuilder: "@since 1.9" --> since 9; {@code createPooledConnectionBuilder} --> {@code PooledConnectionBuilder} @see createPooledConnectionBuilder --> @see PooledConnectionBuilder DataSource: Lic header: 2013 --> 2016; 122 * @since 1.9 --> since 9 123 * @see createConnectionBuilder --> ConnectionBuilder XAConnection: Lic header: 2015 --> 2016; XADataSource: Lic header: 2015 --> 2016; createXAConnectionBuilder: "@since 1.9" --> since 9 @return The ConnectionBuilder --> XAConnectionBuilder PooledConnectionBuilder: Lic header: 2015 --> 2016; class desc: @since 1.9 --> since 9 build() method: @return The built object --> a {@code PooledConnection} instance? Best, Joe On 1/19/2016 11:32 AM, Lance Andersen wrote: > Hi, > > Please review the attached webrev which addresses some changes we needed to make to the sharding API. The CCC has been approved. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8146803/webrev.00/ > > > Here is a brief description of the changes > > -ConnectionPooledDataSource needed to add a PooledConnectionBuilder as like DataSource and XADataSource this will create a physical connection and the ShardingKey will be needed > - PooledConnectionBuilder - is used to create a PooledConnection ->physical connection so it needed access to the ShardingKey > - CommonDataSource added createShardingKeyBuilder making it available to DataSource, XADataSource and ConnectionPoolDatasource. > - DataSource/XADataSource removed createShardingKeyBuilder as it now is on CommonDataSource which they both extend > - XAConnection removed the setShardingKey methods. Having the methods on Connection should be adequate as the ShardingKey is needed for the Connection which they already exist on. > -XAConnectionBuilder had a minor typo in the javadoc > > > Best > Lance > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From david.holmes at oracle.com Tue Jan 19 22:39:05 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jan 2016 08:39:05 +1000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569E4D31.1010006@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> Message-ID: <569EBB09.7030800@oracle.com> On 20/01/2016 12:50 AM, Roger Riggs wrote: > Hi, > > How about "jdk.process.reaperStackSize". Based on some internal discussion that seems appropriate to me ... > It will need a CCC . ... and if not then the CCC will show the way. :) Thanks, David > Roger > > > On 1/19/16 7:10 AM, David Holmes wrote: >> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>> | >>> Hi Cheleswer, I think Martin is suggesting something like: >>> | >>> >>> // Use a modest stack size, unless requested otherwise: >>> long stackSize = >>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >> >> Someone from core-libs needs to chime on what the appropriate >> namespace for such a property would be. >> >> David >> ----- >> >>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>> reaper", stackSize); >>> >>> ||| >>> If that tests OK for you, it may be the way we can go ahead with the >>> point fix for this. >>> >>> Regards >>> Kevin >>> | >>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>> Historical note - I never liked having a reaper thread for each >>>> subprocess, but no other reliable implementation is known. Given the >>>> potential for many reaper threads, I at least wanted to keep the >>>> memory waste low. >>>> >>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>> wrote: >>>> >>>>> + Thread t = null; >>>>> + if >>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>> "process reaper"); >>>>> + } else { >>>>> + // Our thread stack requirement is quite >>>>> modest. >>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>> "process reaper", 32768); >>>>> + } >>>> If we do end up using this strategy, cleaner would be to use >>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>> >>>> stackSize - the desired stack size for the new thread, or zero to >>>> indicate that this parameter is to be ignored. >>> > From lance.andersen at oracle.com Wed Jan 20 00:02:08 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 19 Jan 2016 19:02:08 -0500 Subject: RFR 8146803: Update to the sharding API In-Reply-To: <569EB918.1020603@oracle.com> References: <2F31E1E6-E5CA-4693-9604-835A3A41C692@oracle.com> <569EB918.1020603@oracle.com> Message-ID: Thank you Joe. I had made those changes earlier and will sanity check before I push? Best Lance On Jan 19, 2016, at 5:30 PM, huizhe wang wrote: > Looks good overall. > > A few minor things and typos: > CommonDataSource: > Lic header: 2013 --> 2016; createShardingKeyBuilder: "@since 1.9" --> since 9 > > ConnectionPoolDataSource: > Lic header: 2013 --> 2016; > > createShardingKeyBuilder: > "@since 1.9" --> since 9; > {@code createPooledConnectionBuilder} --> {@code PooledConnectionBuilder} > @see createPooledConnectionBuilder --> @see PooledConnectionBuilder > > DataSource: > Lic header: 2013 --> 2016; > 122 * @since 1.9 --> since 9 > 123 * @see createConnectionBuilder --> ConnectionBuilder > > XAConnection: > Lic header: 2015 --> 2016; > > XADataSource: > Lic header: 2015 --> 2016; createXAConnectionBuilder: "@since 1.9" --> since 9 > @return The ConnectionBuilder --> XAConnectionBuilder > > PooledConnectionBuilder: > Lic header: 2015 --> 2016; class desc: @since 1.9 --> since 9 > > build() method: @return The built object > --> a {@code PooledConnection} instance? > > > Best, > Joe > > On 1/19/2016 11:32 AM, Lance Andersen wrote: >> Hi, >> >> Please review the attached webrev which addresses some changes we needed to make to the sharding API. The CCC has been approved. The webrev can be found at: http://cr.openjdk.java.net/~lancea/8146803/webrev.00/ >> >> >> Here is a brief description of the changes >> >> -ConnectionPooledDataSource needed to add a PooledConnectionBuilder as like DataSource and XADataSource this will create a physical connection and the ShardingKey will be needed >> - PooledConnectionBuilder - is used to create a PooledConnection ->physical connection so it needed access to the ShardingKey >> - CommonDataSource added createShardingKeyBuilder making it available to DataSource, XADataSource and ConnectionPoolDatasource. >> - DataSource/XADataSource removed createShardingKeyBuilder as it now is on CommonDataSource which they both extend >> - XAConnection removed the setShardingKey methods. Having the methods on Connection should be adequate as the ShardingKey is needed for the Connection which they already exist on. >> -XAConnectionBuilder had a minor typo in the javadoc >> >> >> Best >> Lance >> >> >> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 >> Oracle Java Engineering >> 1 Network Drive >> Burlington, MA 01803 >> Lance.Andersen at oracle.com >> >> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From david.holmes at oracle.com Wed Jan 20 05:05:14 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 20 Jan 2016 15:05:14 +1000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569EBB09.7030800@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> <569EBB09.7030800@oracle.com> Message-ID: <569F158A.5060309@oracle.com> On 20/01/2016 8:39 AM, David Holmes wrote: > On 20/01/2016 12:50 AM, Roger Riggs wrote: >> Hi, >> >> How about "jdk.process.reaperStackSize". > > Based on some internal discussion that seems appropriate to me ... Except of course that the property is just a boolean not a size. :) So jdk.process.reaperUsesdefaultStackSize seems appropriate. David >> It will need a CCC . > > ... and if not then the CCC will show the way. :) > > Thanks, > David > >> Roger >> >> >> On 1/19/16 7:10 AM, David Holmes wrote: >>> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>>> | >>>> Hi Cheleswer, I think Martin is suggesting something like: >>>> | >>>> >>>> // Use a modest stack size, unless requested otherwise: >>>> long stackSize = >>>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >>> >>> Someone from core-libs needs to chime on what the appropriate >>> namespace for such a property would be. >>> >>> David >>> ----- >>> >>>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>>> reaper", stackSize); >>>> >>>> ||| >>>> If that tests OK for you, it may be the way we can go ahead with the >>>> point fix for this. >>>> >>>> Regards >>>> Kevin >>>> | >>>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>>> Historical note - I never liked having a reaper thread for each >>>>> subprocess, but no other reliable implementation is known. Given the >>>>> potential for many reaper threads, I at least wanted to keep the >>>>> memory waste low. >>>>> >>>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>>> wrote: >>>>> >>>>>> + Thread t = null; >>>>>> + if >>>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>>> "process reaper"); >>>>>> + } else { >>>>>> + // Our thread stack requirement is quite >>>>>> modest. >>>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>>> "process reaper", 32768); >>>>>> + } >>>>> If we do end up using this strategy, cleaner would be to use >>>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>>> >>>>> >>>>> stackSize - the desired stack size for the new thread, or zero to >>>>> indicate that this parameter is to be ignored. >>>> >> From chris.hegarty at oracle.com Wed Jan 20 06:36:57 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Jan 2016 06:36:57 +0000 Subject: RFR 8065076/9, test/java/net/SocketPermission/SocketPermissionTest.java failed intermittently In-Reply-To: <56973B1A.30000@oracle.com> References: <568F74F7.70100@oracle.com> <56973B1A.30000@oracle.com> Message-ID: Felix, On 14 Jan 2016, at 06:07, Felix Yang wrote: > Hi all, > please review the fix for test/java/net/SocketPermission/SocketPermissionTest.java, which fails frequently with "java.net.BindException: Address already in use". > > Bug: https://bugs.openjdk.java.net/browse/JDK-8065076 > Webrev: http://cr.openjdk.java.net/~xiaofeya/8065076/webrev.00 My preference is to avoid getFreePort. It is problematic and I believe just obfuscates the intermittent failures further. In many of the test scenarios the ?listening? socket can be created before the specific access control context and associated permission are created. I?ll see if I can get some time to try this out. -Chris. > The test uses getFreePort() test util library to verify permission on specific "host:port". This fix added retries to avoid possible binding failures: > 1. A port may be still occupied for a short moment after socket close. > 2. Another application is possible to occupy the same port during the short time window of open-close-reuse. This has been observed in experimental test runs. > > Thanks, > Felix > > From shilpi.rastogi at oracle.com Wed Jan 20 12:42:24 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Wed, 20 Jan 2016 18:12:24 +0530 Subject: [9u-dev] 8138884:MethodHandles.Lookup.findVirtual() Javadoc fails to consider private interface methods Message-ID: <569F80B0.3040103@oracle.com> Hi All, Please review my fix related to java doc. MethodHandles.Lookup.findVirtual() Javadoc fails to consider private methods of an interface. MethodHandles.Lookup.findVirtual() throws IllegalAccessException if the method MH is being created for is private and declared in an interface. Bug link- https://bugs.openjdk.java.net/browse/JDK-8138884 Webrev link- http://cr.openjdk.java.net/~srastogi/8138884/webrev.00/ Thanks, Shilpi From shilpi.rastogi at oracle.com Wed Jan 20 12:42:30 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Wed, 20 Jan 2016 18:12:30 +0530 Subject: [9u-dev] 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods Message-ID: <569F80B6.7080702@oracle.com> Hi All, Please review my fix related to java doc. MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods- MethodHandles.Lookup.findSpecial() throws IllegalAccessException if the target method is static. Bug link- https://bugs.openjdk.java.net/browse/JDK-8138578 Webrev link- http://cr.openjdk.java.net/~srastogi/8138578/webrev.00/ Thnaks, Shilpi From Roger.Riggs at Oracle.com Wed Jan 20 14:09:25 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 20 Jan 2016 09:09:25 -0500 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569F158A.5060309@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> <569EBB09.7030800@oracle.com> <569F158A.5060309@oracle.com> Message-ID: <569F9515.5010506@Oracle.com> Hi David, I read an old description; I was expecting the value of the property to be the stack size for the reaper. That would allow more control and less waste when it needed to be overridden. Set it to zero if the default size is desired. Roger On 1/20/2016 12:05 AM, David Holmes wrote: > On 20/01/2016 8:39 AM, David Holmes wrote: >> On 20/01/2016 12:50 AM, Roger Riggs wrote: >>> Hi, >>> >>> How about "jdk.process.reaperStackSize". >> >> Based on some internal discussion that seems appropriate to me ... > > Except of course that the property is just a boolean not a size. :) So > jdk.process.reaperUsesdefaultStackSize seems appropriate. > > David > >>> It will need a CCC . >> >> ... and if not then the CCC will show the way. :) >> >> Thanks, >> David >> >>> Roger >>> >>> >>> On 1/19/16 7:10 AM, David Holmes wrote: >>>> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>>>> | >>>>> Hi Cheleswer, I think Martin is suggesting something like: >>>>> | >>>>> >>>>> // Use a modest stack size, unless requested otherwise: >>>>> long stackSize = >>>>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >>>> >>>> Someone from core-libs needs to chime on what the appropriate >>>> namespace for such a property would be. >>>> >>>> David >>>> ----- >>>> >>>>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>>>> reaper", stackSize); >>>>> >>>>> ||| >>>>> If that tests OK for you, it may be the way we can go ahead with the >>>>> point fix for this. >>>>> >>>>> Regards >>>>> Kevin >>>>> | >>>>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>>>> Historical note - I never liked having a reaper thread for each >>>>>> subprocess, but no other reliable implementation is known. Given the >>>>>> potential for many reaper threads, I at least wanted to keep the >>>>>> memory waste low. >>>>>> >>>>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>>>> wrote: >>>>>> >>>>>>> + Thread t = null; >>>>>>> + if >>>>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>>>> + t = new Thread(systemThreadGroup, >>>>>>> grimReaper, >>>>>>> "process reaper"); >>>>>>> + } else { >>>>>>> + // Our thread stack requirement is quite >>>>>>> modest. >>>>>>> + t = new Thread(systemThreadGroup, >>>>>>> grimReaper, >>>>>>> "process reaper", 32768); >>>>>>> + } >>>>>> If we do end up using this strategy, cleaner would be to use >>>>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>>>> >>>>>> >>>>>> >>>>>> stackSize - the desired stack size for the new thread, or zero to >>>>>> indicate that this parameter is to be ignored. >>>>> >>> From Roger.Riggs at Oracle.com Wed Jan 20 15:14:56 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 20 Jan 2016 10:14:56 -0500 Subject: RFR 9: 8145459 Cleaner - use Reference.reachabilityFence Message-ID: <569FA470.8030805@Oracle.com> Please review this update to the Cleanable implementations to use Reference.reachabilityFence: Webrev: http://cr.openjdk.java.net/~rriggs/webrev-fence-8145459/ Thanks, Roger From scolebourne at joda.org Wed Jan 20 15:15:02 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 20 Jan 2016 15:15:02 +0000 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <1448206903.20160116193150@gmail.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> Message-ID: I'm happy with the logic and specification of this proposal. I think it will be a useful addition. I'll let the Oracle team chime in to do a further review. thanks Stephen On 16 January 2016 at 13:31, Tagir F. Valeev wrote: > Hello! > > Thanks for review! Here's the updated patch: > http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ > > SC> The docs say that if the end date is before the start date, the > SC> stream is empty. While I can see just about why LongStream.range() > SC> works that way, I don't think this API should. The minimum is an > SC> exception, but it would be easy to support negative in the > SC> days-only case or the months-only case. The problem is where there > SC> are both months/years and days and those should be exceptions. > > Now datesUntil(endExclusive) throws an IllegalArgumentException if end > date is before start date. > > datesUntil(endExclusive, step) supports negative periods. It throws > IllegalArgumentException if: > - step is zero > - step.toTotalMonths() and step.getDays() both non-zero and have > opposite sign > - step is negative and end date is after start date > - step is positive and end date is before start date > > Otherwise it works correctly: you can use > LocalDate.of(2016, 1, 1) > .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); > > Steps like Period.of(-1, -1, -1) are also supported. > > SC> The single-arg implementation uses plusDays() with an > SC> incrementing number. When the performance patch goes in, the > SC> proposed streaming implementation will be optimal for small > SC> streams but less optimal for large ones. The fastest way to loop > SC> over a list of dates would be to manually generate them by > SC> incrementing the day until it exceeds the length of month, and so > SC> on. However, this would be serial. > > As I understand, plusDays performance patch is already pushed. > > It's possible to write custom Spliterator which would use > previous.plusDays(1) in tryAdvance() and jump (via > LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is > requested. However this adds at least one additional class and more > complexity. I guess, such optimization can be considered later as > separate issue when API is stabilized. > > SC> As such, I think the best way to write this, taking account of > SC> how plusDays() is implemented, is as follows: > > SC> LongStream.range(start.toEpochDay(), > SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); > > Ok, now it's done this way. > > SC> In the period-based method, it would be best to capture the case > SC> where totalMonths == 0 and days > 0 and forward to another method > SC> that ignores months. That private method can share implementation > SC> with the public single-arg method (passing in 1). > > This case still more complex than one-day case as it requires division > and multiplication. Thus I'd keep these case separately. However I > simplified "months = 0" path using ofEpochDay, now it should be > faster. > > SC> The docs for the period-based method should specify the strategy > SC> that produces the results (in abstract terms). This needs to cover > SC> that the result is equivalent to always adding to the start period > SC> a multiple of the period. > > I added some clarifications, please check. > > SC> Some nits: > SC> I prefer to avoid @link in the first sentence. Just using 'stream' > would be sufficient. > > Done. > > SC> The first sentence should be a summary. In this case it probably has a > bit too much detail. > > Done. > > SC> The @return has 'values' on a new line when it could be on the same > line. > > I set now line length = 100 characters in my IDE. Is it ok? > > SC> If statements need braces. > > Done. > > With best regards, > Tagir Valeev. > > From sanne at redhat.com Wed Jan 20 15:18:19 2016 From: sanne at redhat.com (Sanne Grinovero) Date: Wed, 20 Jan 2016 10:18:19 -0500 (EST) Subject: Moving sun.misc.VM In-Reply-To: <1784289825.10142934.1453302329687.JavaMail.zimbra@redhat.com> Message-ID: <1887615038.10182453.1453303099409.JavaMail.zimbra@redhat.com> Hello all, while testing latest Java 9 build 9-ea+101-2016-01-13-182959.javare.4276.nc with some popular OSS libraries, I noticed that sun.misc.VM is gone and this will cause some issues. This is causing compilation failures of type "cannot find symbol". Similarly the same projects are using sun.misc.Unsafe, but in that case we're getting a warning "sun.misc.Unsafe is internal proprietary API and may be removed in a future release". Is it intentional that sun.misc.VM takes a more aggressive migration path than the nice warnings we're getting for Unsafe? Thanks, Sanne From peter.levart at gmail.com Wed Jan 20 15:19:58 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 20 Jan 2016 16:19:58 +0100 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569F9515.5010506@Oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> <569EBB09.7030800@oracle.com> <569F158A.5060309@oracle.com> <569F9515.5010506@Oracle.com> Message-ID: <569FA59E.1000204@gmail.com> Hi, On 01/20/2016 03:09 PM, Roger Riggs wrote: > Hi David, > > I read an old description; I was expecting the value of the property > to be the stack size for the reaper. > That would allow more control and less waste when it needed to be > overridden. > Set it to zero if the default size is desired. > > Roger Wouldn't it be more appropriate that for modest use there would be no need to specify a system property. By modest use, I mean starting just a few sub-processes. So if we add 32k (or whatever the max. possible overhead of TLS is) to the existing 32k, that should be the default for reaper thread. If one needs to make it even smaller to accommodate for exceptional use, he can then use this new system property. Regards, Peter > > On 1/20/2016 12:05 AM, David Holmes wrote: >> On 20/01/2016 8:39 AM, David Holmes wrote: >>> On 20/01/2016 12:50 AM, Roger Riggs wrote: >>>> Hi, >>>> >>>> How about "jdk.process.reaperStackSize". >>> >>> Based on some internal discussion that seems appropriate to me ... >> >> Except of course that the property is just a boolean not a size. :) >> So jdk.process.reaperUsesdefaultStackSize seems appropriate. >> >> David >> >>>> It will need a CCC . >>> >>> ... and if not then the CCC will show the way. :) >>> >>> Thanks, >>> David >>> >>>> Roger >>>> >>>> >>>> On 1/19/16 7:10 AM, David Holmes wrote: >>>>> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>>>>> | >>>>>> Hi Cheleswer, I think Martin is suggesting something like: >>>>>> | >>>>>> >>>>>> // Use a modest stack size, unless requested otherwise: >>>>>> long stackSize = >>>>>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >>>>> >>>>> Someone from core-libs needs to chime on what the appropriate >>>>> namespace for such a property would be. >>>>> >>>>> David >>>>> ----- >>>>> >>>>>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>>>>> reaper", stackSize); >>>>>> >>>>>> ||| >>>>>> If that tests OK for you, it may be the way we can go ahead with the >>>>>> point fix for this. >>>>>> >>>>>> Regards >>>>>> Kevin >>>>>> | >>>>>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>>>>> Historical note - I never liked having a reaper thread for each >>>>>>> subprocess, but no other reliable implementation is known. Given >>>>>>> the >>>>>>> potential for many reaper threads, I at least wanted to keep the >>>>>>> memory waste low. >>>>>>> >>>>>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>>>>> wrote: >>>>>>> >>>>>>>> + Thread t = null; >>>>>>>> + if >>>>>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>> grimReaper, >>>>>>>> "process reaper"); >>>>>>>> + } else { >>>>>>>> + // Our thread stack requirement is quite >>>>>>>> modest. >>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>> grimReaper, >>>>>>>> "process reaper", 32768); >>>>>>>> + } >>>>>>> If we do end up using this strategy, cleaner would be to use >>>>>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>>>>> >>>>>>> >>>>>>> >>>>>>> stackSize - the desired stack size for the new thread, or zero to >>>>>>> indicate that this parameter is to be ignored. >>>>>> >>>> > From Roger.Riggs at Oracle.com Wed Jan 20 15:26:44 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 20 Jan 2016 10:26:44 -0500 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569FA59E.1000204@gmail.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> <569EBB09.7030800@oracle.com> <569F158A.5060309@oracle.com> <569F9515.5010506@Oracle.com> <569FA59E.1000204@gmail.com> Message-ID: <569FA734.1020302@Oracle.com> Hi Peter, David, on serviceability-dev [1], was of the opinion that a simple increase would be too temporary. Roger [1] On 1/20/2016 10:19 AM, Peter Levart wrote: > Hi, > > On 01/20/2016 03:09 PM, Roger Riggs wrote: >> Hi David, >> >> I read an old description; I was expecting the value of the property >> to be the stack size for the reaper. >> That would allow more control and less waste when it needed to be >> overridden. >> Set it to zero if the default size is desired. >> >> Roger > > Wouldn't it be more appropriate that for modest use there would be no > need to specify a system property. By modest use, I mean starting just > a few sub-processes. So if we add 32k (or whatever the max. possible > overhead of TLS is) to the existing 32k, that should be the default > for reaper thread. If one needs to make it even smaller to accommodate > for exceptional use, he can then use this new system property. > > Regards, Peter > >> >> On 1/20/2016 12:05 AM, David Holmes wrote: >>> On 20/01/2016 8:39 AM, David Holmes wrote: >>>> On 20/01/2016 12:50 AM, Roger Riggs wrote: >>>>> Hi, >>>>> >>>>> How about "jdk.process.reaperStackSize". >>>> >>>> Based on some internal discussion that seems appropriate to me ... >>> >>> Except of course that the property is just a boolean not a size. :) >>> So jdk.process.reaperUsesdefaultStackSize seems appropriate. >>> >>> David >>> >>>>> It will need a CCC . >>>> >>>> ... and if not then the CCC will show the way. :) >>>> >>>> Thanks, >>>> David >>>> >>>>> Roger >>>>> >>>>> >>>>> On 1/19/16 7:10 AM, David Holmes wrote: >>>>>> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>>>>>> | >>>>>>> Hi Cheleswer, I think Martin is suggesting something like: >>>>>>> | >>>>>>> >>>>>>> // Use a modest stack size, unless requested otherwise: >>>>>>> long stackSize = >>>>>>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >>>>>> >>>>>> Someone from core-libs needs to chime on what the appropriate >>>>>> namespace for such a property would be. >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>>>>>> reaper", stackSize); >>>>>>> >>>>>>> ||| >>>>>>> If that tests OK for you, it may be the way we can go ahead with >>>>>>> the >>>>>>> point fix for this. >>>>>>> >>>>>>> Regards >>>>>>> Kevin >>>>>>> | >>>>>>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>>>>>> Historical note - I never liked having a reaper thread for each >>>>>>>> subprocess, but no other reliable implementation is known. >>>>>>>> Given the >>>>>>>> potential for many reaper threads, I at least wanted to keep the >>>>>>>> memory waste low. >>>>>>>> >>>>>>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>>>>>> wrote: >>>>>>>> >>>>>>>>> + Thread t = null; >>>>>>>>> + if >>>>>>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>>> grimReaper, >>>>>>>>> "process reaper"); >>>>>>>>> + } else { >>>>>>>>> + // Our thread stack requirement is quite >>>>>>>>> modest. >>>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>>> grimReaper, >>>>>>>>> "process reaper", 32768); >>>>>>>>> + } >>>>>>>> If we do end up using this strategy, cleaner would be to use >>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> stackSize - the desired stack size for the new thread, or zero to >>>>>>>> indicate that this parameter is to be ignored. >>>>>>> >>>>> >> > From paul.sandoz at oracle.com Wed Jan 20 15:30:46 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 20 Jan 2016 16:30:46 +0100 Subject: RFR 9: 8145459 Cleaner - use Reference.reachabilityFence In-Reply-To: <569FA470.8030805@Oracle.com> References: <569FA470.8030805@Oracle.com> Message-ID: <39735826-6553-407F-9677-66B962B69AEA@oracle.com> > On 20 Jan 2016, at 16:14, Roger Riggs wrote: > > Please review this update to the Cleanable implementations to use Reference.reachabilityFence: > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fence-8145459/ > Yay, +1 Paul. From aph at redhat.com Wed Jan 20 15:33:10 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 20 Jan 2016 15:33:10 +0000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569FA59E.1000204@gmail.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> <569EBB09.7030800@oracle.com> <569F158A.5060309@oracle.com> <569F9515.5010506@Oracle.com> <569FA59E.1000204@gmail.com> Message-ID: <569FA8B6.4000602@redhat.com> On 01/20/2016 03:19 PM, Peter Levart wrote: > So if we add 32k (or whatever the max. possible > overhead of TLS is) to the existing 32k, that should be the default for > reaper thread. I'm not sure how many thread-local variables are possible in theory, but I suspect it's unlimited. In machines with growable memory regions for stacks allocating the default stack has very little overhead, so it doesn't matter. Systems which pre-allocate their thread stacks have their own problems. These tend to be special kinds of cloud containers and some turnkey systems which cannot tolerate an out-of-memory error. Andrew. From Alan.Bateman at oracle.com Wed Jan 20 15:36:48 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jan 2016 15:36:48 +0000 Subject: Moving sun.misc.VM In-Reply-To: <1887615038.10182453.1453303099409.JavaMail.zimbra@redhat.com> References: <1887615038.10182453.1453303099409.JavaMail.zimbra@redhat.com> Message-ID: <569FA990.8030201@oracle.com> On 20/01/2016 15:18, Sanne Grinovero wrote: > Hello all, > while testing latest Java 9 build 9-ea+101-2016-01-13-182959.javare.4276.nc with some popular OSS libraries, I noticed that sun.misc.VM is gone and this will cause some issues. > > This is causing compilation failures of type "cannot find symbol". Similarly the same projects are using sun.misc.Unsafe, but in that case we're getting a warning "sun.misc.Unsafe is internal proprietary API and may be removed in a future release". > > Is it intentional that sun.misc.VM takes a more aggressive migration path than the nice warnings we're getting for Unsafe? > sun.misc.VM is not one of the "critical internal APIs" that identified in JEP 260 [1]. There is ongoing work (mostly by Chris Hegarty) to move the non-critical APIs out of sun.misc and sun.reflect so that all that remains is the critical internal APIs. Are you seeing a lot of usage of sun.misc.VM? If so then best to bring usage data to jigsaw-dev to lobby for it to be considered as a critical internal API. -Alan. [1] http://openjdk.java.net/jeps/260 From mandy.chung at oracle.com Wed Jan 20 15:52:14 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 20 Jan 2016 07:52:14 -0800 Subject: RFR 9: 8145459 Cleaner - use Reference.reachabilityFence In-Reply-To: <569FA470.8030805@Oracle.com> References: <569FA470.8030805@Oracle.com> Message-ID: > On Jan 20, 2016, at 7:14 AM, Roger Riggs wrote: > > Please review this update to the Cleanable implementations to use Reference.reachabilityFence: > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fence-8145459/ +1 Mandy From sanne at redhat.com Wed Jan 20 16:13:24 2016 From: sanne at redhat.com (Sanne Grinovero) Date: Wed, 20 Jan 2016 11:13:24 -0500 (EST) Subject: Moving sun.misc.VM In-Reply-To: <569FA990.8030201@oracle.com> References: <1887615038.10182453.1453303099409.JavaMail.zimbra@redhat.com> <569FA990.8030201@oracle.com> Message-ID: <2054191890.10231374.1453306404850.JavaMail.zimbra@redhat.com> Hello, thanks for clarifying about some APIs being considered "critical". No, I'm not seeing lots of usage: actually just one instance so far so it should be possible to patch already with minimal impact. I'm understanding that this API will be hidden. Will there be an alternative? In this case it's just being used to invoke: sun.misc.VM.getSavedProperty("java.lang.Integer.IntegerCache.high") I'm not entirely familiar with this code, but it seems the purpose is to estimate memory usage at runtime for objects, so it's attempting to read this value to disregard the Integer instances which are being reused from the pool. Wondering if you could suggest an alternative, or should we plan for not having this level of detail anymore? Thanks, Sanne ----- Original Message ----- > > > On 20/01/2016 15:18, Sanne Grinovero wrote: > > Hello all, > > while testing latest Java 9 build 9-ea+101-2016-01-13-182959.javare.4276.nc > > with some popular OSS libraries, I noticed that sun.misc.VM is gone and > > this will cause some issues. > > > > This is causing compilation failures of type "cannot find symbol". > > Similarly the same projects are using sun.misc.Unsafe, but in that case > > we're getting a warning "sun.misc.Unsafe is internal proprietary API and > > may be removed in a future release". > > > > Is it intentional that sun.misc.VM takes a more aggressive migration path > > than the nice warnings we're getting for Unsafe? > > > sun.misc.VM is not one of the "critical internal APIs" that identified > in JEP 260 [1]. There is ongoing work (mostly by Chris Hegarty) to move > the non-critical APIs out of sun.misc and sun.reflect so that all that > remains is the critical internal APIs. > > Are you seeing a lot of usage of sun.misc.VM? If so then best to bring > usage data to jigsaw-dev to lobby for it to be considered as a critical > internal API. > > -Alan. > > [1] http://openjdk.java.net/jeps/260 > From mark.sheppard at oracle.com Wed Jan 20 16:16:19 2016 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Wed, 20 Jan 2016 16:16:19 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <568FF6A5.4080700@oracle.com> References: <568FF6A5.4080700@oracle.com> Message-ID: <569FB2D3.6080607@oracle.com> Hi, an update has been made to the webrev http://cr.openjdk.java.net/~msheppar/8144144/webrev.03/ an anomaly was found in the select loop of the SelectorImpl.run() method some defensive programming, for selector null references, have been added, also. regards Mark On 08/01/2016 17:49, Mark Sheppard wrote: > Hi > please oblige and review the following changes > http://cr.openjdk.java.net/~msheppar/8144144/webrev/ > > which addresses the issue > https://bugs.openjdk.java.net/browse/JDK-8144144 > > the changes ensure that an Acceptor and its associated > ServerSocket/ServerSocketChannel > are closed. > > regards > Mark From mandy.chung at oracle.com Wed Jan 20 16:20:26 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 20 Jan 2016 08:20:26 -0800 Subject: Ephemerons In-Reply-To: <569BDAF2.2080404@gmail.com> References: <569BDAF2.2080404@gmail.com> Message-ID: <63A40C76-1D63-47F4-B65A-1BDBE7EE000E@oracle.com> Hi Peter, This is interesting and useful for cases when key/value in a reference chain cycle. This seems worth exploring for a future release. I don?t have any cycle to study this in depth and begin any discussion unfortunately at this time. Have you tried prototype Ephemerons with WeakReference for the key and PhantomReference for the value? Would that satisfy your cases - it would not work if SoftReference is part of the chain - just wondering if you have explored that? Mandy > On Jan 17, 2016, at 10:18 AM, Peter Levart wrote: > > Hi, > > Ephemerons are special kind of references described by Barry Hayes [1]. In the simple variant, they contain two "referents". One is called the key, which can be viewed as a "weak" referent in the style of Java reference types (SoftReference, WeakReference, PhantomReference), the other is called the value, which is also treated specially by GC. It's reachability is dependent on the reachability of the key. > > Ephemerons solve the problem seen for example in the java.util.WeakHashMap when a value in this map directly or indirectly refers to it's key. Such entry will never be purged as the value is always reachable from the WeakHashMap instance and through the value, it's key is reachable too. There are other places where Ephemerons could be used - for example in ClassValue API. Try googling for "java ephemeron" (without quotes) to find out other situations where Ephemerons would be beneficial. > > If one was to implement Ephemerons in Java, the main question he would be asking is how Ephemeron(s) were supposed to interact with existing Java reference types. Hayes only defines their behavior in isolation, but Java already has 4 "weak" reference types with different "strengths": SoftReference, WeakReference, FinalReference (for implementing finalization) and PhantomReference. In the prototype I choose to define Ephemerons as a new reference type with it's own "strength" for the key referent positioned between WeakReference and FinalReference. It would be possible to merge it with an existing reference type like WeakReference or position it between SoftReference and WeakReference or even entirely "on the top" of the strength list, but I think it would not be wise to position it below FinalReference or even PhantomReference. What's best is an open question. What is also not so obvious is how to define the reachability of the Ephemeron's value and it's interaction with existing reference types. I think I defined it soundly (see the reachability types in the javadoc of [4]). The reason for defining the reachability of the value to alternate between ephemeraly-reachable and weakly-reachable and not between ephemeraly-reachable and strongly-reachable, while theoretically possible, is the desire to have a separate phase of processing for each reachability strength, like it is done currently, and which doesn't affect the performance of processing of existing reference types. > > Having skimmed through the reference processing code in the VM for a couple of times, I thought it should not be too complicated to implement another type of "weak" reference. Encouraged by how little changes were needed to remove the sun.misc.Cleaner special type of reference [2], I began experimenting and came up with a prototype that seems to work [3]. Luckily most of the logic to process Reference objects in VM is encapsulated in the ReferenceProcessor class and this logic is invoked from various GC implementations of the hotspot. Changes needed are therefore mostly localized to this class. The webrev also contains recent changes from JDK-8143847 [2] that have not yet propagated to the jdk9-dev repo. When they do, I will prepare a rebased webrev without those changes. > > I'm publishing this prototype here to get the answer to the main question: Is there an interest to have Ephemeron reference type in Java? > > Given that there was interest, I would also like to initiate a discussion about: > - What would be the most appropriate "strength" of reachabillity for Epehmeron referents (key and value) and the desired interactions with existing reachabilities. > - The prototype itself. Since I'm new to this part of code (that's my 1st not so shallow dive into the VM), I surely must have missed places that should be changed. Although the prototype seems to work, I have not created extensive tests for the functionality and have not exposed it to all the various GC algorithms and situations yet. I could use some advise on how to exercise all GC algorithm combinations possible in the VM (what flags to use, for example). > > Regards, Peter > > [1] https://static.aminer.org/pdf/PDF/000/522/273/ephemerons_a_new_finalization_mechanism.pdf > [2] https://bugs.openjdk.java.net/browse/JDK-8143847 > [3] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/ > [4] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.01/src/java.base/share/classes/java/lang/ref/Ephemeron.java.html > > P.S. > > To wet some appetite, with the above prototype (applied to current jdk9-dev), it is possible to run the following example: > > import java.lang.ref.Ephemeron; > import java.util.ArrayList; > import java.util.List; > > public class EphemeronTest { > > static class Key { > final int i; > > Key(int i) { > this.i = i; > } > > @Override > public String toString() { > return "k" + i; > } > } > > static class Value { > final Key key; > > Value(Key key) { > this.key = key; > } > > @Override > public String toString() { > return "v(" + key + ")"; > } > } > > static class Eph extends Ephemeron { > public Eph(Key key, Value value) { > super(key, value); > } > > @Override > public String toString() { > return getKey() + "=" + getValue(); > } > } > > static void test(int size, boolean forwardChaining) throws Exception { > System.out.println(); > System.out.println((forwardChaining ? "Forward" : "Backward") + > " chaining of value->key ..."); > System.out.println(); > List ephs = new ArrayList<>(size); > > Key k1 = new Key(1); > { > Key kp = k1; > for (int i = 2; i <= size; i++) { > Key ki = new Key(i); > ephs.add( > forwardChaining > ? new Eph(kp, new Value(ki)) > : new Eph(ki, new Value(kp)) > ); > kp = ki; > } > ephs.add( > forwardChaining > ? new Eph(kp, new Value(k1)) > : new Eph(k1, new Value(kp)) > ); > kp = null; > } > > System.gc(); > System.out.println("1: " + ephs); > k1.getClass(); // reachabilityFence > > k1 = null; > System.gc(); > System.out.println("2: " + ephs); > } > > public static void main(String[] args) throws Exception { > > int size = (args.length < 1) > ? 5 > : Math.max(2, Integer.parseInt(args[0])); > > test(size, true); > test(size, false); > } > } > > > Which prints: > > Forward chaining of value->key ... > > 1: [k1=v(k2), k2=v(k3), k3=v(k4), k4=v(k5), k5=v(k1)] > 2: [null=null, null=null, null=null, null=null, null=null] > > Backward chaining of value->key ... > > 1: [k2=v(k1), k3=v(k2), k4=v(k3), k5=v(k4), k1=v(k5)] > 2: [null=null, null=null, null=null, null=null, null=null] > > > If you compile the JDK with --enable-debug and use the following VM switches when running: -XX:+PrintGCDetails -XX:+TraceReferenceGC, you can also observe the inner workings of the reference processing, including Ephemerons. > From chris.hegarty at oracle.com Wed Jan 20 16:28:20 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Jan 2016 16:28:20 +0000 Subject: RFR 9: 8145459 Cleaner - use Reference.reachabilityFence In-Reply-To: <569FA470.8030805@Oracle.com> References: <569FA470.8030805@Oracle.com> Message-ID: <33A0BD0E-8AA9-42A0-B401-8B36D8431AF1@oracle.com> On 20 Jan 2016, at 15:14, Roger Riggs wrote: > Please review this update to the Cleanable implementations to use Reference.reachabilityFence: > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-fence-8145459/ Looks good to me Roger. -Chris. > Thanks, Roger > From claes.redestad at oracle.com Wed Jan 20 16:35:31 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Wed, 20 Jan 2016 17:35:31 +0100 Subject: Moving sun.misc.VM In-Reply-To: <2054191890.10231374.1453306404850.JavaMail.zimbra@redhat.com> References: <1887615038.10182453.1453303099409.JavaMail.zimbra@redhat.com> <569FA990.8030201@oracle.com> <2054191890.10231374.1453306404850.JavaMail.zimbra@redhat.com> Message-ID: <569FB753.6010302@oracle.com> Hi! How about: int findIntegerCacheHighPropValue() { int test = 128; // minimum mandated by JLS is 127 while (Integer.valueOf(test) == Integer.valueOf(test)) { test++; } return test - 1; } Since this runs through and hits the cache until the last step, it should be about as fast. Should also be more robust since it'll give the true value for any existing JLS-compliant VM no matter what mechanism they use to set up their integer cache. Thanks! /Claes On 2016-01-20 17:13, Sanne Grinovero wrote: > Hello, > thanks for clarifying about some APIs being considered "critical". > > No, I'm not seeing lots of usage: actually just one instance so far so it should be possible to patch already with minimal impact. > > I'm understanding that this API will be hidden. Will there be an alternative? > In this case it's just being used to invoke: > > sun.misc.VM.getSavedProperty("java.lang.Integer.IntegerCache.high") > > I'm not entirely familiar with this code, but it seems the purpose is to estimate memory usage at runtime for objects, > so it's attempting to read this value to disregard the Integer instances which are being reused from the pool. > > Wondering if you could suggest an alternative, or should we plan for not having this level of detail anymore? > > Thanks, > Sanne > > > ----- Original Message ----- >> >> On 20/01/2016 15:18, Sanne Grinovero wrote: >>> Hello all, >>> while testing latest Java 9 build 9-ea+101-2016-01-13-182959.javare.4276.nc >>> with some popular OSS libraries, I noticed that sun.misc.VM is gone and >>> this will cause some issues. >>> >>> This is causing compilation failures of type "cannot find symbol". >>> Similarly the same projects are using sun.misc.Unsafe, but in that case >>> we're getting a warning "sun.misc.Unsafe is internal proprietary API and >>> may be removed in a future release". >>> >>> Is it intentional that sun.misc.VM takes a more aggressive migration path >>> than the nice warnings we're getting for Unsafe? >>> >> sun.misc.VM is not one of the "critical internal APIs" that identified >> in JEP 260 [1]. There is ongoing work (mostly by Chris Hegarty) to move >> the non-critical APIs out of sun.misc and sun.reflect so that all that >> remains is the critical internal APIs. >> >> Are you seeing a lot of usage of sun.misc.VM? If so then best to bring >> usage data to jigsaw-dev to lobby for it to be considered as a critical >> internal API. >> >> -Alan. >> >> [1] http://openjdk.java.net/jeps/260 >> From steve.drach at oracle.com Wed Jan 20 17:56:35 2016 From: steve.drach at oracle.com (Steve Drach) Date: Wed, 20 Jan 2016 09:56:35 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> Message-ID: Hi, This is a repeat of the RFR I sent last Wed (Jan 13). CCC had some suggestions to improve the code that was previously approved during review. As a consequence, I had to make significant changes to the API, and I believe the code need further review. Please review the latest iteration of the webrev for runtime support of multi-release jar files. Issue: https://bugs.openjdk.java.net/browse/JDK-8132734 JEP 238: https://bugs.openjdk.java.net/browse/JDK-8047305 Webrev: http://cr.openjdk.java.net/~sdrach/8132734/webrev.02/ Here?s a list of things that were changed: 1. Removed setVersion, setRuntimeVersioned, and getRuntimeVersionedEntry. Also removed the configuration lock. 2. Added a Release enum to represent releases. 3. Added a new constructor that uses the enum as one of it?s arguments. This is the only way to make multi-release JarFile objects. 4. Added new code to provide runtime versioning for class loaders using the new constructor. 4. Made the ?version? int final, removing the volatile modifier. 5. Made the two remaining public methods, getVersion and isMultiRelease final. 6. Added implSpec to getEntry and getJarEntry so potential subclassers are aware of how things work. 7. Rewrote and added tests to deal with the new constructor and the Release enum. 8. Added a simple http server to test getting jar files from both a local files system and from the network. Thanks, Steve From sean.coffey at oracle.com Wed Jan 20 18:17:24 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 20 Jan 2016 18:17:24 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <569FB2D3.6080607@oracle.com> References: <568FF6A5.4080700@oracle.com> <569FB2D3.6080607@oracle.com> Message-ID: <569FCF34.8020401@oracle.com> Hi Mark, SelectorImpl.java: line 125, could you use a 2 arg method call to dprint. It'll print the stacktrace instead : dprint(".close: selector.close: " + t); --> dprint(".close: selector.close", t); The "while (!isClosed()) " change introduces a new hot lock on closed variable. Hopefully, it won't impact performance too much. Looks good to me otherwise. Regards, Sean. On 20/01/16 16:16, Mark Sheppard wrote: > Hi, > an update has been made to the webrev > > http://cr.openjdk.java.net/~msheppar/8144144/webrev.03/ > > an anomaly was found in the select loop of the SelectorImpl.run() method > some defensive programming, for selector null references, have been > added, also. > > regards > Mark > > On 08/01/2016 17:49, Mark Sheppard wrote: >> Hi >> please oblige and review the following changes >> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ >> >> which addresses the issue >> https://bugs.openjdk.java.net/browse/JDK-8144144 >> >> the changes ensure that an Acceptor and its associated >> ServerSocket/ServerSocketChannel >> are closed. >> >> regards >> Mark > From sanne at redhat.com Wed Jan 20 19:21:47 2016 From: sanne at redhat.com (Sanne Grinovero) Date: Wed, 20 Jan 2016 14:21:47 -0500 (EST) Subject: Moving sun.misc.VM In-Reply-To: <569FB753.6010302@oracle.com> References: <1887615038.10182453.1453303099409.JavaMail.zimbra@redhat.com> <569FA990.8030201@oracle.com> <2054191890.10231374.1453306404850.JavaMail.zimbra@redhat.com> <569FB753.6010302@oracle.com> Message-ID: <1621736889.10324254.1453317707103.JavaMail.zimbra@redhat.com> Hi Claes, excellent idea, I'll point them to this thread. Sanne ----- Original Message ----- > Hi! > > How about: > > int findIntegerCacheHighPropValue() { > int test = 128; // minimum mandated by JLS is 127 > while (Integer.valueOf(test) == Integer.valueOf(test)) { > test++; > } > return test - 1; > } > > Since this runs through and hits the cache until the last step, it > should be about as fast. Should also be more robust since it'll give the > true value for any existing JLS-compliant VM no matter what mechanism > they use to set up their integer cache. > > Thanks! > > /Claes > > On 2016-01-20 17:13, Sanne Grinovero wrote: > > Hello, > > thanks for clarifying about some APIs being considered "critical". > > > > No, I'm not seeing lots of usage: actually just one instance so far so it > > should be possible to patch already with minimal impact. > > > > I'm understanding that this API will be hidden. Will there be an > > alternative? > > In this case it's just being used to invoke: > > > > sun.misc.VM.getSavedProperty("java.lang.Integer.IntegerCache.high") > > > > I'm not entirely familiar with this code, but it seems the purpose is to > > estimate memory usage at runtime for objects, > > so it's attempting to read this value to disregard the Integer instances > > which are being reused from the pool. > > > > Wondering if you could suggest an alternative, or should we plan for not > > having this level of detail anymore? > > > > Thanks, > > Sanne > > > > > > ----- Original Message ----- > >> > >> On 20/01/2016 15:18, Sanne Grinovero wrote: > >>> Hello all, > >>> while testing latest Java 9 build > >>> 9-ea+101-2016-01-13-182959.javare.4276.nc > >>> with some popular OSS libraries, I noticed that sun.misc.VM is gone and > >>> this will cause some issues. > >>> > >>> This is causing compilation failures of type "cannot find symbol". > >>> Similarly the same projects are using sun.misc.Unsafe, but in that case > >>> we're getting a warning "sun.misc.Unsafe is internal proprietary API and > >>> may be removed in a future release". > >>> > >>> Is it intentional that sun.misc.VM takes a more aggressive migration path > >>> than the nice warnings we're getting for Unsafe? > >>> > >> sun.misc.VM is not one of the "critical internal APIs" that identified > >> in JEP 260 [1]. There is ongoing work (mostly by Chris Hegarty) to move > >> the non-critical APIs out of sun.misc and sun.reflect so that all that > >> remains is the critical internal APIs. > >> > >> Are you seeing a lot of usage of sun.misc.VM? If so then best to bring > >> usage data to jigsaw-dev to lobby for it to be considered as a critical > >> internal API. > >> > >> -Alan. > >> > >> [1] http://openjdk.java.net/jeps/260 > >> > > From Alan.Bateman at oracle.com Wed Jan 20 20:05:38 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 20 Jan 2016 20:05:38 +0000 Subject: GNU-style options for the Jar Tool In-Reply-To: <433503D4-FF76-4F41-BF4F-2853F57A692A@oracle.com> References: <433503D4-FF76-4F41-BF4F-2853F57A692A@oracle.com> Message-ID: <569FE892.40808@oracle.com> On 19/01/2016 16:54, Chris Hegarty wrote: > GNU-style options for Jar Tool > > Prompted by the impending addition of new options for both multi-release jars > and jigsaw, it became immediately obvious that we were running out of single > characters that could be used to represent these new options. I'd like to > propose an update to the Jar Tool CLI to accept GNU-style options ( all short > forms being compatible with the existing current options ), so that we can > evolve the CLI in a reasonable fashion going forward. This looks quite good. For me at least then I know it will take time to get used to typing --file=FILE. The only option that I wonder about is --pack200 as it hints that it does pack200 compression. Should it be --normalize instead? The help output might not even need to mention pack200. The only other comment is something that Stephen Colebourne brought up on jigsaw-dev a few months ago. The updated `jar --help` is too long. The useful options scroll up to leave the user looking at options under the "Compatibility Interface" heading. I wonder if it would be better to move this to --help:compat. -Alan. From chris.hegarty at oracle.com Wed Jan 20 20:45:37 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 20 Jan 2016 20:45:37 +0000 Subject: RFR 8065076/9, test/java/net/SocketPermission/SocketPermissionTest.java failed intermittently In-Reply-To: References: <568F74F7.70100@oracle.com> <56973B1A.30000@oracle.com> Message-ID: On 20 Jan 2016, at 06:36, Chris Hegarty wrote: > Felix, > > On 14 Jan 2016, at 06:07, Felix Yang wrote: > >> Hi all, >> please review the fix for test/java/net/SocketPermission/SocketPermissionTest.java, which fails frequently with "java.net.BindException: Address already in use". >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8065076 >> Webrev: http://cr.openjdk.java.net/~xiaofeya/8065076/webrev.00 > > My preference is to avoid getFreePort. It is problematic and I believe just obfuscates > the intermittent failures further. > > In many of the test scenarios the ?listening? socket can be created before the specific > access control context and associated permission are created. > > I?ll see if I can get some time to try this out. I spent a little time on this today. I basically rewrote the test, but kept the same test scenarios. The use of data providers was cute, but not workable since there is no common supertype for the socket classes. I decided to just expand out the test cases manually. This will give the same test coverage, but should be stable since it creates the sockets first, on an ephemeral port, and then constructs the permissions appropriately given that port. http://cr.openjdk.java.net/~chegar/8065076/ The webrev diffs are almost useless, just review the new file, and compare test scenarios against the what is in the old file. -Chris. From kedar.mhaswade at gmail.com Wed Jan 20 21:33:30 2016 From: kedar.mhaswade at gmail.com (kedar mhaswade) Date: Wed, 20 Jan 2016 13:33:30 -0800 Subject: GNU-style options for the Jar Tool In-Reply-To: <569FE892.40808@oracle.com> References: <433503D4-FF76-4F41-BF4F-2853F57A692A@oracle.com> <569FE892.40808@oracle.com> Message-ID: Looks good! I suggest we use -h (in addition to or instead of -?) for help. I believe saying "GNU-style" means the boolean short options (e.g. -t, -v) can be combined (like -tv) to specify multiple actions, but maybe the compatibility interface should mention that that is the case. Will displaying manifest contents (especially the main-class attribute, if any) be a worthwhile option to explore or will that be out of scope for this one? Usually, I end up unjar-ing META-INF/MANIFEST.MF just to see what the main-class is. Regards, Kedar On Wed, Jan 20, 2016 at 12:05 PM, Alan Bateman wrote: > > On 19/01/2016 16:54, Chris Hegarty wrote: > >> GNU-style options for Jar Tool >> >> Prompted by the impending addition of new options for both multi-release >> jars >> and jigsaw, it became immediately obvious that we were running out of >> single >> characters that could be used to represent these new options. I'd like to >> propose an update to the Jar Tool CLI to accept GNU-style options ( all >> short >> forms being compatible with the existing current options ), so that we can >> evolve the CLI in a reasonable fashion going forward. >> > This looks quite good. For me at least then I know it will take time to > get used to typing --file=FILE. > > The only option that I wonder about is --pack200 as it hints that it does > pack200 compression. Should it be --normalize instead? The help output > might not even need to mention pack200. > > The only other comment is something that Stephen Colebourne brought up on > jigsaw-dev a few months ago. The updated `jar --help` is too long. The > useful options scroll up to leave the user looking at options under the > "Compatibility Interface" heading. I wonder if it would be better to move > this to --help:compat. > > -Alan. > From david.holmes at oracle.com Thu Jan 21 08:57:34 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 21 Jan 2016 18:57:34 +1000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569FA59E.1000204@gmail.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> <569EBB09.7030800@oracle.com> <569F158A.5060309@oracle.com> <569F9515.5010506@Oracle.com> <569FA59E.1000204@gmail.com> Message-ID: <56A09D7E.8060203@oracle.com> The cc's are going dropped unfortunately - adding by serviceability. On 21/01/2016 1:19 AM, Peter Levart wrote: > Hi, > > On 01/20/2016 03:09 PM, Roger Riggs wrote: >> Hi David, >> >> I read an old description; I was expecting the value of the property >> to be the stack size for the reaper. >> That would allow more control and less waste when it needed to be >> overridden. >> Set it to zero if the default size is desired. >> >> Roger > > Wouldn't it be more appropriate that for modest use there would be no > need to specify a system property. By modest use, I mean starting just a > few sub-processes. So if we add 32k (or whatever the max. possible > overhead of TLS is) to the existing 32k, that should be the default for > reaper thread. If one needs to make it even smaller to accommodate for > exceptional use, he can then use this new system property. There is no reasonable "max possible" - the 32K is from the current problem that needs to be solved. But simply adding 32K today doesn't help if the third-part-library increases it's TLS use to 33K next week. We need a means for the person running the JVM to expand the stack size of the reaper thread if they hit this TLS-based problem. A flag that says "use the Java Thread default" suffices because the Java stack size can be further adjusted if even its default is not enough. Or we could just say forget about trying to manage this one thread's stack and let it behave like a normal Java thread. Might be the simplest solution by far. As Martin said he was just trying to be a good citizen with limiting the potential memory use. Cheers, David > Regards, Peter > >> >> On 1/20/2016 12:05 AM, David Holmes wrote: >>> On 20/01/2016 8:39 AM, David Holmes wrote: >>>> On 20/01/2016 12:50 AM, Roger Riggs wrote: >>>>> Hi, >>>>> >>>>> How about "jdk.process.reaperStackSize". >>>> >>>> Based on some internal discussion that seems appropriate to me ... >>> >>> Except of course that the property is just a boolean not a size. :) >>> So jdk.process.reaperUsesdefaultStackSize seems appropriate. >>> >>> David >>> >>>>> It will need a CCC . >>>> >>>> ... and if not then the CCC will show the way. :) >>>> >>>> Thanks, >>>> David >>>> >>>>> Roger >>>>> >>>>> >>>>> On 1/19/16 7:10 AM, David Holmes wrote: >>>>>> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>>>>>> | >>>>>>> Hi Cheleswer, I think Martin is suggesting something like: >>>>>>> | >>>>>>> >>>>>>> // Use a modest stack size, unless requested otherwise: >>>>>>> long stackSize = >>>>>>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >>>>>> >>>>>> Someone from core-libs needs to chime on what the appropriate >>>>>> namespace for such a property would be. >>>>>> >>>>>> David >>>>>> ----- >>>>>> >>>>>>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>>>>>> reaper", stackSize); >>>>>>> >>>>>>> ||| >>>>>>> If that tests OK for you, it may be the way we can go ahead with the >>>>>>> point fix for this. >>>>>>> >>>>>>> Regards >>>>>>> Kevin >>>>>>> | >>>>>>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>>>>>> Historical note - I never liked having a reaper thread for each >>>>>>>> subprocess, but no other reliable implementation is known. Given >>>>>>>> the >>>>>>>> potential for many reaper threads, I at least wanted to keep the >>>>>>>> memory waste low. >>>>>>>> >>>>>>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>>>>>> wrote: >>>>>>>> >>>>>>>>> + Thread t = null; >>>>>>>>> + if >>>>>>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>>> grimReaper, >>>>>>>>> "process reaper"); >>>>>>>>> + } else { >>>>>>>>> + // Our thread stack requirement is quite >>>>>>>>> modest. >>>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>>> grimReaper, >>>>>>>>> "process reaper", 32768); >>>>>>>>> + } >>>>>>>> If we do end up using this strategy, cleaner would be to use >>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> stackSize - the desired stack size for the new thread, or zero to >>>>>>>> indicate that this parameter is to be ignored. >>>>>>> >>>>> >> > From adinn at redhat.com Thu Jan 21 09:07:41 2016 From: adinn at redhat.com (Andrew Dinn) Date: Thu, 21 Jan 2016 09:07:41 +0000 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: Message-ID: <56A09FDD.9080305@redhat.com> Hi Rafael, On 01/12/15 12:45, Rafael Winterhalter wrote: > I can tell that this behavior has not only affected me as I had this > question comming up by multiple users of my open-source code generation > library. > > What is your view on this? > > Thank you for your feedback! I have an interest in this issue because I lead a Java agent-based project (Byteman). I have not (yet) come across a case where the behaviour you describe has affected the operation of my agent -- I suspect that is because it will never attempt to transform a class defined via Unsafe::defineAnonymousClass. However, I agree with you that this behaviour appears anomalous and the outcome you describe is counter-intuitive. I would be very interested to know if there was a good reason for excluding anonymous classes from transformation. regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in UK and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul Argiry (US) From peter.levart at gmail.com Thu Jan 21 09:59:37 2016 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 21 Jan 2016 10:59:37 +0100 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <56A09D7E.8060203@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <569E4D31.1010006@oracle.com> <569EBB09.7030800@oracle.com> <569F158A.5060309@oracle.com> <569F9515.5010506@Oracle.com> <569FA59E.1000204@gmail.com> <56A09D7E.8060203@oracle.com> Message-ID: <56A0AC09.7010407@gmail.com> On 01/21/2016 09:57 AM, David Holmes wrote: > The cc's are going dropped unfortunately - adding by serviceability. > > On 21/01/2016 1:19 AM, Peter Levart wrote: >> Hi, >> >> On 01/20/2016 03:09 PM, Roger Riggs wrote: >>> Hi David, >>> >>> I read an old description; I was expecting the value of the property >>> to be the stack size for the reaper. >>> That would allow more control and less waste when it needed to be >>> overridden. >>> Set it to zero if the default size is desired. >>> >>> Roger >> >> Wouldn't it be more appropriate that for modest use there would be no >> need to specify a system property. By modest use, I mean starting just a >> few sub-processes. So if we add 32k (or whatever the max. possible >> overhead of TLS is) to the existing 32k, that should be the default for >> reaper thread. If one needs to make it even smaller to accommodate for >> exceptional use, he can then use this new system property. > > There is no reasonable "max possible" - the 32K is from the current > problem that needs to be solved. But simply adding 32K today doesn't > help if the third-part-library increases it's TLS use to 33K next > week. We need a means for the person running the JVM to expand the > stack size of the reaper thread if they hit this TLS-based problem. A > flag that says "use the Java Thread default" suffices because the Java > stack size can be further adjusted if even its default is not enough. > > Or we could just say forget about trying to manage this one thread's > stack and let it behave like a normal Java thread. Might be the > simplest solution by far. As Martin said he was just trying to be a > good citizen with limiting the potential memory use. > > Cheers, > David Ok, I see. But what about a more "complicated" ;-) solution, like: public class ProbeMinThreadStackSize { static class ProbeThread extends Thread { ProbeThread(Runnable target, long stackSize) { super(Thread.currentThread().getThreadGroup(), target, "ProbeThread-stackSize-" + stackSize, stackSize); } boolean failed; @Override public void run() { try { super.run(); } catch (Throwable e) { failed = true; } } } static long probeMinStackSize(long min, long step, long max, Runnable probeCode) { for (long ss = min; ss <= max; ss += step) { try { ProbeThread pt = new ProbeThread(probeCode, ss); pt.start(); pt.join(); if (!pt.failed) { // got it! return ss; } } catch (Throwable t) { // continue with next } } // return 0 indicating usage of default Thread's stack size return 0L; } static void recurse(int n, long p1, long p2, long p3, long p4) { if (n > 0) recurse(n-1, p2, p3, p4, p1); } public static void main(String[] args) { long minStackSize = probeMinStackSize(32768L, 8192L, 1024L*1024L, () -> recurse(100, 1L, 2L, 3L, 4L)); System.out.println(minStackSize); } } Regards, Peter > >> Regards, Peter >> >>> >>> On 1/20/2016 12:05 AM, David Holmes wrote: >>>> On 20/01/2016 8:39 AM, David Holmes wrote: >>>>> On 20/01/2016 12:50 AM, Roger Riggs wrote: >>>>>> Hi, >>>>>> >>>>>> How about "jdk.process.reaperStackSize". >>>>> >>>>> Based on some internal discussion that seems appropriate to me ... >>>> >>>> Except of course that the property is just a boolean not a size. :) >>>> So jdk.process.reaperUsesdefaultStackSize seems appropriate. >>>> >>>> David >>>> >>>>>> It will need a CCC . >>>>> >>>>> ... and if not then the CCC will show the way. :) >>>>> >>>>> Thanks, >>>>> David >>>>> >>>>>> Roger >>>>>> >>>>>> >>>>>> On 1/19/16 7:10 AM, David Holmes wrote: >>>>>>> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>>>>>>> | >>>>>>>> Hi Cheleswer, I think Martin is suggesting something like: >>>>>>>> | >>>>>>>> >>>>>>>> // Use a modest stack size, unless requested otherwise: >>>>>>>> long stackSize = >>>>>>>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : >>>>>>>> 32768; >>>>>>> >>>>>>> Someone from core-libs needs to chime on what the appropriate >>>>>>> namespace for such a property would be. >>>>>>> >>>>>>> David >>>>>>> ----- >>>>>>> >>>>>>>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>>>>>>> reaper", stackSize); >>>>>>>> >>>>>>>> ||| >>>>>>>> If that tests OK for you, it may be the way we can go ahead >>>>>>>> with the >>>>>>>> point fix for this. >>>>>>>> >>>>>>>> Regards >>>>>>>> Kevin >>>>>>>> | >>>>>>>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>>>>>>> Historical note - I never liked having a reaper thread for each >>>>>>>>> subprocess, but no other reliable implementation is known. Given >>>>>>>>> the >>>>>>>>> potential for many reaper threads, I at least wanted to keep the >>>>>>>>> memory waste low. >>>>>>>>> >>>>>>>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> + Thread t = null; >>>>>>>>>> + if >>>>>>>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>>>> grimReaper, >>>>>>>>>> "process reaper"); >>>>>>>>>> + } else { >>>>>>>>>> + // Our thread stack requirement is quite >>>>>>>>>> modest. >>>>>>>>>> + t = new Thread(systemThreadGroup, >>>>>>>>>> grimReaper, >>>>>>>>>> "process reaper", 32768); >>>>>>>>>> + } >>>>>>>>> If we do end up using this strategy, cleaner would be to use >>>>>>>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> stackSize - the desired stack size for the new thread, or zero to >>>>>>>>> indicate that this parameter is to be ignored. >>>>>>>> >>>>>> >>> >> From amaembo at gmail.com Thu Jan 21 16:07:55 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Thu, 21 Jan 2016 22:07:55 +0600 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> Message-ID: <1563202311.20160121220755@gmail.com> Hello! Thank you for review! I did very small change in unit test comments to make them less confusing. Here's updated webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r3/ With best regards, Tagir Valeev. SC> I'm happy with the logic and specification of this proposal. I think it SC> will be a useful addition. SC> I'll let the Oracle team chime in to do a further review. SC> thanks SC> Stephen SC> On 16 January 2016 at 13:31, Tagir F. Valeev wrote: >> Hello! >> >> Thanks for review! Here's the updated patch: >> http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ >> >> SC> The docs say that if the end date is before the start date, the >> SC> stream is empty. While I can see just about why LongStream.range() >> SC> works that way, I don't think this API should. The minimum is an >> SC> exception, but it would be easy to support negative in the >> SC> days-only case or the months-only case. The problem is where there >> SC> are both months/years and days and those should be exceptions. >> >> Now datesUntil(endExclusive) throws an IllegalArgumentException if end >> date is before start date. >> >> datesUntil(endExclusive, step) supports negative periods. It throws >> IllegalArgumentException if: >> - step is zero >> - step.toTotalMonths() and step.getDays() both non-zero and have >> opposite sign >> - step is negative and end date is after start date >> - step is positive and end date is before start date >> >> Otherwise it works correctly: you can use >> LocalDate.of(2016, 1, 1) >> .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); >> >> Steps like Period.of(-1, -1, -1) are also supported. >> >> SC> The single-arg implementation uses plusDays() with an >> SC> incrementing number. When the performance patch goes in, the >> SC> proposed streaming implementation will be optimal for small >> SC> streams but less optimal for large ones. The fastest way to loop >> SC> over a list of dates would be to manually generate them by >> SC> incrementing the day until it exceeds the length of month, and so >> SC> on. However, this would be serial. >> >> As I understand, plusDays performance patch is already pushed. >> >> It's possible to write custom Spliterator which would use >> previous.plusDays(1) in tryAdvance() and jump (via >> LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is >> requested. However this adds at least one additional class and more >> complexity. I guess, such optimization can be considered later as >> separate issue when API is stabilized. >> >> SC> As such, I think the best way to write this, taking account of >> SC> how plusDays() is implemented, is as follows: >> >> SC> LongStream.range(start.toEpochDay(), >> SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); >> >> Ok, now it's done this way. >> >> SC> In the period-based method, it would be best to capture the case >> SC> where totalMonths == 0 and days > 0 and forward to another method >> SC> that ignores months. That private method can share implementation >> SC> with the public single-arg method (passing in 1). >> >> This case still more complex than one-day case as it requires division >> and multiplication. Thus I'd keep these case separately. However I >> simplified "months = 0" path using ofEpochDay, now it should be >> faster. >> >> SC> The docs for the period-based method should specify the strategy >> SC> that produces the results (in abstract terms). This needs to cover >> SC> that the result is equivalent to always adding to the start period >> SC> a multiple of the period. >> >> I added some clarifications, please check. >> >> SC> Some nits: >> SC> I prefer to avoid @link in the first sentence. Just using 'stream' >> would be sufficient. >> >> Done. >> >> SC> The first sentence should be a summary. In this case it probably has a >> bit too much detail. >> >> Done. >> >> SC> The @return has 'values' on a new line when it could be on the same >> line. >> >> I set now line length = 100 characters in my IDE. Is it ok? >> >> SC> If statements need braces. >> >> Done. >> >> With best regards, >> Tagir Valeev. >> >> From Alan.Bateman at oracle.com Thu Jan 21 16:12:10 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Jan 2016 16:12:10 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> Message-ID: <56A1035A.9020805@oracle.com> On 20/01/2016 17:56, Steve Drach wrote: > Hi, > > This is a repeat of the RFR I sent last Wed (Jan 13). > > : > > Webrev: http://cr.openjdk.java.net/~sdrach/8132734/webrev.02/ > Overall I think the API looks much better. For Release then I have to admit that I dislike _9 and wonder if other options were considered? javax.lang.model.SourceVersion uses the RELEASE_xx convention for example. Also I wonder about Release.ROOT and whether Release.UNVERSIONED was considered? In general the phrase "root entry" in the javadoc makes me think the root or top-most directory. An alternative that might be clearer is to say "unversioned entry" and define that term clearly in the class description. The javadoc for Release.RUNTIME looks like it will have a javadoc link to jdk.Version but that is a JDK-specific API. Could the text starting "The effective runtime ..." move to an @implNote? I assume @since will be added to Release before this is pushed. The new JarFile ctor doesn't seem to handle the case that version is null when multi release is forced. Also I assume it should specify @throws SecurityException. Could the legacy JarFile ctor be changed to this(file, verify, mode, Release.ROOT) to avoid duplication? I don't have time to do a detailed pass over the updated tests but I wonder if SimpleHttpServer is really a candidate to put in the testlibrary tree. It looks like it is very specific to multi-release JARs and so I would expect to be co-located with those tests rather than being a hazard in the testlibrary tree. A small comment is that it would be good to fix the really long lines before these changes are pushed. This will help future side-by-side reviews where it would be otherwise annoying to have to do horizontal scrolling to view diffs. -Alan. From martinrb at google.com Thu Jan 21 16:38:45 2016 From: martinrb at google.com (Martin Buchholz) Date: Thu, 21 Jan 2016 08:38:45 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: Message-ID: PIng. How about Alan, Chris, Xueming ? On Tue, Jan 19, 2016 at 9:24 AM, Martin Buchholz wrote: > Hi Stuart et al, > Please review my fix for this corner case bug, including more > importantly some performance improvements. > > https://bugs.openjdk.java.net/browse/JDK-8146568 > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/ArrayList-grow/ From mark.sheppard at oracle.com Thu Jan 21 17:45:28 2016 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Thu, 21 Jan 2016 17:45:28 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <569FCF34.8020401@oracle.com> References: <568FF6A5.4080700@oracle.com> <569FB2D3.6080607@oracle.com> <569FCF34.8020401@oracle.com> Message-ID: <56A11938.8040804@oracle.com> Thanks Sean i can revert the isClosed(), I put them in to make the access consistent regards Mark On 20/01/2016 18:17, Se?n Coffey wrote: > Hi Mark, > > SelectorImpl.java: > > line 125, could you use a 2 arg method call to dprint. It'll print the > stacktrace instead : > dprint(".close: selector.close: " + t); --> dprint(".close: > selector.close", t); > > The "while (!isClosed()) " change introduces a new hot lock on closed > variable. Hopefully, it won't impact performance too much. > > Looks good to me otherwise. > > Regards, > Sean. > > On 20/01/16 16:16, Mark Sheppard wrote: >> Hi, >> an update has been made to the webrev >> >> http://cr.openjdk.java.net/~msheppar/8144144/webrev.03/ >> >> an anomaly was found in the select loop of the SelectorImpl.run() method >> some defensive programming, for selector null references, have been >> added, also. >> >> regards >> Mark >> >> On 08/01/2016 17:49, Mark Sheppard wrote: >>> Hi >>> please oblige and review the following changes >>> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ >>> >>> which addresses the issue >>> https://bugs.openjdk.java.net/browse/JDK-8144144 >>> >>> the changes ensure that an Acceptor and its associated >>> ServerSocket/ServerSocketChannel >>> are closed. >>> >>> regards >>> Mark >> > From steve.drach at oracle.com Thu Jan 21 18:02:54 2016 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 21 Jan 2016 10:02:54 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56A1035A.9020805@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> Message-ID: <1641DFFF-D00F-4108-B40D-7E042487CA48@oracle.com> Thank you for the review Alan. See comments in line below. > Overall I think the API looks much better. With the advantage of being much simpler too. > For Release then I have to admit that I dislike _9 and wonder if other options were considered? javax.lang.model.SourceVersion uses the RELEASE_xx convention for example. I suspected this is a bike shed candidate. I think Release._9 is nicer and it conveys the same information in a less cluttered way than Release.RELEASE_9. > > Also I wonder about Release.ROOT and whether Release.UNVERSIONED was considered? In general the phrase "root entry" in the javadoc makes me think the root or top-most directory. An alternative that might be clearer is to say "unversioned entry" and define that term clearly in the class description. The entries in a legacy jar (the only entries) or in the unversioned section of a multi-release jar are directly under the top-most directory > > The javadoc for Release.RUNTIME looks like it will have a javadoc link to jdk.Version but that is a JDK-specific API. Could the text starting "The effective runtime ..." move to an @implNote? Okay > > I assume @since will be added to Release before this is pushed. Yes. > > The new JarFile ctor doesn't seem to handle the case that version is null when multi release is forced. Also I assume it should specify @throws SecurityException. Both will be fixed. > > Could the legacy JarFile ctor be changed to this(file, verify, mode, Release.ROOT) to avoid duplication? Yes. > > I don't have time to do a detailed pass over the updated tests but I wonder if SimpleHttpServer is really a candidate to put in the testlibrary tree. It looks like it is very specific to multi-release JARs and so I would expect to be co-located with those tests rather than being a hazard in the testlibrary tree. It?s in the testlibrary under java/util/jar with the other multi-release specific test ?helper? classes. I could make it even more specific by putting it under a java/util/jar/multi-release directory > > A small comment is that it would be good to fix the really long lines before these changes are pushed. This will help future side-by-side reviews where it would be otherwise annoying to have to do horizontal scrolling to view diffs. Do we really have to stick with 80 column hollerith card semantics? Even that was changed to 96 columns about 50 years ago. The one line, other than some ?fixmes" that will be removed when JEP 223 is integrated, that exceeds 96 characters long will be changed by wrapping it to 94 columns. > From Alan.Bateman at oracle.com Thu Jan 21 19:37:56 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 21 Jan 2016 19:37:56 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <1641DFFF-D00F-4108-B40D-7E042487CA48@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <1641DFFF-D00F-4108-B40D-7E042487CA48@oracle.com> Message-ID: <56A13394.7030600@oracle.com> On 21/01/2016 18:02, Steve Drach wrote: > : > I suspected this is a bike shed candidate. I think Release._9 is nicer and it conveys the same information in a less cluttered way than Release.RELEASE_9. Yes a bike shed, I'm just saying that Release._9 looks odd/inconsistent when we have SourceVersion.RELEASE_9 elsewhere. Maybe there has been discussion on this topic already. With a static import then RELEASE_9 isn't too bad. > : > The entries in a legacy jar (the only entries) or in the unversioned section of a multi-release jar are directly under the top-most directory All I'm saying is that Release.ROOT doesn't feel quite right, esp. when ROOT is defined as the unversioned entries. > : >> I don't have time to do a detailed pass over the updated tests but I wonder if SimpleHttpServer is really a candidate to put in the testlibrary tree. It looks like it is very specific to multi-release JARs and so I would expect to be co-located with those tests rather than being a hazard in the testlibrary tree. > It?s in the testlibrary under java/util/jar with the other multi-release specific test ?helper? classes. I could make it even more specific by putting it under a java/util/jar/multi-release directory Yes, it needs to move to somewhere specific because it's not general purpose. > : > Do we really have to stick with 80 column hollerith card semantics? Even that was changed to 96 columns about 50 years ago. The one line, other than some ?fixmes" that will be removed when JEP 223 is integrated, that exceeds 96 characters long will be changed by wrapping it to 94 columns. I didn't mention 80. If you looks at the sdiffs for URLClassPath and JarFile when the outliers should be obvious. All I can suggest is to keep thing consistent with the existing code where possible. -Alan From stuart.marks at oracle.com Thu Jan 21 19:39:40 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 21 Jan 2016 11:39:40 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: Message-ID: <56A133FC.4040806@oracle.com> Hi Martin, Thanks for digging into this. There are some subtle issues here. I have a few questions. One is that in list.addAll(other), the sizes of list and other exceeds Integer.MAX_VALUE, then grow(int) will be called with a negative value for minCapacity. The code *seems* to handle this ok, but the negative minCapacity value can get pretty deeply into the helper methods before being caught. Would it be better to check it at the top of grow(int) and throw an exception there? (Probably OOME.) I think it would make the subsequent code easier to follow. It looks like there are a variety of ways for minCapacity that is positive but greater than MAX_ARRAY_SIZE to reach newCapacity(). If this occurs, and other conditions are right, it looks like the code will end up attempting to allocate an array greater than MAX_ARRAY_SIZE. One style point I noticed (which might be an issue of me not being used to it) is the use of an elementData local variable to shadow the elementData field. This is more-or-less ok in places where elementData is initialized from this.elementData, but in readObject(), the local elementData is introduced in a nested scope. This means that elementData has different meanings in different parts of the method. For the test Bug8146568 I think the preferred way to disable a test with extreme memory requirements is to use @ignore; see jdk/test/java/math/BigInteger/SymmetricRangeTests.java for example. s'marks On 1/21/16 8:38 AM, Martin Buchholz wrote: > PIng. How about Alan, Chris, Xueming ? > > On Tue, Jan 19, 2016 at 9:24 AM, Martin Buchholz wrote: >> Hi Stuart et al, >> Please review my fix for this corner case bug, including more >> importantly some performance improvements. >> >> https://bugs.openjdk.java.net/browse/JDK-8146568 >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/ArrayList-grow/ From martinrb at google.com Thu Jan 21 21:57:58 2016 From: martinrb at google.com (Martin Buchholz) Date: Thu, 21 Jan 2016 13:57:58 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: <56A133FC.4040806@oracle.com> References: <56A133FC.4040806@oracle.com> Message-ID: On Thu, Jan 21, 2016 at 11:39 AM, Stuart Marks wrote: > Hi Martin, > > Thanks for digging into this. There are some subtle issues here. I have a > few questions. > > One is that in list.addAll(other), the sizes of list and other exceeds > Integer.MAX_VALUE, then grow(int) will be called with a negative value for > minCapacity. The code *seems* to handle this ok, but the negative > minCapacity value can get pretty deeply into the helper methods before being > caught. Would it be better to check it at the top of grow(int) and throw an > exception there? (Probably OOME.) I think it would make the subsequent code > easier to follow. It's true that the code is rather tricky, "overflow-conscious code". But this is ArrayList, so it seems worth optimizing even for grow. The common case is that we grow by 50% and then if (newCapacity - MAX_ARRAY_SIZE <= 0) we can be sure that newCapacity is not negative. > It looks like there are a variety of ways for minCapacity that is positive > but greater than MAX_ARRAY_SIZE to reach newCapacity(). If this occurs, and > other conditions are right, it looks like the code will end up attempting to > allocate an array greater than MAX_ARRAY_SIZE. If grow(n) is called with MAX_ARRAY_SIZE < n <= MAX_VALUE, then we no choice but to allocate an array of that size! It's only when we use the grow-by-50% strategy that we can change our minds by scaling back. I don't see a bug. > One style point I noticed (which might be an issue of me not being used to > it) is the use of an elementData local variable to shadow the elementData > field. This is more-or-less ok in places where elementData is initialized > from this.elementData, but in readObject(), the local elementData is > introduced in a nested scope. This means that elementData has different > meanings in different parts of the method. Yeah, elementData is not great but I couldn't find anything better. "a" is already taken. "snapshot" has the wrong connotations. If you prefer e.g. "elements" I will change it throughout, but in either case a reader needs to understand that "elements" and "elementData" are "almost" the same. > For the test Bug8146568 I think the preferred way to disable a test with > extreme memory requirements is to use @ignore; see I've never liked @ignore in practice, because jtreg is very noisy unless you also say -ignore:quiet (which I always do, but does everyone else?) From rafael.wth at gmail.com Thu Jan 21 22:14:29 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Thu, 21 Jan 2016 23:14:29 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A09FDD.9080305@redhat.com> References: <56A09FDD.9080305@redhat.com> Message-ID: Hi Andrew, if there is any update on the matter, I would of course love to hear about it. Unfortunately, I never received an answer to my question, but maybe I picked the wrong list. Thank you for your support on this issue! Regards, Rafael 2016-01-21 10:07 GMT+01:00 Andrew Dinn : > Hi Rafael, > > On 01/12/15 12:45, Rafael Winterhalter wrote: >> I can tell that this behavior has not only affected me as I had this >> question comming up by multiple users of my open-source code generation >> library. >> >> What is your view on this? >> >> Thank you for your feedback! > > I have an interest in this issue because I lead a Java agent-based > project (Byteman). I have not (yet) come across a case where the > behaviour you describe has affected the operation of my agent -- I > suspect that is because it will never attempt to transform a class > defined via Unsafe::defineAnonymousClass. However, I agree with you that > this behaviour appears anomalous and the outcome you describe is > counter-intuitive. I would be very interested to know if there was a > good reason for excluding anonymous classes from transformation. > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in UK and Wales under Company Registration No. 3798903 > Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul > Argiry (US) From stuart.marks at oracle.com Thu Jan 21 22:36:18 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 21 Jan 2016 14:36:18 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: <56A133FC.4040806@oracle.com> Message-ID: <56A15D62.6020303@oracle.com> On 1/21/16 1:57 PM, Martin Buchholz wrote: >> One is that in list.addAll(other), the sizes of list and other exceeds >> Integer.MAX_VALUE, then grow(int) will be called with a negative value for >> minCapacity. The code *seems* to handle this ok, but the negative >> minCapacity value can get pretty deeply into the helper methods before being >> caught. Would it be better to check it at the top of grow(int) and throw an >> exception there? (Probably OOME.) I think it would make the subsequent code >> easier to follow. > > It's true that the code is rather tricky, "overflow-conscious code". > But this is ArrayList, so it seems worth optimizing even for grow. > > The common case is that we grow by 50% and then if (newCapacity - > MAX_ARRAY_SIZE <= 0) we can be sure that newCapacity is not negative. The code may be correct, but I'm concerned about maintenance. If things shift around, it might be easy to miss the possibility that negative minCapacity could be passed to grow() if overflow had occurred. So perhaps at least a comment would be warranted. >> It looks like there are a variety of ways for minCapacity that is positive >> but greater than MAX_ARRAY_SIZE to reach newCapacity(). If this occurs, and >> other conditions are right, it looks like the code will end up attempting to >> allocate an array greater than MAX_ARRAY_SIZE. > > If grow(n) is called with MAX_ARRAY_SIZE < n <= MAX_VALUE, then we no > choice but to allocate an array of that size! It's only when we use > the grow-by-50% strategy that we can change our minds by scaling back. > I don't see a bug. Ah, MAX_ARRAY_SIZE applies only to grow-by-50%, not to all array allocations. Perhaps it was my mistake for having believed its comment, which is The maximum size of array to allocate. You know what they say about comments not matching the code.... :-) I do think this comment needs to be adjusted to say that MAX_ARRAY_SIZE applies only to the 50% growth policy. I was certainly misled by it. >> One style point I noticed (which might be an issue of me not being used to >> it) is the use of an elementData local variable to shadow the elementData >> field. This is more-or-less ok in places where elementData is initialized >> from this.elementData, but in readObject(), the local elementData is >> introduced in a nested scope. This means that elementData has different >> meanings in different parts of the method. > > Yeah, elementData is not great but I couldn't find anything better. > "a" is already taken. "snapshot" has the wrong connotations. If you > prefer e.g. "elements" I will change it throughout, but in either case > a reader needs to understand that "elements" and "elementData" are > "almost" the same. I don't think a global change is necessary, as the prevailing style in this file is to use the elementData field or to have a local elementData that's an alias of the field. I think readObject() is the outlier for using both the field and the local variable. But there are several other funny things going on here in readObject()... well I won't insist that you address them right now, as they're a distraction from this bugfix. So the change as you've proposed is fine. (But let me know if you're interested in discussing readObject() further.) >> For the test Bug8146568 I think the preferred way to disable a test with >> extreme memory requirements is to use @ignore; see > > I've never liked @ignore in practice, because jtreg is very noisy > unless you also say > -ignore:quiet > (which I always do, but does everyone else?) Yes, I think jtreg's default behavior has taught everyone, including our automated systems, to use -ignore:quiet. So I think @ignore is fine. Thanks. s'marks From paul.sandoz at oracle.com Thu Jan 21 22:42:47 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 21 Jan 2016 23:42:47 +0100 Subject: API review of VarHandles Message-ID: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Hi This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. The specdiff can be found here: http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). Paul. From felix.yang at oracle.com Thu Jan 21 22:55:30 2016 From: felix.yang at oracle.com (Felix Yang) Date: Thu, 21 Jan 2016 14:55:30 -0800 Subject: RFR 8065076/9, test/java/net/SocketPermission/SocketPermissionTest.java failed intermittently In-Reply-To: References: <568F74F7.70100@oracle.com> <56973B1A.30000@oracle.com> Message-ID: <67D3FEF2-7B00-4C90-8D87-6258B7AE547A@oracle.com> Hi Chris, your fix is cool. I will assign the bug to you:) a comment on this fix. The test changed system SecurityManager and it is not executed with othervm mode. I think you need to rollback the change after test. Otherwise it may affect other tests. Thanks, Felix > On Jan 20, 2016, at 12:45 PM, Chris Hegarty wrote: > > On 20 Jan 2016, at 06:36, Chris Hegarty > wrote: > >> Felix, >> >> On 14 Jan 2016, at 06:07, Felix Yang wrote: >> >>> Hi all, >>> please review the fix for test/java/net/SocketPermission/SocketPermissionTest.java, which fails frequently with "java.net.BindException: Address already in use". >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065076 >>> Webrev: http://cr.openjdk.java.net/~xiaofeya/8065076/webrev.00 >> >> My preference is to avoid getFreePort. It is problematic and I believe just obfuscates >> the intermittent failures further. >> >> In many of the test scenarios the ?listening? socket can be created before the specific >> access control context and associated permission are created. >> >> I?ll see if I can get some time to try this out. > > I spent a little time on this today. I basically rewrote the test, but kept the > same test scenarios. The use of data providers was cute, but not workable > since there is no common supertype for the socket classes. I decided to > just expand out the test cases manually. This will give the same test > coverage, but should be stable since it creates the sockets first, on an > ephemeral port, and then constructs the permissions appropriately given > that port. > > http://cr.openjdk.java.net/~chegar/8065076/ > > The webrev diffs are almost useless, just review the new file, and compare > test scenarios against the what is in the old file. > > -Chris. From david.lloyd at redhat.com Thu Jan 21 23:05:53 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 21 Jan 2016 17:05:53 -0600 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <56A16451.6010709@redhat.com> On 01/21/2016 04:42 PM, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). Language like that used in the VarHandle documentation is very precise and relevant to a JVM maintainer. But from a use perspective, this is terrible! Are we really expecting users to make sense of this API? I can tell you that as a user, I'd be looking for methods that look and act just like what I see on the existing Atomic*FieldUpdater style classes. Understanding the dissertation that is the description of VarHandles as a prerequisite for using this feature is pretty rough for users. In summary, this API might be expedient implementation-wise (I mean, it must be, right?), but in terms of style, future comprehensibility, ease of use, understandability, consistency with existing constructs, and (I can only imagine) implementation complexity, this monster cannot possibly be the best we can do. I am baffled as to how the original language syntax proposal of using some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two new bytecodes was considered unacceptable; looking at this API, I know that none of the aforementioned metrics were considered as acceptance criteria. How did we get to this strange place? Is it just that MethodHandles are the latest shiny golden hammer? -- - DML From steve.drach at oracle.com Thu Jan 21 23:49:09 2016 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 21 Jan 2016 15:49:09 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56A13394.7030600@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <1641DFFF-D00F-4108-B40D-7E042487CA48@oracle.com> <56A13394! .7030600@oracle.com> Message-ID: <2A396200-9571-4236-985C-59F2BF9F136C@oracle.com> >> I suspected this is a bike shed candidate. I think Release._9 is nicer and it conveys the same information in a less cluttered way than Release.RELEASE_9. > Yes a bike shed, I'm just saying that Release._9 looks odd/inconsistent when we have SourceVersion.RELEASE_9 elsewhere. Maybe there has been discussion on this topic already. With a static import then RELEASE_9 isn't too bad. I?ll leave this as an open issue for awhile in case I get another reviewer that feels as strongly about it you do, or as I do. > >> : >> The entries in a legacy jar (the only entries) or in the unversioned section of a multi-release jar are directly under the top-most directory > All I'm saying is that Release.ROOT doesn't feel quite right, esp. when ROOT is defined as the unversioned entries. How about Release.BASE for base entries? > >> : >>> I don't have time to do a detailed pass over the updated tests but I wonder if SimpleHttpServer is really a candidate to put in the testlibrary tree. It looks like it is very specific to multi-release JARs and so I would expect to be co-located with those tests rather than being a hazard in the testlibrary tree. >> It?s in the testlibrary under java/util/jar with the other multi-release specific test ?helper? classes. I could make it even more specific by putting it under a java/util/jar/multi-release directory > Yes, it needs to move to somewhere specific because it's not general purpose. I moved it to the same file as the test itself, making it a package-private class >> Do we really have to stick with 80 column hollerith card semantics? Even that was changed to 96 columns about 50 years ago. The one line, other than some ?fixmes" that will be removed when JEP 223 is integrated, that exceeds 96 characters long will be changed by wrapping it to 94 columns. > I didn't mention 80. If you looks at the sdiffs for URLClassPath and JarFile when the outliers should be obvious. All I can suggest is to keep thing consistent with the existing code where possible. I only found one instance in JarFile and one instance in URLClassPath that seemed too long. I wrapped them both to be less than 95 columns. Even for short lines when using sdiff I sometimes need to ?left scroll? ? not sure that making it sdiff friendly is a reasonable constraint. From vitalyd at gmail.com Fri Jan 22 00:01:47 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 21 Jan 2016 19:01:47 -0500 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: I think the get/setOpaque methods need a bit more explanation ("opaque" is an odd naming choice, IMO). Specifically, it says the operations are done in program order but have no effect on inter-thread ordering/visibility. Is this spec verbiage for a full compiler-only fence? On Thursday, January 21, 2016, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and > pushes will occur separately, and flow through the hs-comp repo, most > likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just > ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was > previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: > byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to > revisit that functionality w.r.t. alignment and proposed enhancements to > ByteBuffer (in discussion on valhalla-dev). > > Paul. > > > -- Sent from my phone From mandy.chung at oracle.com Fri Jan 22 00:39:57 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 21 Jan 2016 16:39:57 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <2A396200-9571-4236-985C-59F2BF9F136C@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <1641DFFF-D00F-4108-B40D-7E042487CA48@oracle.com> <56A13394! ! .7030600@oracle.com> <2A396200-9571-4236-985C-59F2BF9F136C@oracle.com> Message-ID: > On Jan 21, 2016, at 3:49 PM, Steve Drach wrote: > >>> I suspected this is a bike shed candidate. I think Release._9 is nicer and it conveys the same information in a less cluttered way than Release.RELEASE_9. >> Yes a bike shed, I'm just saying that Release._9 looks odd/inconsistent when we have SourceVersion.RELEASE_9 elsewhere. Maybe there has been discussion on this topic already. With a static import then RELEASE_9 isn't too bad. > > I?ll leave this as an open issue for awhile in case I get another reviewer that feels as strongly about it you do, or as I do. > I only started looking at some files on the webrev. Release._9 catches my attention too and it looks very odd. I think RELEASE_9 is a much better constant name than _9. Mandy From brian.goetz at oracle.com Fri Jan 22 01:32:29 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Thu, 21 Jan 2016 20:32:29 -0500 Subject: API review of VarHandles In-Reply-To: <56A16451.6010709@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> Message-ID: <56A186AD.8090100@oracle.com> > I am baffled as to how the original language syntax proposal of using > some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two > new bytecodes was considered unacceptable; looking at this API, I know > that none of the aforementioned metrics were considered as acceptance > criteria. How did we get to this strange place? The path was actually quite obvious. The first question was, "OK, if the language had this magic syntax, what bytecode would get generated? And clearly it couldn't be Unsafe. So either a pile of new bytecodes, or an API, was needed that the front-end compiler could generate against. Building APIs is generally preferable to new bytecodes, if an API can do the job, and it turned out that it could do it just as well. Once we had an API that met the requirements, it was pretty clear that language syntax was not only unnecessary, but likely undesirable -- these exotic access modes are power tools for power users (do you really want typical users to reason about mixed volatile and relaxed access to variables?), and they meet the needs of such users just fine (those users have been using Unsafe all along, and this is clearly better and safer.) So it was not the either-or of "either an API *or* magic language syntax", it was "either an API *or* syntax plus an API". At which point it was clear that the syntax didn't carry its own weight. I think this is a fine example of how the obvious idea was wrong, and that most of the value of the initial syntax idea was framing the problem space. I'm glad we saw fit to jettison that when it ceased to provide value. From martinrb at google.com Fri Jan 22 02:58:25 2016 From: martinrb at google.com (Martin Buchholz) Date: Thu, 21 Jan 2016 18:58:25 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: <56A15D62.6020303@oracle.com> References: <56A133FC.4040806@oracle.com> <56A15D62.6020303@oracle.com> Message-ID: We have a new webrev. Bug8146568.java now uses @ignore. readObject has a minor rewrite, only assigning to elementData once. (Yes, we can talk more about future improvements to ArrayList) (maybe MAX_ARRAY_SIZE could have a better name ...) We have some hopefully clearer internal comments: /** * The maximum size of array to allocate (unless necessary). * Some VMs reserve some header words in an array. * Attempts to allocate larger arrays may result in * OutOfMemoryError: Requested array size exceeds VM limit */ private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; /** * Increases the capacity to ensure that it can hold at least the * number of elements specified by the minimum capacity argument. * * @param minCapacity the desired minimum capacity * @throws OutOfMemoryError if minCapacity is less than zero */ private Object[] grow(int minCapacity) { return elementData = Arrays.copyOf(elementData, newCapacity(minCapacity)); } private Object[] grow() { return grow(size + 1); } /** * Returns a capacity at least as large as the given minimum capacity. * Returns the current capacity increased by 50% if that suffices. * Will not return a capacity greater than MAX_ARRAY_SIZE unless * the given minimum capacity is greater than MAX_ARRAY_SIZE. * * @param minCapacity the desired minimum capacity * @throws OutOfMemoryError if minCapacity is less than zero */ private int newCapacity(int minCapacity) { On Thu, Jan 21, 2016 at 2:36 PM, Stuart Marks wrote: > > > On 1/21/16 1:57 PM, Martin Buchholz wrote: >>> >>> One is that in list.addAll(other), the sizes of list and other exceeds >>> Integer.MAX_VALUE, then grow(int) will be called with a negative value >>> for >>> minCapacity. The code *seems* to handle this ok, but the negative >>> minCapacity value can get pretty deeply into the helper methods before >>> being >>> caught. Would it be better to check it at the top of grow(int) and throw >>> an >>> exception there? (Probably OOME.) I think it would make the subsequent >>> code >>> easier to follow. >> >> >> It's true that the code is rather tricky, "overflow-conscious code". >> But this is ArrayList, so it seems worth optimizing even for grow. >> >> The common case is that we grow by 50% and then if (newCapacity - >> MAX_ARRAY_SIZE <= 0) we can be sure that newCapacity is not negative. > > > The code may be correct, but I'm concerned about maintenance. If things > shift around, it might be easy to miss the possibility that negative > minCapacity could be passed to grow() if overflow had occurred. So perhaps > at least a comment would be warranted. > >>> It looks like there are a variety of ways for minCapacity that is >>> positive >>> but greater than MAX_ARRAY_SIZE to reach newCapacity(). If this occurs, >>> and >>> other conditions are right, it looks like the code will end up attempting >>> to >>> allocate an array greater than MAX_ARRAY_SIZE. >> >> >> If grow(n) is called with MAX_ARRAY_SIZE < n <= MAX_VALUE, then we no >> choice but to allocate an array of that size! It's only when we use >> the grow-by-50% strategy that we can change our minds by scaling back. >> I don't see a bug. > > > Ah, MAX_ARRAY_SIZE applies only to grow-by-50%, not to all array > allocations. Perhaps it was my mistake for having believed its comment, > which is > > The maximum size of array to allocate. > > You know what they say about comments not matching the code.... :-) > > I do think this comment needs to be adjusted to say that MAX_ARRAY_SIZE > applies only to the 50% growth policy. I was certainly misled by it. > >>> One style point I noticed (which might be an issue of me not being used >>> to >>> it) is the use of an elementData local variable to shadow the elementData >>> field. This is more-or-less ok in places where elementData is initialized >>> from this.elementData, but in readObject(), the local elementData is >>> introduced in a nested scope. This means that elementData has different >>> meanings in different parts of the method. >> >> >> Yeah, elementData is not great but I couldn't find anything better. >> "a" is already taken. "snapshot" has the wrong connotations. If you >> prefer e.g. "elements" I will change it throughout, but in either case >> a reader needs to understand that "elements" and "elementData" are >> "almost" the same. > > > I don't think a global change is necessary, as the prevailing style in this > file is to use the elementData field or to have a local elementData that's > an alias of the field. I think readObject() is the outlier for using both > the field and the local variable. But there are several other funny things > going on here in readObject()... well I won't insist that you address them > right now, as they're a distraction from this bugfix. So the change as > you've proposed is fine. > > (But let me know if you're interested in discussing readObject() further.) > >>> For the test Bug8146568 I think the preferred way to disable a test with >>> extreme memory requirements is to use @ignore; see >> >> >> I've never liked @ignore in practice, because jtreg is very noisy >> unless you also say >> -ignore:quiet >> (which I always do, but does everyone else?) > > > Yes, I think jtreg's default behavior has taught everyone, including our > automated systems, to use -ignore:quiet. So I think @ignore is fine. > > Thanks. > > s'marks From xueming.shen at oracle.com Fri Jan 22 03:44:12 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 21 Jan 2016 19:44:12 -0800 Subject: RFR JDK-8147531,To add named character construct \N{...} to support Unicode name property In-Reply-To: References: <569DEB35.7020006@oracle.com> Message-ID: <56A1A58C.9000907@oracle.com> On 1/19/16 11:43 AM, Martin Buchholz wrote: > Many years ago I considered implementing this cool feature. > I thought that few would find it worth the cost - it would be hard to > keep the cost low if this feature is used only rarely. You might want > an expiring cache of character name mappings, and the JDK doesn't have > such a thing yet. As a matter of fact. The compressed data file is about 130k in the file system. The inflated runtime data for the name string table is about 700k. The cp->name lookup table is about 160k and the name->cp lookup mapping is about 400k+ (there might be a little more space can be cut from the homemade hashmap...). So the overall runtime cost is about 1.2mb for this "cool" feature. Yes, it's a little bigger than the zt_tw charset, but consider you can have a round trip mapping between all the codepoints and their names, 1.3mb might not be that expensive, consider a "normal" pic now takes couple mb memory. How about you help take a look to see if we can squeeze out more space? really need a reviewer :-) -sherman > > (I haven't actually reviewed the implementation) > On Mon, Jan 18, 2016 at 11:52 PM, Xueming Shen wrote: >> Hi, >> >> Please help review the change to add \N support in regex. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 >> webrev: http://cr.openjdk.java.net/~sherman/8147531/webrev >> >> This is one of the items we were planning to address via JEP111 >> http://openjdk.java.net/jeps/111 >> https://bugs.openjdk.java.net/browse/JDK-8046101 >> >> Some of the constructs had been added already in early release. I'm >> planning to address the rest as individual rfe separately. >> >> Thanks, >> Sherman From amaembo at gmail.com Fri Jan 22 06:19:06 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 22 Jan 2016 12:19:06 +0600 Subject: Stream.findFirst for unordered source optimization Message-ID: <1152739344.20160122121906@gmail.com> Hello! Seems that currently Stream.findFirst is not optimized for unordered source. I think it should work as findAny in this case. Here's a small patch which fixes this: http://cr.openjdk.java.net/~tvaleev/patches/findFirst/find_patch.txt Simple JMH test: http://cr.openjdk.java.net/~tvaleev/patches/findFirst/FindTest.java Original: http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_orig.txt # JMH 1.11.2 (released 85 days ago) # VM version: JDK 9-ea, VM 9-ea+99-2015-12-23-183325.javare.4146.nc Benchmark Mode Cnt Score Error Units FindTest.findAny avgt 30 12439,631 ? 1787,866 us/op FindTest.findAnyUnordered avgt 30 12923,080 ? 1072,537 us/op FindTest.findFirst avgt 30 48047,467 ? 2713,489 us/op FindTest.findFirstUnordered avgt 30 52648,893 ? 3934,682 us/op Patched: http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_patched.txt Benchmark Mode Cnt Score Error Units FindTest.findAny avgt 30 11312,238 ? 386,627 us/op FindTest.findAnyUnordered avgt 30 12136,953 ? 1536,817 us/op FindTest.findFirst avgt 30 47517,776 ? 2844,607 us/op FindTest.findFirstUnordered avgt 30 13147,492 ? 1140,592 us/op If you think it's a reasonable thing to patch, I can log an issue, generate webrev and check whether jtreg tests pass. With best regards, Tagir Valeev. From peter.levart at gmail.com Fri Jan 22 08:09:19 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jan 2016 09:09:19 +0100 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <56A1E3AF.4010201@gmail.com> Hi Paul, I found a typo (which was probably caused by unconscious influence from jigsaw ;-). Search for "Java Memory Module"... Regards, Peter On 01/21/2016 11:42 PM, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). > > Paul. > > From adinn at redhat.com Fri Jan 22 09:11:40 2016 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 22 Jan 2016 09:11:40 +0000 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A09FDD.9080305@redhat.com> Message-ID: <56A1F24C.7050205@redhat.com> On 21/01/16 22:14, Rafael Winterhalter wrote: > Hi Andrew, > if there is any update on the matter, I would of course love to hear about it. > Unfortunately, I never received an answer to my question, but maybe I > picked the wrong list. > Thank you for your support on this issue! I'm pretty sure this is the right list. But I don't know who cold answer the question. I know that Coleen Phillmore (added explicitly to CC) often deals with ClassFileTransformer issues. Coleen, can you answer? regards, Andrew Dinn ----------- Senior Principal Software Engineer Red Hat UK Ltd Registered in UK and Wales under Company Registration No. 3798903 Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul Argiry (US) From shilpi.rastogi at oracle.com Fri Jan 22 09:21:50 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Fri, 22 Jan 2016 14:51:50 +0530 Subject: RFR 8138884:MethodHandles.Lookup.findVirtual() Javadoc fails to consider private interface methods In-Reply-To: <569F80B0.3040103@oracle.com> References: <569F80B0.3040103@oracle.com> Message-ID: <56A1F4AE.4050201@oracle.com> Gentle Reminder! -------- Original Message -------- Subject: [9u-dev] 8138884:MethodHandles.Lookup.findVirtual() Javadoc fails to consider private interface methods Date: Wed, 20 Jan 2016 18:12:24 +0530 From: shilpi rastogi Organization: Oracle Corporation To: core-libs-dev at openjdk.java.net Hi All, Please review my fix related to java doc. MethodHandles.Lookup.findVirtual() Javadoc fails to consider private methods of an interface. MethodHandles.Lookup.findVirtual() throws IllegalAccessException if the method MH is being created for is private and declared in an interface. Bug link- https://bugs.openjdk.java.net/browse/JDK-8138884 Webrev link- http://cr.openjdk.java.net/~srastogi/8138884/webrev.00/ Thanks, Shilpi From shilpi.rastogi at oracle.com Fri Jan 22 09:22:30 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Fri, 22 Jan 2016 14:52:30 +0530 Subject: Fwd: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <569F80B6.7080702@oracle.com> References: <569F80B6.7080702@oracle.com> Message-ID: <56A1F4D6.80303@oracle.com> Gentle Reminder! -------- Original Message -------- Subject: [9u-dev] 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods Date: Wed, 20 Jan 2016 18:12:30 +0530 From: shilpi rastogi Organization: Oracle Corporation To: core-libs-dev at openjdk.java.net Hi All, Please review my fix related to java doc. MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods- MethodHandles.Lookup.findSpecial() throws IllegalAccessException if the target method is static. Bug link- https://bugs.openjdk.java.net/browse/JDK-8138578 Webrev link- http://cr.openjdk.java.net/~srastogi/8138578/webrev.00/ Thnaks, Shilpi From paul.sandoz at oracle.com Fri Jan 22 09:26:44 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jan 2016 10:26:44 +0100 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <1641DFFF-D00F-4108-B40D-7E042487CA48@oracle.com> <56A13394! ! ! .7030600@oracle.com> <2A396200-9571-4236-985C-59F2BF9F136C@oracle.com> Message-ID: <9AF84AF5-49AA-47FD-A057-4525EB384CF2@oracle.com> > On 22 Jan 2016, at 01:39, Mandy Chung wrote: > > >> On Jan 21, 2016, at 3:49 PM, Steve Drach wrote: >> >>>> I suspected this is a bike shed candidate. I think Release._9 is nicer and it conveys the same information in a less cluttered way than Release.RELEASE_9. >>> Yes a bike shed, I'm just saying that Release._9 looks odd/inconsistent when we have SourceVersion.RELEASE_9 elsewhere. Maybe there has been discussion on this topic already. With a static import then RELEASE_9 isn't too bad. >> >> I?ll leave this as an open issue for awhile in case I get another reviewer that feels as strongly about it you do, or as I do. >> > > I only started looking at some files on the webrev. Release._9 catches my attention too and it looks very odd. I think RELEASE_9 is a much better constant name than _9. > While there is a some naming activity over what to call such constants, i think the use a ?_? as the first character of a public API artefact should be strongly discouraged, such usages are more commonly associated with internal artefacts or generated code and using such a style for public artefacts sets a ?bad" precedence IMO (first use in the Java APIs AFAICT). There needs to be a really strong justification for such public use and at the moment i don?t see one here. Paul. From paul.sandoz at oracle.com Fri Jan 22 09:30:29 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jan 2016 10:30:29 +0100 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <56A1F4D6.80303@oracle.com> References: <569F80B6.7080702@oracle.com> <56A1F4D6.80303@oracle.com> Message-ID: Hi Shilpi, Can you also double check Lookup.unreflectSpecial, it?s documentation might require updating too. Separating the ?or? statements with a comma would help readability in the JavaDoc (same applies to the the patch for findVirtual). Paul. > On 22 Jan 2016, at 10:22, shilpi rastogi wrote: > > > Gentle Reminder! > > -------- Original Message -------- > Subject: [9u-dev] 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods > Date: Wed, 20 Jan 2016 18:12:30 +0530 > From: shilpi rastogi > Organization: Oracle Corporation > To: core-libs-dev at openjdk.java.net > > > > Hi All, > > Please review my fix related to java doc. > > MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods- > > MethodHandles.Lookup.findSpecial() throws IllegalAccessException if the target method is static. > > Bug link- https://bugs.openjdk.java.net/browse/JDK-8138578 > Webrev link- http://cr.openjdk.java.net/~srastogi/8138578/webrev.00/ > > Thnaks, > Shilpi > > From aleksey.shipilev at oracle.com Fri Jan 22 09:31:04 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 12:31:04 +0300 Subject: JEP 280 (Indify String Concat) integration Message-ID: <56A1F6D8.9070808@oracle.com> Hi, I think it is time to integrate JEP 280 ("Indify String Concat") into JDK 9: http://openjdk.java.net/jeps/280 The JEP is Targeted, the CCC is approved, the code reviews and pre-integration checks come clean. In theory, we would like to integrate with some "optimal" concat strategy switched by default, but we have to wait for some Hotspot compiler changes to settle before we can make that call. (Notably, https://bugs.openjdk.java.net/browse/JDK-8136469). So, I would like to proceed in two parts: Part 1: Integrate all code changes, but keep the concatenation strategy to be the same "naive" StringBuilder.append chains. In other words, we are emitting string concat indy, but always link in the naive StringBuilder.append code. Part 2: (After HS changes settle in) Re-run performance tests, and switch to a more optimal concatenation strategy. This amounts to one-liner change in a default value for Java property. Thoughts? I would like relevant Project Leads / Reviewers (langtools and jdk) to OKay the integration. FTR, the webrevs: http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ Thanks, -Aleksey From paul.sandoz at oracle.com Fri Jan 22 09:46:14 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jan 2016 10:46:14 +0100 Subject: JEP 280 (Indify String Concat) integration In-Reply-To: <56A1F6D8.9070808@oracle.com> References: <56A1F6D8.9070808@oracle.com> Message-ID: <2A6DF001-3692-49C6-BEDA-4D80023B8B47@oracle.com> Looks a fine approach to me. Getting code in earlier and tested, even initially under the less optimal strategy, is good in terms of exposure. Paul. > On 22 Jan 2016, at 10:31, Aleksey Shipilev wrote: > > Hi, > > I think it is time to integrate JEP 280 ("Indify String Concat") into JDK 9: > http://openjdk.java.net/jeps/280 > > The JEP is Targeted, the CCC is approved, the code reviews and > pre-integration checks come clean. In theory, we would like to integrate > with some "optimal" concat strategy switched by default, but we have to > wait for some Hotspot compiler changes to settle before we can make that > call. (Notably, https://bugs.openjdk.java.net/browse/JDK-8136469). > > So, I would like to proceed in two parts: > > Part 1: Integrate all code changes, but keep the concatenation strategy > to be the same "naive" StringBuilder.append chains. In other words, we > are emitting string concat indy, but always link in the naive > StringBuilder.append code. > > Part 2: (After HS changes settle in) Re-run performance tests, and > switch to a more optimal concatenation strategy. This amounts to > one-liner change in a default value for Java property. > > Thoughts? I would like relevant Project Leads / Reviewers (langtools and > jdk) to OKay the integration. > > FTR, the webrevs: > http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ > http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ > http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ > > Thanks, > -Aleksey > From paul.sandoz at oracle.com Fri Jan 22 09:48:13 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jan 2016 10:48:13 +0100 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <56A1F4D6.80303@oracle.com> References: <569F80B6.7080702@oracle.com> <56A1F4D6.80303@oracle.com> Message-ID: <5E8C1040-A0CD-4578-B147-DCC62B487F4A@oracle.com> Hi Shilpi, Similar to the previous patch, can you double check Lookup.unreflect. Not sure how far you wanna go in this case, but checking i think would be useful. 860 * or if the method is {@code private} method of interface s/method of interface/method of an interface/ Paul. > On 22 Jan 2016, at 10:22, shilpi rastogi wrote: > > > Gentle Reminder! > > -------- Original Message -------- > Subject: [9u-dev] 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods > Date: Wed, 20 Jan 2016 18:12:30 +0530 > From: shilpi rastogi > Organization: Oracle Corporation > To: core-libs-dev at openjdk.java.net > > > > Hi All, > > Please review my fix related to java doc. > > MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods- > > MethodHandles.Lookup.findSpecial() throws IllegalAccessException if the target method is static. > > Bug link- https://bugs.openjdk.java.net/browse/JDK-8138578 > Webrev link- http://cr.openjdk.java.net/~srastogi/8138578/webrev.00/ > > Thnaks, > Shilpi > > From aph at redhat.com Fri Jan 22 09:51:29 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 22 Jan 2016 09:51:29 +0000 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <56A1FBA1.9010108@redhat.com> On 22/01/16 00:01, Vitaly Davidovich wrote: > I think the get/setOpaque methods need a bit more explanation ("opaque" is > an odd naming choice, IMO). Specifically, it says the operations are done > in program order but have no effect on inter-thread ordering/visibility. > Is this spec verbiage for a full compiler-only fence? It's like C++ memory_order_relaxed, I guessed. I understand that but not "opaque". Andrew. From sean.coffey at oracle.com Fri Jan 22 11:14:17 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 22 Jan 2016 11:14:17 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <56A11938.8040804@oracle.com> References: <568FF6A5.4080700@oracle.com> <569FB2D3.6080607@oracle.com> <569FCF34.8020401@oracle.com> <56A11938.8040804@oracle.com> Message-ID: <56A20F09.2090904@oracle.com> On 21/01/16 17:45, Mark Sheppard wrote: > Thanks Sean > > i can revert the isClosed(), I put them in to make the access consistent I'd question if the synchronized isClosed() and setClosed(boolean) helper methods are needed at all. The volatile closed boolean variable should be enough to manage this state flag. Either way, I don't think the synchronization around the closed variable is root cause of this issue and performance issues could be addressed in another change if necessary (playing safe) regards, Sean. > > regards > Mark > > On 20/01/2016 18:17, Se?n Coffey wrote: >> Hi Mark, >> >> SelectorImpl.java: >> >> line 125, could you use a 2 arg method call to dprint. It'll print >> the stacktrace instead : >> dprint(".close: selector.close: " + t); --> dprint(".close: >> selector.close", t); >> >> The "while (!isClosed()) " change introduces a new hot lock on closed >> variable. Hopefully, it won't impact performance too much. >> >> Looks good to me otherwise. >> >> Regards, >> Sean. >> >> On 20/01/16 16:16, Mark Sheppard wrote: >>> Hi, >>> an update has been made to the webrev >>> >>> http://cr.openjdk.java.net/~msheppar/8144144/webrev.03/ >>> >>> an anomaly was found in the select loop of the SelectorImpl.run() >>> method >>> some defensive programming, for selector null references, have been >>> added, also. >>> >>> regards >>> Mark >>> >>> On 08/01/2016 17:49, Mark Sheppard wrote: >>>> Hi >>>> please oblige and review the following changes >>>> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ >>>> >>>> which addresses the issue >>>> https://bugs.openjdk.java.net/browse/JDK-8144144 >>>> >>>> the changes ensure that an Acceptor and its associated >>>> ServerSocket/ServerSocketChannel >>>> are closed. >>>> >>>> regards >>>> Mark >>> >> > From vitalyd at gmail.com Fri Jan 22 11:15:28 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 06:15:28 -0500 Subject: API review of VarHandles In-Reply-To: <56A1FBA1.9010108@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> Message-ID: On Friday, January 22, 2016, Andrew Haley wrote: > On 22/01/16 00:01, Vitaly Davidovich wrote: > > I think the get/setOpaque methods need a bit more explanation ("opaque" > is > > an odd naming choice, IMO). Specifically, it says the operations are > done > > in program order but have no effect on inter-thread ordering/visibility. > > Is this spec verbiage for a full compiler-only fence? > > It's like C++ memory_order_relaxed, I guessed. I understand that but > not "opaque". I thought so too before reading opaque javadoc (no pun intended). C++ relaxed says nothing about order though, only atomicity. Opaque doesn't mention atomicity at all. I'm guessing opaque is compiler only fence because a method that a compiler doesn't inline (as an example) is considered "opaque" to the compiler and serves like a fence. Perhaps that's how opaque ended up here. At any rate, I think it's a poor name. > > Andrew. > -- Sent from my phone From claes.redestad at oracle.com Fri Jan 22 11:34:05 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 12:34:05 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap Message-ID: <56A213AD.9090605@oracle.com> Hi, this patch removes some arguably unnecessary Enum[0] constants from EnumMap, EnumSet, and an associated test. The one in EnumMap was unused, while the one in EnumSet was only used for serialization purposes. Doing this for performance seems questionable[1]. During testing I discovered that EnumSet was missing a serialVersionUID, which needs to be added. Bug: https://bugs.openjdk.java.net/browse/JDK-8148044 Webrev: http://cr.openjdk.java.net/~redestad/8148044/webrev.01/ Thanks! Claes [1] http://shipilev.net/blog/2016/arrays-wisdom-ancients/#_caching_the_array From jan.lahoda at oracle.com Fri Jan 22 11:41:44 2016 From: jan.lahoda at oracle.com (Jan Lahoda) Date: Fri, 22 Jan 2016 12:41:44 +0100 Subject: RFR 8147984: WindowsTerminal should support function keys Message-ID: <56A21578.5080105@oracle.com> Hello, I'd like to enhance the WindowsTerminal in jdk.internal.le with function keys handling. The intent is so that jshell can bind actions for shortcuts including function keys. The patch for adding the function keys support is here: http://cr.openjdk.java.net/~jlahoda/8147984/webrev.00/ An example of a feature that uses/may use this support is here: http://mail.openjdk.java.net/pipermail/kulla-dev/2016-January/001226.html Any comments are welcome! Thanks, Jan From Alan.Bateman at oracle.com Fri Jan 22 11:53:06 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Jan 2016 11:53:06 +0000 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A213AD.9090605@oracle.com> References: <56A213AD.9090605@oracle.com> Message-ID: <56A21822.2000203@oracle.com> On 22/01/2016 11:34, Claes Redestad wrote: > : > > During testing I discovered that EnumSet was missing a > serialVersionUID, which needs to be added. I assume this is not needed, it already has: @SuppressWarnings("serial") // No serialVersionUID due to usage of serial proxy pattern -Alan From paul.sandoz at oracle.com Fri Jan 22 11:56:25 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jan 2016 12:56:25 +0100 Subject: Stream.findFirst for unordered source optimization In-Reply-To: <1152739344.20160122121906@gmail.com> References: <1152739344.20160122121906@gmail.com> Message-ID: <30A68664-D525-41BF-9688-FAB7E1F6944A@oracle.com> Hi Tagir, Thanks for looking at this. I would prefer if a boolean was passed into the task constructor e.g.: @Override public O evaluateParallel(PipelineHelper helper, Spliterator spliterator) { // This takes into account the upstream ops flags and the terminal // op flags and therefore takes into account findFirst or findAny boolean mustFindFirst = StreamOpFlag.ORDERED.isKnown(helper.getStreamAndOpFlags()); return new FindTask<>(this, mustFindFirst, helper, spliterator).invoke(); } so we keep the flag evaluation logic within the evaluate method. Then change FindOp.mustFindFirst to be an int of the opFlags calculated in the constructor: this.opFlags = StreamOpFlag.IS_SHORT_CIRCUIT | (mustFindFirst ? 0 : StreamOpFlag.NOT_ORDERED); Paul. > On 22 Jan 2016, at 07:19, Tagir F. Valeev wrote: > > Hello! > > Seems that currently Stream.findFirst is not optimized for unordered > source. I think it should work as findAny in this case. Here's a small > patch which fixes this: > > http://cr.openjdk.java.net/~tvaleev/patches/findFirst/find_patch.txt > > Simple JMH test: > http://cr.openjdk.java.net/~tvaleev/patches/findFirst/FindTest.java > > Original: > http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_orig.txt > # JMH 1.11.2 (released 85 days ago) > # VM version: JDK 9-ea, VM 9-ea+99-2015-12-23-183325.javare.4146.nc > Benchmark Mode Cnt Score Error Units > FindTest.findAny avgt 30 12439,631 ? 1787,866 us/op > FindTest.findAnyUnordered avgt 30 12923,080 ? 1072,537 us/op > FindTest.findFirst avgt 30 48047,467 ? 2713,489 us/op > FindTest.findFirstUnordered avgt 30 52648,893 ? 3934,682 us/op > > Patched: > http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_patched.txt > Benchmark Mode Cnt Score Error Units > FindTest.findAny avgt 30 11312,238 ? 386,627 us/op > FindTest.findAnyUnordered avgt 30 12136,953 ? 1536,817 us/op > FindTest.findFirst avgt 30 47517,776 ? 2844,607 us/op > FindTest.findFirstUnordered avgt 30 13147,492 ? 1140,592 us/op > > If you think it's a reasonable thing to patch, I can log an issue, > generate webrev and check whether jtreg tests pass. > > With best regards, > Tagir Valeev. > From claes.redestad at oracle.com Fri Jan 22 11:56:47 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 12:56:47 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A21822.2000203@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> Message-ID: <56A218FF.6070002@oracle.com> On 2016-01-22 12:53, Alan Bateman wrote: > > > On 22/01/2016 11:34, Claes Redestad wrote: >> : >> >> During testing I discovered that EnumSet was missing a >> serialVersionUID, which needs to be added. > I assume this is not needed, it already has: > > @SuppressWarnings("serial") // No serialVersionUID due to usage of > serial proxy pattern That's odd.. there's a jtreg test that fails if I don't explicitly add the serialVersionUID (that's where I got the actual UID from). I'll investigate. Thanks! -Claes From aleksey.shipilev at oracle.com Fri Jan 22 11:57:28 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 14:57:28 +0300 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A213AD.9090605@oracle.com> References: <56A213AD.9090605@oracle.com> Message-ID: <56A21928.3090904@oracle.com> On 01/22/2016 02:34 PM, Claes Redestad wrote: > Bug: https://bugs.openjdk.java.net/browse/JDK-8148044 > Webrev: http://cr.openjdk.java.net/~redestad/8148044/webrev.01/ -1, sorry. I would actually suggest keeping the ZERO_ARRAY in Enum*, because enums often have small cardinality, and there dodging an allocation matters. I'd also make sure we have all ZERO_ARRAY declarations like that as "static final". -Aleksey From claes.redestad at oracle.com Fri Jan 22 12:07:29 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 13:07:29 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A218FF.6070002@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> Message-ID: <56A21B81.7090800@oracle.com> On 2016-01-22 12:56, Claes Redestad wrote: > > > On 2016-01-22 12:53, Alan Bateman wrote: >> >> >> On 22/01/2016 11:34, Claes Redestad wrote: >>> : >>> >>> During testing I discovered that EnumSet was missing a >>> serialVersionUID, which needs to be added. >> I assume this is not needed, it already has: >> >> @SuppressWarnings("serial") // No serialVersionUID due to usage of >> serial proxy pattern > > That's odd.. there's a jtreg test that fails if I don't explicitly add > the serialVersionUID (that's where I got the actual UID from). I'll > investigate. Right... java/util/EnumSet/BogusEnumSet.java fails, which is an explicit test to verify a corrupt EnumSet doesn't deserialize properly: java.io.InvalidClassException: java.util.EnumSet; local class incompatible: stream classdesc serialVersionUID = 1009687484059888093, local class serialVersionUID = 4168005130090799668 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:618) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1626) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1626) at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777) at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1354) at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368) at BogusEnumSet.deserialize(BogusEnumSet.java:91) at BogusEnumSet.main(BogusEnumSet.java:78) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:520) at com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:218) at java.lang.Thread.run(Thread.java:804) It would seem the assessment that serialVersionUID is not needed when using the serial proxy pattern might be misguided (or is this a bug in ObjectInputStream)? Thanks! /Claes From vitalyd at gmail.com Fri Jan 22 12:08:40 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 07:08:40 -0500 Subject: API review of VarHandles In-Reply-To: <56A186AD.8090100@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: On Thursday, January 21, 2016, Brian Goetz wrote: > > I am baffled as to how the original language syntax proposal of using some >> trick like "xx.volatile.imaginaryMethod()" plus maybe one or two new >> bytecodes was considered unacceptable; looking at this API, I know that >> none of the aforementioned metrics were considered as acceptance criteria. >> How did we get to this strange place? >> > > The path was actually quite obvious. > > The first question was, "OK, if the language had this magic syntax, what > bytecode would get generated? And clearly it couldn't be Unsafe. So > either a pile of new bytecodes, or an API, was needed that the front-end > compiler could generate against. > > Building APIs is generally preferable to new bytecodes, if an API can do > the job, and it turned out that it could do it just as well. Once we had an > API that met the requirements, it was pretty clear that language syntax was > not only unnecessary, but likely undesirable -- these exotic access modes > are power tools for power users (do you really want typical users to reason > about mixed volatile and relaxed access to variables?), and they meet the > needs of such users just fine (those users have been using Unsafe all > along, and this is clearly better and safer.) The VarHandle API isn't ergonomic like Unsafe; this being for power users is irrelevant to the ergonomics of the API. Instead, it's fairly verbose with setup ceremony. Now, I personally don't mind that too much and more interested in the features it provides but I'm not surprised by David's reaction. People already provide nicer APIs on top of Unsafe in their own projects, and I suspect this will be even more so with VH. > > So it was not the either-or of "either an API *or* magic language syntax", > it was "either an API *or* syntax plus an API". At which point it was > clear that the syntax didn't carry its own weight. > > I think this is a fine example of how the obvious idea was wrong, and that > most of the value of the initial syntax idea was framing the problem > space. I'm glad we saw fit to jettison that when it ceased to provide > value. > > > -- Sent from my phone From peter.levart at gmail.com Fri Jan 22 12:10:16 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jan 2016 13:10:16 +0100 Subject: Ephemerons In-Reply-To: <63A40C76-1D63-47F4-B65A-1BDBE7EE000E@oracle.com> References: <569BDAF2.2080404@gmail.com> <63A40C76-1D63-47F4-B65A-1BDBE7EE000E@oracle.com> Message-ID: <56A21C28.40201@gmail.com> Hi Mandy, Thanks for noticing this post. I think Ephemerons would be a nice tool that would solve some "memory leak" problems people have when using data structures with weak references. On 01/20/2016 05:20 PM, Mandy Chung wrote: > Hi Peter, > > This is interesting and useful for cases when key/value in a reference chain cycle. This seems worth exploring for a future release. I don?t have any cycle to study this in depth and begin any discussion unfortunately at this time. > > Have you tried prototype Ephemerons with WeakReference for the key and PhantomReference for the value? Would that satisfy your cases - it would not work if SoftReference is part of the chain - just wondering if you have explored that? I don't know exactly what you mean by that. I think there's no way to "simulate" the semantics of Ephemeron with any combination of existing Reference types, because the reachabilities of their referents are defined to be unconditional, if that was what you were asking. If you are just referring to the desired "reachability" of the Ephemeron's key and value, then let me explain a bit about it... 1st, the phantom-reachability of the Ephemeron's value wouldn't have much sense, because then the Ephemeron's value would have to be unretrievable (like PhantomReference's referent) in order to preserve the semantics of existing PhantomReferences which guarantee that once a referent becomes phantom-reachable, it can never be made more strongly reachable again. The whole point of Ephemeron is it's ability to retrieve the value until it has been cleared. Ephemeron's key can be compared with any existing Java Reference's referent. When it becomes xxx-reachable, GC will clear the XxxReference atomically with all other XxxReferences of the same type that refer to the same referent. The strength of xxx-reachability determines when that happens relative to YyyReference(s) of other types when they refer to the same referent. For example: Object referent = new Object() { protected void finalize() { System.out.println("finalized"); } }; SoftReference softRef = new SoftReference<>(referent); WeakReference weakRef = new WeakReference<>(referent); PhantomReference phantomRef = new PhantomReference<>(referent); referent = null; ... In above example, phantomRef will never be cleared before finalize() is called and filalize() will never be called before weakRef is cleared and weakRef will never be cleared before softRef. In my prototyle, Ephemeron's key behaves as though there was a 5th strength positioned between WeakReference and finalization (FinalReference). Ephemeron could be specified to have exactly the same strength for it's key as WeakReference has for it's referent, for example, but I choose to introduce another strength so that Ephemeron can not influence classical WeakReference behaviour - and it's also easier to explain the behavior that way and reason about it. So for the purpose of this prototype, an Object is defined to be ephemeraly-reachable if it is neither weakly, softly nor strongly reachable but can be reached through the Ephemeron by following it's key (or sometimes also the value as we'll see later). The "weak" reachabilities table, from the strongest to the weakest: softly-reachable weakly-reachable ephemeraly-reachable final-reachable (finalizable) phantom-reachable Unlike the Ephemeron's key, it's value is something that can't be compared with the referents of existing Reference types. It's reachability is "conditional". As with all definitions that contain recursion, one has to be careful to not construct a paradox. The prototype defines the reachability of Ephemeron's value as: - an object is ephemerally-reachable if it is neither weakly, softly nor strongly reachable, but can be reached through the Ephemeron by following it's value while the Ephemeron's key is at most ephemeraly-reachable - an object is weakly-reachable if it is neither softly nor strongly reachable, but can be reached through the Ephemeron by following it's value while the Ephemeron's key is at least weakly-reachable The value's reachability is therefore dependent on the key's reachability. The above definition is sound because there's no paradox when other definitions of reachabilities are taken into account. Here's the complete definition of all reachabilities for the prototype: *
  • An object is strongly reachable if it can be reached * by some thread without traversing any reference objects. A * newly-created object is strongly reachable by the thread that * created it. *
  • An object is softly reachable if it is not strongly * reachable but can be reached by traversing a soft reference. *
  • An object is weakly reachable if it is neither * strongly nor softly reachable but can be reached by traversing a * weak reference or by traversing an ephemeron through it's value while * the ephemeron's key is at least weakly reachable. *
  • An object is ephemerally reachable if it is neither * strongly, softly nor weakly reachable but can be reached by traversing an * ephemeron through it's key or by traversing an ephemeron through it's value * while it's key is at most ephemerally reachable. When the ephemerons that * refer to ephemerally reachable key object are cleared, the key object becomes * eligible for finalization. *
  • An object is phantom reachable if it is neither * strongly, softly, weakly nor ephemerally reachable, it has been finalized, * and some phantom reference refers to it. *
  • Finally, an object is unreachable, and therefore * eligible for reclamation, when it is not reachable in any of the * above ways. In General, if an object can only be reached through the Epehemron by traversing the value, it can never be defined to be more strongly reachable than the Ephemeron's key at the same time, because a value can refer to the key. So a possible alternative to the prototype would be for Ephemeron's value to follow exactly the reachability of Ephemeron's key until the Ephemeron gets cleared because it's key reaches certain level of reachability. As one might have noticed, the algorithm to process Ephemerons is iterative, with complexity O(n*(d+1)) where "n" is the number of discovered Ephemerons and "d" is the maximum number of "hops" in object graph where a "hop" is defined as a point in chain where the value of some ephemeron refers directly or indirectly to the key of some other ephemeron and the value if that other ephemeron continues the chain. In practical data structures, d is rarely > 1 and mostly very low. The algorithm in the prototype also reduces the number of outer iterations to less than 'd' in many situations. In the prototype where the reachability of the value has an upper bound (weakly-reachable), "n" can be reduced to the number of discovered Ephemerons. If the reachability of value followed exactly the reachability of key, then "n" would have to count the number of discovered Ephemerons + WeakReferences + SoftReferences together. Such implementation is therefore possible, but has a cost. Regards, Peter > Mandy > >> On Jan 17, 2016, at 10:18 AM, Peter Levart wrote: >> >> Hi, >> >> Ephemerons are special kind of references described by Barry Hayes [1]. In the simple variant, they contain two "referents". One is called the key, which can be viewed as a "weak" referent in the style of Java reference types (SoftReference, WeakReference, PhantomReference), the other is called the value, which is also treated specially by GC. It's reachability is dependent on the reachability of the key. >> >> Ephemerons solve the problem seen for example in the java.util.WeakHashMap when a value in this map directly or indirectly refers to it's key. Such entry will never be purged as the value is always reachable from the WeakHashMap instance and through the value, it's key is reachable too. There are other places where Ephemerons could be used - for example in ClassValue API. Try googling for "java ephemeron" (without quotes) to find out other situations where Ephemerons would be beneficial. >> >> If one was to implement Ephemerons in Java, the main question he would be asking is how Ephemeron(s) were supposed to interact with existing Java reference types. Hayes only defines their behavior in isolation, but Java already has 4 "weak" reference types with different "strengths": SoftReference, WeakReference, FinalReference (for implementing finalization) and PhantomReference. In the prototype I choose to define Ephemerons as a new reference type with it's own "strength" for the key referent positioned between WeakReference and FinalReference. It would be possible to merge it with an existing reference type like WeakReference or position it between SoftReference and WeakReference or even entirely "on the top" of the strength list, but I think it would not be wise to position it below FinalReference or even PhantomReference. What's best is an open question. What is also not so obvious is how to define the reachability of the Ephemeron's value and it's interaction with existing refe >> rence types. I think I defined it soundly (see the reachability types in the javadoc of [4]). The reason for defining the reachability of the value to alternate between ephemeraly-reachable and weakly-reachable and not between ephemeraly-reachable and strongly-reachable, while theoretically possible, is the desire to have a separate phase of processing for each reachability strength, like it is done currently, and which doesn't affect the performance of processing of existing reference types. >> >> Having skimmed through the reference processing code in the VM for a couple of times, I thought it should not be too complicated to implement another type of "weak" reference. Encouraged by how little changes were needed to remove the sun.misc.Cleaner special type of reference [2], I began experimenting and came up with a prototype that seems to work [3]. Luckily most of the logic to process Reference objects in VM is encapsulated in the ReferenceProcessor class and this logic is invoked from various GC implementations of the hotspot. Changes needed are therefore mostly localized to this class. The webrev also contains recent changes from JDK-8143847 [2] that have not yet propagated to the jdk9-dev repo. When they do, I will prepare a rebased webrev without those changes. >> >> I'm publishing this prototype here to get the answer to the main question: Is there an interest to have Ephemeron reference type in Java? >> >> Given that there was interest, I would also like to initiate a discussion about: >> - What would be the most appropriate "strength" of reachabillity for Epehmeron referents (key and value) and the desired interactions with existing reachabilities. >> - The prototype itself. Since I'm new to this part of code (that's my 1st not so shallow dive into the VM), I surely must have missed places that should be changed. Although the prototype seems to work, I have not created extensive tests for the functionality and have not exposed it to all the various GC algorithms and situations yet. I could use some advise on how to exercise all GC algorithm combinations possible in the VM (what flags to use, for example). >> >> Regards, Peter >> >> [1]https://static.aminer.org/pdf/PDF/000/522/273/ephemerons_a_new_finalization_mechanism.pdf >> [2]https://bugs.openjdk.java.net/browse/JDK-8143847 >> [3]http://cr.openjdk.java.net/~plevart/misc/Ephemeron/ >> [4]http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.01/src/java.base/share/classes/java/lang/ref/Ephemeron.java.html >> >> P.S. >> >> To wet some appetite, with the above prototype (applied to current jdk9-dev), it is possible to run the following example: >> >> import java.lang.ref.Ephemeron; >> import java.util.ArrayList; >> import java.util.List; >> >> public class EphemeronTest { >> >> static class Key { >> final int i; >> >> Key(int i) { >> this.i = i; >> } >> >> @Override >> public String toString() { >> return "k" + i; >> } >> } >> >> static class Value { >> final Key key; >> >> Value(Key key) { >> this.key = key; >> } >> >> @Override >> public String toString() { >> return "v(" + key + ")"; >> } >> } >> >> static class Eph extends Ephemeron { >> public Eph(Key key, Value value) { >> super(key, value); >> } >> >> @Override >> public String toString() { >> return getKey() + "=" + getValue(); >> } >> } >> >> static void test(int size, boolean forwardChaining) throws Exception { >> System.out.println(); >> System.out.println((forwardChaining ? "Forward" : "Backward") + >> " chaining of value->key ..."); >> System.out.println(); >> List ephs = new ArrayList<>(size); >> >> Key k1 = new Key(1); >> { >> Key kp = k1; >> for (int i = 2; i <= size; i++) { >> Key ki = new Key(i); >> ephs.add( >> forwardChaining >> ? new Eph(kp, new Value(ki)) >> : new Eph(ki, new Value(kp)) >> ); >> kp = ki; >> } >> ephs.add( >> forwardChaining >> ? new Eph(kp, new Value(k1)) >> : new Eph(k1, new Value(kp)) >> ); >> kp = null; >> } >> >> System.gc(); >> System.out.println("1: " + ephs); >> k1.getClass(); // reachabilityFence >> >> k1 = null; >> System.gc(); >> System.out.println("2: " + ephs); >> } >> >> public static void main(String[] args) throws Exception { >> >> int size = (args.length < 1) >> ? 5 >> : Math.max(2, Integer.parseInt(args[0])); >> >> test(size, true); >> test(size, false); >> } >> } >> >> >> Which prints: >> >> Forward chaining of value->key ... >> >> 1: [k1=v(k2), k2=v(k3), k3=v(k4), k4=v(k5), k5=v(k1)] >> 2: [null=null, null=null, null=null, null=null, null=null] >> >> Backward chaining of value->key ... >> >> 1: [k2=v(k1), k3=v(k2), k4=v(k3), k5=v(k4), k1=v(k5)] >> 2: [null=null, null=null, null=null, null=null, null=null] >> >> >> If you compile the JDK with --enable-debug and use the following VM switches when running: -XX:+PrintGCDetails -XX:+TraceReferenceGC, you can also observe the inner workings of the reference processing, including Ephemerons. >> From claes.redestad at oracle.com Fri Jan 22 12:19:18 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 13:19:18 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A21928.3090904@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21928.3090904@oracle.com> Message-ID: <56A21E46.5070107@oracle.com> On 2016-01-22 12:57, Aleksey Shipilev wrote: > On 01/22/2016 02:34 PM, Claes Redestad wrote: >> Bug: https://bugs.openjdk.java.net/browse/JDK-8148044 >> Webrev: http://cr.openjdk.java.net/~redestad/8148044/webrev.01/ > -1, sorry. > > I would actually suggest keeping the ZERO_ARRAY in Enum*, because enums > often have small cardinality, and there dodging an allocation matters. > I'd also make sure we have all ZERO_ARRAY declarations like that as > "static final". Sure, I realized we can have the cake and eat it by moving the ZERO_ARRAY* into the SerializationProxy class: this way we get the miniscule benefit of having a cached empty array while deferring the miniscule cost of the Enum[].class load until we actually serialize an EnumSet (if ever). http://cr.openjdk.java.net/~redestad/8148044/webrev.02/ Thanks! /Claes From aleksey.shipilev at oracle.com Fri Jan 22 12:25:27 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 15:25:27 +0300 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A21E46.5070107@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21928.3090904@oracle.com> <56A21E46.5070107@oracle.com> Message-ID: <56A21FB7.2030701@oracle.com> On 01/22/2016 03:19 PM, Claes Redestad wrote: > http://cr.openjdk.java.net/~redestad/8148044/webrev.02/ This one's good. -Aleksey From claes.redestad at oracle.com Fri Jan 22 12:29:56 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 13:29:56 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A21FB7.2030701@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21928.3090904@oracle.com> <56A21E46.5070107@oracle.com> <56A21FB7.2030701@oracle.com> Message-ID: <56A220C4.6060808@oracle.com> On 2016-01-22 13:25, Aleksey Shipilev wrote: > On 01/22/2016 03:19 PM, Claes Redestad wrote: >> http://cr.openjdk.java.net/~redestad/8148044/webrev.02/ > This one's good. > > -Aleksey Thanks! /Claes From aph at redhat.com Fri Jan 22 12:54:05 2016 From: aph at redhat.com (Andrew Haley) Date: Fri, 22 Jan 2016 12:54:05 +0000 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> Message-ID: <56A2266D.5060202@redhat.com> On 01/22/2016 11:15 AM, Vitaly Davidovich wrote: > I'm guessing opaque is compiler only fence because a method that a > compiler doesn't inline (as an example) is considered "opaque" to > the compiler and serves like a fence. Perhaps that's how opaque > ended up here. At any rate, I think it's a poor name. Humm. I guess so, but I'm not sure why you'd want a compiler fence in Java. It's not like C where you can access memory-mapped I/O. I suppose it disallows roach-motel code motion. Andrew. From aleksey.shipilev at oracle.com Fri Jan 22 13:06:09 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 16:06:09 +0300 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: <56A22941.1050702@oracle.com> On 01/22/2016 03:08 PM, Vitaly Davidovich wrote: > The VarHandle API isn't ergonomic like Unsafe; this being for power users > is irrelevant to the ergonomics of the API. Instead, it's fairly verbose > with setup ceremony. Now, I personally don't mind that too much and more > interested in the features it provides but I'm not surprised by David's > reaction. Um, I don't get the "fairly verbose" and "setup ceremony" parts, especially in comparison with Unsafe. Unsafe also comes with a setup ceremony, don't you think? See: public class C { static class CU { static final Unsafe U; static final long OFFSET; static { try { Field f = Unsafe.class.getDeclaredField("theUnsafe"); f.setAccessible(true); U = (Unsafe) f.get(null); OFFSET = U.objectFieldOffset(CU.class.getDeclaredField("x")); } catch (ReflectiveOperationException eh) { throw new IllegalStateException(eh); } } private int x; public boolean doCas(int expected, int value) { return U.compareAndSwapInt(this, OFFSET, expected, value); } } static class VU { static final VarHandle VH; static { try { VH = MethodHandles.lookup().findVarHandle( VU.class, "x", int.class); } catch (ReflectiveOperationException eh) { throw new IllegalStateException(eh); } } private int x; public boolean doCas(int expected, int value) { return VH.compareAndSet(this, expected, value); } } } Even if you pull off Unsafe lookup into separate "holder" class, you'd still need to lookup OFFSET. Granted, VH lookup requires learning new stuff, but so does Unsafe, A*FU, Reflection, or any other special API/syntax. The mind trick is that you already *know* all those APIs, and so they appear conceptually simple to you. Once you internalize that VarHandle is a "handle", which you have to acquire before use, it clicks. > People already provide nicer APIs on top of Unsafe in their own projects, > and I suspect this will be even more so with VH. Yes, but lacking the syntax support, that's the best you can do: static class MyAwesomeWrapper { public static VarHandle iAmPrettySureThatFieldExists( MethodHandles.Lookup lookup, Class recv, String name, Class type) { try { return lookup.findVarHandle(recv, name, type); } catch (ReflectiveOperationException eh) { throw new IllegalStateException(eh); } } } static class VUH { static final VarHandle VH = MyAwesomeWrapper.iAmPrettySureThatFieldExists( MethodHandles.lookup(), VUH.class, "x", int.class); private int x; public boolean doCas(int expected, int value) { return VH.compareAndSet(this, expected, value); } } ...which is not that bad? I am not discounting David's comments about the conceptual complexity of the documentation. It looks like a normative text that 99% users would not read, but stroll straight into code examples. -Aleksey From vitalyd at gmail.com Fri Jan 22 13:09:29 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:09:29 -0500 Subject: API review of VarHandles In-Reply-To: <56A2266D.5060202@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> <56A2266D.5060202@redhat.com> Message-ID: On Friday, January 22, 2016, Andrew Haley wrote: > On 01/22/2016 11:15 AM, Vitaly Davidovich wrote: > > > I'm guessing opaque is compiler only fence because a method that a > > compiler doesn't inline (as an example) is considered "opaque" to > > the compiler and serves like a fence. Perhaps that's how opaque > > ended up here. At any rate, I think it's a poor name. > > Humm. I guess so, but I'm not sure why you'd want a compiler fence in > Java. It's not like C where you can access memory-mapped I/O. I > suppose it disallows roach-motel code motion. Well that's part of my confusion on what this is :). C++ relaxed makes sense to me - don't care about anything other than atomicity, which could be useful for Java too so that one could reliably/portably do atomic stores to, e.g, longs and have that compile to plain mov on 64 bit machines and something possibly more expensive on 32 bit. Maybe that's what this actually is but "program order" is too vague here. > > Andrew. > -- Sent from my phone From david.lloyd at redhat.com Fri Jan 22 13:11:01 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 07:11:01 -0600 Subject: API review of VarHandles In-Reply-To: <56A186AD.8090100@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: <56A22A65.4040200@redhat.com> On 01/21/2016 07:32 PM, Brian Goetz wrote: > >> I am baffled as to how the original language syntax proposal of using >> some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two >> new bytecodes was considered unacceptable; looking at this API, I know >> that none of the aforementioned metrics were considered as acceptance >> criteria. How did we get to this strange place? > > The path was actually quite obvious. > > The first question was, "OK, if the language had this magic syntax, what > bytecode would get generated? And clearly it couldn't be Unsafe. So > either a pile of new bytecodes, or an API, was needed that the front-end > compiler could generate against. > > Building APIs is generally preferable to new bytecodes, if an API can do > the job, and it turned out that it could do it just as well. Once we had > an API that met the requirements, it was pretty clear that language > syntax was not only unnecessary, but likely undesirable -- these exotic > access modes are power tools for power users (do you really want typical > users to reason about mixed volatile and relaxed access to variables?), > and they meet the needs of such users just fine (those users have been > using Unsafe all along, and this is clearly better and safer.) > > So it was not the either-or of "either an API *or* magic language > syntax", it was "either an API *or* syntax plus an API". At which point > it was clear that the syntax didn't carry its own weight. > > I think this is a fine example of how the obvious idea was wrong, and > that most of the value of the initial syntax idea was framing the > problem space. I'm glad we saw fit to jettison that when it ceased to > provide value. I will contest the "APIs is generally preferable to new bytecodes, if an API can do the job" tack because it clearly doesn't track logically: most of the bytecodes in the JVM today could have been intrinsic API methods instead, e.g. Integer.add(int, int) and that sort of thing, but they clearly benefit from being bytecodes just from a simplicity standpoint. Having new bytecodes would certainly be simpler for javac; it would be simpler for the user also, especially the user of class generators and transformers (who by this point will now have to do some pretty bananas things to transform field references); I can only speculate that it would also be simpler for the interpreter/JIT itself as well because really you're just generating variations on getfield, *aload, etc. But that is all besides the point. Even the API itself is really heavy and weird. Experts need simple APIs more than anyone - especially those writing complex concurrent algorithms. They're the ones who aren't content to copy and paste blobs off of StackOverflow; rather they need to be able to prove that their program is correct, and the more complexity there is, the more difficult a task that becomes. But even the premise that this API is experts-only is self-fulfilling: atomics and fences are quite useful to less advanced users too, especially simple get-and-set and CAS operations. But there is little chance that this API is going to be used (or used correctly) by such users. I have no problem with the methods present on the final product, in terms of the types of fences and operations. But the setup procedures and concepts are far more complex than necessary, and I don't see this being useful to most users in its current shape as most people will flee from it. -- - DML From vitalyd at gmail.com Fri Jan 22 13:22:04 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:22:04 -0500 Subject: API review of VarHandles In-Reply-To: <56A22941.1050702@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22941.1050702@oracle.com> Message-ID: On Friday, January 22, 2016, Aleksey Shipilev wrote: > On 01/22/2016 03:08 PM, Vitaly Davidovich wrote: > > The VarHandle API isn't ergonomic like Unsafe; this being for power users > > is irrelevant to the ergonomics of the API. Instead, it's fairly verbose > > with setup ceremony. Now, I personally don't mind that too much and more > > interested in the features it provides but I'm not surprised by David's > > reaction. > > Um, I don't get the "fairly verbose" and "setup ceremony" parts, > especially in comparison with Unsafe. Unsafe also comes with a setup > ceremony, don't you think? Yes unsafe also has some ceremony, but it seems like VH is a bit more so. As I mentioned, I'm not too bothered by that aspect given wrappers will be built anyway (just like Unsafe). But, if this were in the language it would likely be more ergonomic. Not saying ergonomics trumps everything else, but it's something David alluded to (lang vs API). > > See: > > public class C { > static class CU { > static final Unsafe U; > static final long OFFSET; > > static { > try { > Field f = Unsafe.class.getDeclaredField("theUnsafe"); > f.setAccessible(true); > U = (Unsafe) f.get(null); > OFFSET = U.objectFieldOffset(CU.class.getDeclaredField("x")); > } catch (ReflectiveOperationException eh) { > throw new IllegalStateException(eh); > } > } > > private int x; > > public boolean doCas(int expected, int value) { > return U.compareAndSwapInt(this, OFFSET, expected, value); > } > } > > static class VU { > static final VarHandle VH; > > static { > try { > VH = MethodHandles.lookup().findVarHandle( > VU.class, "x", int.class); > } catch (ReflectiveOperationException eh) { > throw new IllegalStateException(eh); > } > } > > private int x; > > public boolean doCas(int expected, int value) { > return VH.compareAndSet(this, expected, value); > } > } > } > > Even if you pull off Unsafe lookup into separate "holder" class, you'd > still need to lookup OFFSET. > > Granted, VH lookup requires learning new stuff, but so does Unsafe, > A*FU, Reflection, or any other special API/syntax. The mind trick is > that you already *know* all those APIs, and so they appear conceptually > simple to you. Once you internalize that VarHandle is a "handle", which > you have to acquire before use, it clicks. I get the handle part and don't have a problem with the concept. I do agree that one will get used to it. Again, for me, features matter more than syntax (for the most part). > > > > People already provide nicer APIs on top of Unsafe in their own projects, > > and I suspect this will be even more so with VH. > > Yes, but lacking the syntax support, that's the best you can do: > > static class MyAwesomeWrapper { > public static VarHandle iAmPrettySureThatFieldExists( > MethodHandles.Lookup lookup, Class recv, String name, > Class type) { > try { > return lookup.findVarHandle(recv, name, type); > } catch (ReflectiveOperationException eh) { > throw new IllegalStateException(eh); > } > } > } > > static class VUH { > static final VarHandle VH = > MyAwesomeWrapper.iAmPrettySureThatFieldExists( > MethodHandles.lookup(), > VUH.class, "x", int.class); > > private int x; > > public boolean doCas(int expected, int value) { > return VH.compareAndSet(this, expected, value); > } > } > > ...which is not that bad? > > I am not discounting David's comments about the conceptual complexity of > the documentation. It looks like a normative text that 99% users would > not read, but stroll straight into code examples. I'm actually not bothered by the spec-like docs; in fact I like the extra detail in the same place as the API. Perhaps it can be rearranged so the impl details are not the first thing a reader sees. This is, by definition, a low level API with misuse leading to very subtle bugs - over documenting is much better than under documenting. > > -Aleksey > > -- Sent from my phone From dl at cs.oswego.edu Fri Jan 22 13:23:53 2016 From: dl at cs.oswego.edu (Doug Lea) Date: Fri, 22 Jan 2016 08:23:53 -0500 Subject: API review of VarHandles In-Reply-To: <56A1FBA1.9010108@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> Message-ID: <56A22D69.7040404@cs.oswego.edu> On 01/22/2016 04:51 AM, Andrew Haley wrote: > On 22/01/16 00:01, Vitaly Davidovich wrote: >> I think the get/setOpaque methods need a bit more explanation ("opaque" is >> an odd naming choice, IMO). Specifically, it says the operations are done >> in program order but have no effect on inter-thread ordering/visibility. >> Is this spec verbiage for a full compiler-only fence? Basically yes. It would be the same as C "volatile" (not C++/C11 "atomic"), if C volatile had a memory model spec. But in any case we can't use "volatile" for this in Java because it already means something else. > > It's like C++ memory_order_relaxed, I guessed. I understand that but > not "opaque". > The new name is required because opaque is only "like" C++ memory_order_relaxed. See the jmm-dev list discussions last year for details, but annoyingly, Java normal variables and C++ memory_order_relaxed are not quite the same, and neither are the same as C-volatile/opaque. -Doug From adinn at redhat.com Fri Jan 22 13:27:54 2016 From: adinn at redhat.com (Andrew Dinn) Date: Fri, 22 Jan 2016 13:27:54 +0000 Subject: Ephemerons In-Reply-To: <56A21C28.40201@gmail.com> References: <569BDAF2.2080404@gmail.com> <63A40C76-1D63-47F4-B65A-1BDBE7EE000E@oracle.com> <56A21C28.40201@gmail.com> Message-ID: <56A22E5A.60202@redhat.com> Hi Peter, On 22/01/16 12:10, Peter Levart wrote: > Thanks for noticing this post. ... Hey, I noticed it too! I think it's a good proposal very clearly explained and well thought through (as so often when you post). I have not commented because I don't really have a good use case against which to measure it. I can definitely see why and how it might be useful. Apologies for not being able to add more. regards, Andrew Dinn ----------- From forax at univ-mlv.fr Fri Jan 22 13:27:54 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 22 Jan 2016 14:27:54 +0100 (CET) Subject: JEP 280 (Indify String Concat) integration In-Reply-To: <2A6DF001-3692-49C6-BEDA-4D80023B8B47@oracle.com> References: <56A1F6D8.9070808@oracle.com> <2A6DF001-3692-49C6-BEDA-4D80023B8B47@oracle.com> Message-ID: <2103539321.1927034.1453469274765.JavaMail.zimbra@u-pem.fr> Yes, it's time to integrate that code, thumb up ! BTW, once the second part will be done, varargs calls should be the next target, see https://bugs.openjdk.java.net/browse/JDK-8013269. R?mi ----- Mail original ----- > De: "Paul Sandoz" > Cc: compiler-dev at openjdk.java.net, "Java Core Libs" > Envoy?: Vendredi 22 Janvier 2016 10:46:14 > Objet: Re: JEP 280 (Indify String Concat) integration > > Looks a fine approach to me. > > Getting code in earlier and tested, even initially under the less optimal > strategy, is good in terms of exposure. > > Paul. > > > On 22 Jan 2016, at 10:31, Aleksey Shipilev > > wrote: > > > > Hi, > > > > I think it is time to integrate JEP 280 ("Indify String Concat") into JDK > > 9: > > http://openjdk.java.net/jeps/280 > > > > The JEP is Targeted, the CCC is approved, the code reviews and > > pre-integration checks come clean. In theory, we would like to integrate > > with some "optimal" concat strategy switched by default, but we have to > > wait for some Hotspot compiler changes to settle before we can make that > > call. (Notably, https://bugs.openjdk.java.net/browse/JDK-8136469). > > > > So, I would like to proceed in two parts: > > > > Part 1: Integrate all code changes, but keep the concatenation strategy > > to be the same "naive" StringBuilder.append chains. In other words, we > > are emitting string concat indy, but always link in the naive > > StringBuilder.append code. > > > > Part 2: (After HS changes settle in) Re-run performance tests, and > > switch to a more optimal concatenation strategy. This amounts to > > one-liner change in a default value for Java property. > > > > Thoughts? I would like relevant Project Leads / Reviewers (langtools and > > jdk) to OKay the integration. > > > > FTR, the webrevs: > > http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ > > http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ > > http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ > > > > Thanks, > > -Aleksey > > > > From vitalyd at gmail.com Fri Jan 22 13:29:09 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:29:09 -0500 Subject: API review of VarHandles In-Reply-To: <56A22A65.4040200@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> Message-ID: On Friday, January 22, 2016, David M. Lloyd wrote: > On 01/21/2016 07:32 PM, Brian Goetz wrote: > >> >> I am baffled as to how the original language syntax proposal of using >>> some trick like "xx.volatile.imaginaryMethod()" plus maybe one or two >>> new bytecodes was considered unacceptable; looking at this API, I know >>> that none of the aforementioned metrics were considered as acceptance >>> criteria. How did we get to this strange place? >>> >> >> The path was actually quite obvious. >> >> The first question was, "OK, if the language had this magic syntax, what >> bytecode would get generated? And clearly it couldn't be Unsafe. So >> either a pile of new bytecodes, or an API, was needed that the front-end >> compiler could generate against. >> >> Building APIs is generally preferable to new bytecodes, if an API can do >> the job, and it turned out that it could do it just as well. Once we had >> an API that met the requirements, it was pretty clear that language >> syntax was not only unnecessary, but likely undesirable -- these exotic >> access modes are power tools for power users (do you really want typical >> users to reason about mixed volatile and relaxed access to variables?), >> and they meet the needs of such users just fine (those users have been >> using Unsafe all along, and this is clearly better and safer.) >> >> So it was not the either-or of "either an API *or* magic language >> syntax", it was "either an API *or* syntax plus an API". At which point >> it was clear that the syntax didn't carry its own weight. >> >> I think this is a fine example of how the obvious idea was wrong, and >> that most of the value of the initial syntax idea was framing the >> problem space. I'm glad we saw fit to jettison that when it ceased to >> provide value. >> > > I will contest the "APIs is generally preferable to new bytecodes, if an > API can do the job" tack because it clearly doesn't track logically: most > of the bytecodes in the JVM today could have been intrinsic API methods > instead, e.g. Integer.add(int, int) and that sort of thing, but they > clearly benefit from being bytecodes just from a simplicity standpoint. > Having new bytecodes would certainly be simpler for javac; it would be > simpler for the user also, especially the user of class generators and > transformers (who by this point will now have to do some pretty bananas > things to transform field references); I can only speculate that it would > also be simpler for the interpreter/JIT itself as well because really > you're just generating variations on getfield, *aload, etc. But that is > all besides the point. > > Even the API itself is really heavy and weird. Experts need simple APIs > more than anyone - especially those writing complex concurrent algorithms. > They're the ones who aren't content to copy and paste blobs off of > StackOverflow; rather they need to be able to prove that their program is > correct, and the more complexity there is, the more difficult a task that > becomes. But even the premise that this API is experts-only is > self-fulfilling: atomics and fences are quite useful to less advanced users > too, especially simple get-and-set and CAS operations. But there is little > chance that this API is going to be used (or used correctly) by such users. Experts need control more than anything else. This is not to say they'll be happy with a subpar API but control typically means lots of details so there's a limit on how abstracted the API can be. As mentioned, given the API provides low level control one can build a higher level one for their own needs. The topic of ordering is complex by its nature, there's a danger in hiding details in more abstract API. > > I have no problem with the methods present on the final product, in terms > of the types of fences and operations. But the setup procedures and > concepts are far more complex than necessary, and I don't see this being > useful to most users in its current shape as most people will flee from it. > > -- > - DML > -- Sent from my phone From david.lloyd at redhat.com Fri Jan 22 13:34:53 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 07:34:53 -0600 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> Message-ID: <56A22FFD.3090104@redhat.com> On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: > Experts need control more than anything else. This is not to say they'll > be happy with a subpar API but control typically means lots of details so > there's a limit on how abstracted the API can be. As mentioned, given the > API provides low level control one can build a higher level one for their > own needs. The topic of ordering is complex by its nature, there's a > danger in hiding details in more abstract API. I understand that, and that is reasonable. However, will it be possible to build higher level APIs on this that do not suffer the same costs that make the existing Atomic* classes unattractive? -- - DML From vitalyd at gmail.com Fri Jan 22 13:40:51 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:40:51 -0500 Subject: API review of VarHandles In-Reply-To: <56A22FFD.3090104@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> Message-ID: Which existing Atomic* classes? I take it you mean the field updaters and not AtomicInteger and friends. The biggest issue with field updaters is their performance, for me. I take it you mean something else by "costs" though? Having to specify the target (field) as a string is annoying, and I do wish there was a way javac could assist there. VH is no worse in this respect, but also no better. On Fri, Jan 22, 2016 at 8:34 AM, David M. Lloyd wrote: > On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: > >> Experts need control more than anything else. This is not to say they'll >> be happy with a subpar API but control typically means lots of details so >> there's a limit on how abstracted the API can be. As mentioned, given the >> API provides low level control one can build a higher level one for their >> own needs. The topic of ordering is complex by its nature, there's a >> danger in hiding details in more abstract API. >> > > I understand that, and that is reasonable. However, will it be possible > to build higher level APIs on this that do not suffer the same costs that > make the existing Atomic* classes unattractive? > > -- > - DML > From david.lloyd at redhat.com Fri Jan 22 13:54:26 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 07:54:26 -0600 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> Message-ID: <56A23492.4030006@redhat.com> The costs are both performance and memory overhead. The Atomic*FieldUpdaters are good for memory usage and reasonably usable, but suffer from performance problems and generics issues. The "regular" Atomic* classes have good usability (including with generic types) and generally very good performance on the actual operations, but generally unacceptable memory overhead. The only way to do atomic array element access is via the Atomic*Array though, which has pretty good performance but again has memory overhead that can be unacceptable. Overall this whole thing is a lot of gymnastics for something that's basically a fancy way of accessing a plain field or array element - one of the most basic concepts in Java. On 01/22/2016 07:40 AM, Vitaly Davidovich wrote: > Which existing Atomic* classes? I take it you mean the field updaters and > not AtomicInteger and friends. The biggest issue with field updaters is > their performance, for me. I take it you mean something else by "costs" > though? Having to specify the target (field) as a string is annoying, and I > do wish there was a way javac could assist there. VH is no worse in this > respect, but also no better. > > On Fri, Jan 22, 2016 at 8:34 AM, David M. Lloyd > wrote: > >> On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: >> >>> Experts need control more than anything else. This is not to say they'll >>> be happy with a subpar API but control typically means lots of details so >>> there's a limit on how abstracted the API can be. As mentioned, given the >>> API provides low level control one can build a higher level one for their >>> own needs. The topic of ordering is complex by its nature, there's a >>> danger in hiding details in more abstract API. >>> >> >> I understand that, and that is reasonable. However, will it be possible >> to build higher level APIs on this that do not suffer the same costs that >> make the existing Atomic* classes unattractive? >> >> -- >> - DML >> > -- - DML From aleksey.shipilev at oracle.com Fri Jan 22 13:57:13 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 16:57:13 +0300 Subject: API review of VarHandles In-Reply-To: <56A23492.4030006@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> Message-ID: <56A23539.4000702@oracle.com> On 01/22/2016 04:54 PM, David M. Lloyd wrote: > The costs are both performance and memory overhead. The > Atomic*FieldUpdaters are good for memory usage and reasonably usable, > but suffer from performance problems and generics issues. Hopefully, not anymore: http://shipilev.net/blog/2015/faster-atomic-fu/ BTW, those are the same techniques we use to compile VarHandles down to the naked memory accesses. Cheers, -Aleksey From vitalyd at gmail.com Fri Jan 22 13:58:48 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 08:58:48 -0500 Subject: API review of VarHandles In-Reply-To: <56A23492.4030006@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> Message-ID: So in this regard, VH ought to be better since it will have performance like Unsafe (rather than field updaters - is that accurate by the way? Is VH on par with Unsafe for issuing atomic ops?) and not have the memory bloat of the instance atomic classes. Generics is a non-issue for VH. I do agree with the "this is all just hoop jumping to issue basic memory ops", but that's more a reflection on Java (memory footprint of atomic instance classes, generics issues) rather than VH. On Fri, Jan 22, 2016 at 8:54 AM, David M. Lloyd wrote: > The costs are both performance and memory overhead. The > Atomic*FieldUpdaters are good for memory usage and reasonably usable, but > suffer from performance problems and generics issues. The "regular" > Atomic* classes have good usability (including with generic types) and > generally very good performance on the actual operations, but generally > unacceptable memory overhead. The only way to do atomic array element > access is via the Atomic*Array though, which has pretty good performance > but again has memory overhead that can be unacceptable. > > Overall this whole thing is a lot of gymnastics for something that's > basically a fancy way of accessing a plain field or array element - one of > the most basic concepts in Java. > > > On 01/22/2016 07:40 AM, Vitaly Davidovich wrote: > >> Which existing Atomic* classes? I take it you mean the field updaters and >> not AtomicInteger and friends. The biggest issue with field updaters is >> their performance, for me. I take it you mean something else by "costs" >> though? Having to specify the target (field) as a string is annoying, and >> I >> do wish there was a way javac could assist there. VH is no worse in this >> respect, but also no better. >> >> On Fri, Jan 22, 2016 at 8:34 AM, David M. Lloyd >> wrote: >> >> On 01/22/2016 07:29 AM, Vitaly Davidovich wrote: >>> >>> Experts need control more than anything else. This is not to say they'll >>>> be happy with a subpar API but control typically means lots of details >>>> so >>>> there's a limit on how abstracted the API can be. As mentioned, given >>>> the >>>> API provides low level control one can build a higher level one for >>>> their >>>> own needs. The topic of ordering is complex by its nature, there's a >>>> danger in hiding details in more abstract API. >>>> >>>> >>> I understand that, and that is reasonable. However, will it be possible >>> to build higher level APIs on this that do not suffer the same costs that >>> make the existing Atomic* classes unattractive? >>> >>> -- >>> - DML >>> >>> >> > -- > - DML > From coleen.phillimore at oracle.com Fri Jan 22 14:00:53 2016 From: coleen.phillimore at oracle.com (Coleen Phillimore) Date: Fri, 22 Jan 2016 09:00:53 -0500 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A1F24C.7050205@redhat.com> References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> Message-ID: <56A23615.6090105@oracle.com> On 1/22/16 4:11 AM, Andrew Dinn wrote: > On 21/01/16 22:14, Rafael Winterhalter wrote: >> Hi Andrew, >> if there is any update on the matter, I would of course love to hear about it. >> Unfortunately, I never received an answer to my question, but maybe I >> picked the wrong list. >> Thank you for your support on this issue! > I'm pretty sure this is the right list. But I don't know who cold answer > the question. I know that Coleen Phillmore (added explicitly to CC) > often deals with ClassFileTransformer issues. Coleen, can you answer? Can you send the question again? I didn't see it. I also mostly look at JVM code and rarely deal with the Java side. I'm also adding Dan, Serguei and Markus. Thanks, Coleen > > regards, > > > Andrew Dinn > ----------- > Senior Principal Software Engineer > Red Hat UK Ltd > Registered in UK and Wales under Company Registration No. 3798903 > Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul > Argiry (US) From rafael.wth at gmail.com Fri Jan 22 14:04:57 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 22 Jan 2016 15:04:57 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A23615.6090105@oracle.com> References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> Message-ID: Hi Coleen, thanks for looking into this. My original mail was the following: Hello everybody, classes that are loaded via Unsafe::defineAnonymousClass are not transformed by a registered ClassFileTransformer. At the same time, it is possible to retransform / redefine such an anonymous classes using the instrumentation API. Here is a rather confusing bug that I encountered when working with an internally used Java agent after upgrading a code base to Java 8: The Java agent attaches at runtime (agentmain). It then registers a ClassFileTransformer that also applies for retransformation. Afterwards, all loaded classes that fullfil a given condition are explicitly registered to be retransformed by the agent. (Doing so, anonymous classes are returned by Instrumentation::getAllLoadedClasses.) This resulted in the following behavior of the agent: a) If the agent was attached after an anonymous class was loaded, the retransformation would apply to an anonymous class. b) If the agent was attached "too early", the (non-re-)transformation would not apply to an anonymous class. I wonder if it is intentional that a ClassFileTransformer is not applied when an anonymous class is loaded. Personally, I find this counter-intuitive, especially when converting anonymous inner classes to lambda expressions where many users of instrumentation do not forsee the behavioral change. It also puts a very unforseeable limit to the instrumentation API. I would therefore like to suggest that ClassFileTransformers are also applied to anonymous classes when Unsafe::defineAnonymousClass is called just as when going via ClassLoader::defineClass. I can tell that this behavior has not only affected me as I had this question comming up by multiple users of my open-source code generation library. What is your view on this? Thank you for your feedback! Best regards, Rafael 2016-01-22 15:00 GMT+01:00 Coleen Phillimore : > > > On 1/22/16 4:11 AM, Andrew Dinn wrote: >> >> On 21/01/16 22:14, Rafael Winterhalter wrote: >>> >>> Hi Andrew, >>> if there is any update on the matter, I would of course love to hear >>> about it. >>> Unfortunately, I never received an answer to my question, but maybe I >>> picked the wrong list. >>> Thank you for your support on this issue! >> >> I'm pretty sure this is the right list. But I don't know who cold answer >> the question. I know that Coleen Phillmore (added explicitly to CC) >> often deals with ClassFileTransformer issues. Coleen, can you answer? > > > Can you send the question again? I didn't see it. I also mostly look at > JVM code and rarely deal with the Java side. > > I'm also adding Dan, Serguei and Markus. > > Thanks, > Coleen > >> >> regards, >> >> >> Andrew Dinn >> ----------- >> Senior Principal Software Engineer >> Red Hat UK Ltd >> Registered in UK and Wales under Company Registration No. 3798903 >> Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul >> Argiry (US) > > From david.lloyd at redhat.com Fri Jan 22 14:04:23 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 22 Jan 2016 08:04:23 -0600 Subject: API review of VarHandles In-Reply-To: <56A23539.4000702@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> <56A23539.4000702@oracle.com> Message-ID: <56A236E7.8070601@redhat.com> On 01/22/2016 07:57 AM, Aleksey Shipilev wrote: > On 01/22/2016 04:54 PM, David M. Lloyd wrote: >> The costs are both performance and memory overhead. The >> Atomic*FieldUpdaters are good for memory usage and reasonably usable, >> but suffer from performance problems and generics issues. > > Hopefully, not anymore: > http://shipilev.net/blog/2015/faster-atomic-fu/ > > BTW, those are the same techniques we use to compile VarHandles down to > the naked memory accesses. Great, nice work, and nice article! -- - DML From vitalyd at gmail.com Fri Jan 22 14:14:43 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 09:14:43 -0500 Subject: API review of VarHandles In-Reply-To: <56A236E7.8070601@redhat.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A22A65.4040200@redhat.com> <56A22FFD.3090104@redhat.com> <56A23492.4030006@redhat.com> <56A23539.4000702@oracle.com> <56A236E7.8070601@redhat.com> Message-ID: +1, good stuff Aleksey (I had actually seen chatter about this change on one of these openjdk lists, but people have already moved away from the field updaters), except every time I think about final instance fields not being treated as constants I get sad :). The other wrinkle is the "am I lucky today to get proper inlining" in order to peel away costs. I'm not sure whether VH machinery will make this work reliably, irrespective of callsite frequency, inlining depth, etc (that's another advantage of bytecode -- there's no need to crack method chains). On Fri, Jan 22, 2016 at 9:04 AM, David M. Lloyd wrote: > On 01/22/2016 07:57 AM, Aleksey Shipilev wrote: > >> On 01/22/2016 04:54 PM, David M. Lloyd wrote: >> >>> The costs are both performance and memory overhead. The >>> Atomic*FieldUpdaters are good for memory usage and reasonably usable, >>> but suffer from performance problems and generics issues. >>> >> >> Hopefully, not anymore: >> http://shipilev.net/blog/2015/faster-atomic-fu/ >> >> BTW, those are the same techniques we use to compile VarHandles down to >> the naked memory accesses. >> > > Great, nice work, and nice article! > -- > - DML > From chris.hegarty at oracle.com Fri Jan 22 14:22:05 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Jan 2016 14:22:05 +0000 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A21B81.7090800@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> Message-ID: <56A23B0D.2050407@oracle.com> Claes, On 22/01/16 12:07, Claes Redestad wrote: > On 2016-01-22 12:56, Claes Redestad wrote: >> On 2016-01-22 12:53, Alan Bateman wrote: >>> On 22/01/2016 11:34, Claes Redestad wrote: >>>> : >>>> During testing I discovered that EnumSet was missing a >>>> serialVersionUID, which needs to be added. >>> I assume this is not needed, it already has: >>> >>> @SuppressWarnings("serial") // No serialVersionUID due to usage of >>> serial proxy pattern >> >> That's odd.. there's a jtreg test that fails if I don't explicitly add >> the serialVersionUID (that's where I got the actual UID from). I'll >> investigate. > > Right... java/util/EnumSet/BogusEnumSet.java fails, which is an > explicit test to verify a corrupt EnumSet doesn't deserialize properly: Hmmm... this would indicate that the changes you have made has affected the serial version UID automatically generated. I'm not sure why removing a 'private static' field would do this. Is it possible that you ran into this during development, but it is not actually necessary with the final changes? > java.io.InvalidClassException: java.util.EnumSet; local class > incompatible: stream classdesc serialVersionUID = 1009687484059888093, > local class serialVersionUID = 4168005130090799668 > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:618) > at > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1626) > at > java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1626) > at > java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521) > at > java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1777) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1354) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:368) > at BogusEnumSet.deserialize(BogusEnumSet.java:91) > at BogusEnumSet.main(BogusEnumSet.java:78) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:520) > at > com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:218) > > at java.lang.Thread.run(Thread.java:804) This test is a little fragile, but should be ok. > It would seem the assessment that serialVersionUID is not needed when > using the serial proxy pattern might be misguided (or is this a bug in > ObjectInputStream)? It is debate-able whether this is a test bug or not. But I think we should not change the serialVersionUID for this class, whether truly Serializable or not. Even though in practice it is not all that useful. Can you please revert the serialization parts of the change, and rerun the test? To see if still passes or fails. -Chris. From Alan.Bateman at oracle.com Fri Jan 22 14:29:46 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 22 Jan 2016 14:29:46 +0000 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A23615.6090105@oracle.com> References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> Message-ID: <56A23CDA.1030202@oracle.com> On 22/01/2016 14:00, Coleen Phillimore wrote: > Can you send the question again? I didn't see it. I also mostly look > at JVM code and rarely deal with the Java side. > The question was asked about ClassFileTransformer which is implemented by the JPLIS agent via the JVM TI CFLH event. So I think the question is really on whether this event can be posted for classes defined via defineAnonymousClass. None of the standard APIs know about HotSpot specific concepts like "host class" but maybe the host class defining loader and PD can be used with this event. I don't know if this has been explored elsewhere already. -Alan. From peter.levart at gmail.com Fri Jan 22 14:44:41 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jan 2016 15:44:41 +0100 Subject: JEP 280 (Indify String Concat) integration In-Reply-To: <56A1F6D8.9070808@oracle.com> References: <56A1F6D8.9070808@oracle.com> Message-ID: <56A24059.5030206@gmail.com> Hi Aleksey, Good work. Nice to see this integrated. If StringConcatException ever gets thrown, will it propagate out of the concatenation expression execution? If yes, then it should probably be an unchecked exception, right? Regards, Peter On 01/22/2016 10:31 AM, Aleksey Shipilev wrote: > Hi, > > I think it is time to integrate JEP 280 ("Indify String Concat") into JDK 9: > http://openjdk.java.net/jeps/280 > > The JEP is Targeted, the CCC is approved, the code reviews and > pre-integration checks come clean. In theory, we would like to integrate > with some "optimal" concat strategy switched by default, but we have to > wait for some Hotspot compiler changes to settle before we can make that > call. (Notably, https://bugs.openjdk.java.net/browse/JDK-8136469). > > So, I would like to proceed in two parts: > > Part 1: Integrate all code changes, but keep the concatenation strategy > to be the same "naive" StringBuilder.append chains. In other words, we > are emitting string concat indy, but always link in the naive > StringBuilder.append code. > > Part 2: (After HS changes settle in) Re-run performance tests, and > switch to a more optimal concatenation strategy. This amounts to > one-liner change in a default value for Java property. > > Thoughts? I would like relevant Project Leads / Reviewers (langtools and > jdk) to OKay the integration. > > FTR, the webrevs: > http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ > http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ > http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ > > Thanks, > -Aleksey > From brian.goetz at oracle.com Fri Jan 22 14:46:08 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Fri, 22 Jan 2016 09:46:08 -0500 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> Message-ID: <56A240B0.5040108@oracle.com> > The VarHandle API isn't ergonomic like Unsafe; this being for power > users is irrelevant to the ergonomics of the API. Instead, it's fairly > verbose with setup ceremony. Now, I personally don't mind that too > much and more interested in the features it provides but I'm not > surprised by David's reaction. > > People already provide nicer APIs on top of Unsafe in their own > projects, and I suspect this will be even more so with VH. And that's absolutely fine with us. We wanted to provide a secure, safe, fast substrate that code generators and brave library writers alike will use; if people want to wrap it, we view that as success, not failure. From chris.hegarty at oracle.com Fri Jan 22 14:49:46 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Jan 2016 14:49:46 +0000 Subject: RFR 8065076/9, test/java/net/SocketPermission/SocketPermissionTest.java failed intermittently In-Reply-To: <67D3FEF2-7B00-4C90-8D87-6258B7AE547A@oracle.com> References: <568F74F7.70100@oracle.com> <56973B1A.30000@oracle.com> <67D3FEF2-7B00-4C90-8D87-6258B7AE547A@oracle.com> Message-ID: <56A2418A.1060903@oracle.com> On 21/01/16 22:55, Felix Yang wrote: > Hi Chris, > your fix is cool. I will assign the bug to you:) > a comment on this fix. The test changed system SecurityManager and > it is not executed with othervm mode. I think you need to rollback the > change after test. I will revert the change and have the test run in othervm mode. I did do a complete test run with this change and it did not cause any problems, but then again the policy is all permissions! Thanks Felix. -Chris. > Otherwise it may affect other tests. > > Thanks, > Felix >> On Jan 20, 2016, at 12:45 PM, Chris Hegarty > > wrote: >> >> On 20 Jan 2016, at 06:36, Chris Hegarty > > wrote: >> >>> Felix, >>> >>> On 14 Jan 2016, at 06:07, Felix Yang >> > wrote: >>> >>>> Hi all, >>>> please review the fix for >>>> test/java/net/SocketPermission/SocketPermissionTest.java, which >>>> fails frequently with "java.net.BindException: Address already in use". >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065076 >>>> Webrev: http://cr.openjdk.java.net/~xiaofeya/8065076/webrev.00 >>> >>> My preference is to avoid getFreePort. It is problematic and I >>> believe just obfuscates >>> the intermittent failures further. >>> >>> In many of the test scenarios the ?listening? socket can be created >>> before the specific >>> access control context and associated permission are created. >>> >>> I?ll see if I can get some time to try this out. >> >> I spent a little time on this today. I basically rewrote the test, but >> kept the >> same test scenarios. The use of data providers was cute, but not workable >> since there is no common supertype for the socket classes. I decided to >> just expand out the test cases manually. This will give the same test >> coverage, but should be stable since it creates the sockets first, on an >> ephemeral port, and then constructs the permissions appropriately given >> that port. >> >> http://cr.openjdk.java.net/~chegar/8065076/ >> >> The webrev diffs are almost useless, just review the new file, and compare >> test scenarios against the what is in the old file. >> >> -Chris. > From vitalyd at gmail.com Fri Jan 22 14:52:26 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 22 Jan 2016 09:52:26 -0500 Subject: API review of VarHandles In-Reply-To: <56A240B0.5040108@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A16451.6010709@redhat.com> <56A186AD.8090100@oracle.com> <56A240B0.5040108@oracle.com> Message-ID: Agreed; it's a building block, not an assembled piece. On Friday, January 22, 2016, Brian Goetz wrote: > > The VarHandle API isn't ergonomic like Unsafe; this being for power users >> is irrelevant to the ergonomics of the API. Instead, it's fairly verbose >> with setup ceremony. Now, I personally don't mind that too much and more >> interested in the features it provides but I'm not surprised by David's >> reaction. >> >> People already provide nicer APIs on top of Unsafe in their own projects, >> and I suspect this will be even more so with VH. >> > > And that's absolutely fine with us. We wanted to provide a secure, safe, > fast substrate that code generators and brave library writers alike will > use; if people want to wrap it, we view that as success, not failure. > > > -- Sent from my phone From daniel.daugherty at oracle.com Fri Jan 22 15:15:04 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Fri, 22 Jan 2016 08:15:04 -0700 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A23CDA.1030202@oracle.com> References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A23CDA.1030202@oracle.com> Message-ID: <56A24778.3090001@oracle.com> On 1/22/16 7:29 AM, Alan Bateman wrote: > > On 22/01/2016 14:00, Coleen Phillimore wrote: >> Can you send the question again? I didn't see it. I also mostly >> look at JVM code and rarely deal with the Java side. >> > The question was asked about ClassFileTransformer which is implemented > by the JPLIS agent via the JVM TI CFLH event. So I think the question > is really on whether this event can be posted for classes defined via > defineAnonymousClass. None of the standard APIs know about HotSpot > specific concepts like "host class" but maybe the host class defining > loader and PD can be used with this event. I don't know if this has > been explored elsewhere already. > > -Alan. > Sounds like defineAnonymousClass() is somehow by-passing the placement of the CFLH event poster. I don't see any information here about which JDK version this issue is being seen with. I suspect JDK8u, but that's just a guess... Dan From rafael.wth at gmail.com Fri Jan 22 15:16:39 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 22 Jan 2016 16:16:39 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A24778.3090001@oracle.com> References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A23CDA.1030202@oracle.com> <56A24778.3090001@oracle.com> Message-ID: You are right. It is observed for all JDK8u releases and persists in the current builds of JDK9. Best regards, Rafael 2016-01-22 16:15 GMT+01:00 Daniel D. Daugherty : > On 1/22/16 7:29 AM, Alan Bateman wrote: >> >> >> On 22/01/2016 14:00, Coleen Phillimore wrote: >>> >>> Can you send the question again? I didn't see it. I also mostly look at >>> JVM code and rarely deal with the Java side. >>> >> The question was asked about ClassFileTransformer which is implemented by >> the JPLIS agent via the JVM TI CFLH event. So I think the question is really >> on whether this event can be posted for classes defined via >> defineAnonymousClass. None of the standard APIs know about HotSpot specific >> concepts like "host class" but maybe the host class defining loader and PD >> can be used with this event. I don't know if this has been explored >> elsewhere already. >> >> -Alan. >> > > Sounds like defineAnonymousClass() is somehow by-passing the placement > of the CFLH event poster. > > I don't see any information here about which JDK version this issue > is being seen with. I suspect JDK8u, but that's just a guess... > > Dan From vladimir.x.ivanov at oracle.com Fri Jan 22 15:42:12 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 22 Jan 2016 18:42:12 +0300 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> Message-ID: <56A24DD4.2080705@oracle.com> Rafael, What you are seeing are just consequences. My impression is that VM anonymous class redefinition/retransformation works mostly by accident. The real problem is that current API (both JVMTI and java.lang.instrument) doesn't serve for them well. When VM anonymous class is defined, a user provides both bytecode and CP patches: public native Class defineAnonymousClass(Class hostClass, byte[] data, Object[] cpPatches); Unless there's a way to adjust CP patches along with the bytecode, patching bytecode is very fragile (e.g., no way to add new patches, should keep patched CP entry indexes intact). The root cause is Unsafe.defineAnonymousClass is part of implementation-specific private API, so no way to expose it a public APIs (special entry points in JVMTI or java.lang.instrument). Also, though the concept of VM anonymous class was a step in the right direction, it isn't good enough to be standardized. The ultimate goal is to get lightweight code loading mechanism, but VM anonymous classes are still loaded with class semantics. So, I don't think we can do much w.r.t. VM anonymous classes. I'd prefer JVM to skip agent notifications or completely forbidding VM anonymous class retransformation/redefinition, but I haven't thought through compatibility consequences. Best regards, Vladimir Ivanov On 1/22/16 5:04 PM, Rafael Winterhalter wrote: > Hi Coleen, thanks for looking into this. My original mail was the following: > > Hello everybody, > > classes that are loaded via Unsafe::defineAnonymousClass are not > transformed by a registered ClassFileTransformer. At the same time, it > is possible to retransform / redefine such an anonymous classes using > the instrumentation API. > > Here is a rather confusing bug that I encountered when working with an > internally used Java agent after upgrading a code base to Java 8: > > The Java agent attaches at runtime (agentmain). It then registers a > ClassFileTransformer that also applies for retransformation. > Afterwards, all loaded classes that fullfil a given condition are > explicitly registered to be retransformed by the agent. (Doing so, > anonymous classes are returned by > Instrumentation::getAllLoadedClasses.) This resulted in the following > behavior of the agent: > > a) If the agent was attached after an anonymous class was loaded, the > retransformation would apply to an anonymous class. > b) If the agent was attached "too early", the (non-re-)transformation > would not apply to an anonymous class. > > I wonder if it is intentional that a ClassFileTransformer is not > applied when an anonymous class is loaded. Personally, I find this > counter-intuitive, especially when converting anonymous inner classes > to lambda expressions where many users of instrumentation do not > forsee the behavioral change. It also puts a very unforseeable limit > to the instrumentation API. I would therefore like to suggest that > ClassFileTransformers are also applied to anonymous classes when > Unsafe::defineAnonymousClass is called just as when going via > ClassLoader::defineClass. > > I can tell that this behavior has not only affected me as I had this > question comming up by multiple users of my open-source code > generation library. > > What is your view on this? > > Thank you for your feedback! > Best regards, Rafael > > 2016-01-22 15:00 GMT+01:00 Coleen Phillimore : >> >> >> On 1/22/16 4:11 AM, Andrew Dinn wrote: >>> >>> On 21/01/16 22:14, Rafael Winterhalter wrote: >>>> >>>> Hi Andrew, >>>> if there is any update on the matter, I would of course love to hear >>>> about it. >>>> Unfortunately, I never received an answer to my question, but maybe I >>>> picked the wrong list. >>>> Thank you for your support on this issue! >>> >>> I'm pretty sure this is the right list. But I don't know who cold answer >>> the question. I know that Coleen Phillmore (added explicitly to CC) >>> often deals with ClassFileTransformer issues. Coleen, can you answer? >> >> >> Can you send the question again? I didn't see it. I also mostly look at >> JVM code and rarely deal with the Java side. >> >> I'm also adding Dan, Serguei and Markus. >> >> Thanks, >> Coleen >> >>> >>> regards, >>> >>> >>> Andrew Dinn >>> ----------- >>> Senior Principal Software Engineer >>> Red Hat UK Ltd >>> Registered in UK and Wales under Company Registration No. 3798903 >>> Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul >>> Argiry (US) >> >> From claes.redestad at oracle.com Fri Jan 22 15:44:16 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 16:44:16 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A23B0D.2050407@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> Message-ID: <56A24E50.6050705@oracle.com> On 2016-01-22 15:22, Chris Hegarty wrote: > Claes, > > On 22/01/16 12:07, Claes Redestad wrote: >> On 2016-01-22 12:56, Claes Redestad wrote: >>> On 2016-01-22 12:53, Alan Bateman wrote: > >>>> On 22/01/2016 11:34, Claes Redestad wrote: >>>>> : >>>>> During testing I discovered that EnumSet was missing a >>>>> serialVersionUID, which needs to be added. >>>> I assume this is not needed, it already has: >>>> >>>> @SuppressWarnings("serial") // No serialVersionUID due to usage of >>>> serial proxy pattern >>> >>> That's odd.. there's a jtreg test that fails if I don't explicitly add >>> the serialVersionUID (that's where I got the actual UID from). I'll >>> investigate. >> >> Right... java/util/EnumSet/BogusEnumSet.java fails, which is an >> explicit test to verify a corrupt EnumSet doesn't deserialize properly: > > Hmmm... this would indicate that the changes you have made has > affected the serial version UID automatically generated. I'm not > sure why removing a 'private static' field would do this. Is it > possible that you ran into this during development, but it is > not actually necessary with the final changes? > This test is a little fragile, but should be ok. > >> It would seem the assessment that serialVersionUID is not needed when >> using the serial proxy pattern might be misguided (or is this a bug in >> ObjectInputStream)? > > It is debate-able whether this is a test bug or not. But I think > we should not change the serialVersionUID for this class, whether > truly Serializable or not. Even though in practice it is not all > that useful. > > Can you please revert the serialization parts of the change, and > rerun the test? To see if still passes or fails. I backed out the serial version UID change and reran the test, same thing. I dug deeper, as this is a bit vexxing... While a static field does not in and of itself cause the generated UID to change, the private ZERO_LENGTH_ENUM_ARRAY is being accessed from SerializationProxy class, thus javac generates a package-private synthetic method (access$000) which will be accounted for when generating serialVersionUID. That's unfortunate. I ran a few experiments to confirm this: - Adding a package-private static method to EnumSet makes the test fail - Adding any static field and the test keep passing So it seems our options here are limited. Thanks! /Claes From chris.hegarty at oracle.com Fri Jan 22 15:50:01 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Jan 2016 15:50:01 +0000 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A24E50.6050705@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> Message-ID: <64870C9D-FDB3-4739-B89B-E49FAA7293EB@oracle.com> > On 22 Jan 2016, at 15:44, Claes Redestad wrote: > > > > On 2016-01-22 15:22, Chris Hegarty wrote: >> Claes, >> >> On 22/01/16 12:07, Claes Redestad wrote: >>> On 2016-01-22 12:56, Claes Redestad wrote: >>>> On 2016-01-22 12:53, Alan Bateman wrote: >> >>>>> On 22/01/2016 11:34, Claes Redestad wrote: >>>>>> : >>>>>> During testing I discovered that EnumSet was missing a >>>>>> serialVersionUID, which needs to be added. >>>>> I assume this is not needed, it already has: >>>>> >>>>> @SuppressWarnings("serial") // No serialVersionUID due to usage of >>>>> serial proxy pattern >>>> >>>> That's odd.. there's a jtreg test that fails if I don't explicitly add >>>> the serialVersionUID (that's where I got the actual UID from). I'll >>>> investigate. >>> >>> Right... java/util/EnumSet/BogusEnumSet.java fails, which is an >>> explicit test to verify a corrupt EnumSet doesn't deserialize properly: >> >> Hmmm... this would indicate that the changes you have made has >> affected the serial version UID automatically generated. I'm not >> sure why removing a 'private static' field would do this. Is it >> possible that you ran into this during development, but it is >> not actually necessary with the final changes? > >> This test is a little fragile, but should be ok. >> >>> It would seem the assessment that serialVersionUID is not needed when >>> using the serial proxy pattern might be misguided (or is this a bug in >>> ObjectInputStream)? >> >> It is debate-able whether this is a test bug or not. But I think >> we should not change the serialVersionUID for this class, whether >> truly Serializable or not. Even though in practice it is not all >> that useful. >> >> Can you please revert the serialization parts of the change, and >> rerun the test? To see if still passes or fails. > > I backed out the serial version UID change and reran the test, same thing. > > I dug deeper, as this is a bit vexxing... > > While a static field does not in and of itself cause the generated UID to change, > the private ZERO_LENGTH_ENUM_ARRAY is being accessed from SerializationProxy > class, thus javac generates a package-private synthetic method (access$000) which > will be accounted for when generating serialVersionUID. That's unfortunate. > > I ran a few experiments to confirm this: > > - Adding a package-private static method to EnumSet makes the test fail > - Adding any static field and the test keep passing > > So it seems our options here are limited. +1 to your latest webrev. I just wanted to understand the reason for the serial version UID change, which you have provided. -Chris. > > Thanks! > > /Claes From Roger.Riggs at Oracle.com Fri Jan 22 15:50:30 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 22 Jan 2016 10:50:30 -0500 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A24E50.6050705@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> Message-ID: <56A24FC6.4060107@Oracle.com> Hi Claes, I'm not sure it helps much, but making the ZERO_LENGTH_ENUM_ARRAY package-private (instead of private) will eliminate the need for the accessor. It is not mutable so there's no extra risk. Roger On 1/22/2016 10:44 AM, Claes Redestad wrote: > > > On 2016-01-22 15:22, Chris Hegarty wrote: >> Claes, >> >> On 22/01/16 12:07, Claes Redestad wrote: >>> On 2016-01-22 12:56, Claes Redestad wrote: >>>> On 2016-01-22 12:53, Alan Bateman wrote: >> >>>>> On 22/01/2016 11:34, Claes Redestad wrote: >>>>>> : >>>>>> During testing I discovered that EnumSet was missing a >>>>>> serialVersionUID, which needs to be added. >>>>> I assume this is not needed, it already has: >>>>> >>>>> @SuppressWarnings("serial") // No serialVersionUID due to usage of >>>>> serial proxy pattern >>>> >>>> That's odd.. there's a jtreg test that fails if I don't explicitly add >>>> the serialVersionUID (that's where I got the actual UID from). I'll >>>> investigate. >>> >>> Right... java/util/EnumSet/BogusEnumSet.java fails, which is an >>> explicit test to verify a corrupt EnumSet doesn't deserialize properly: >> >> Hmmm... this would indicate that the changes you have made has >> affected the serial version UID automatically generated. I'm not >> sure why removing a 'private static' field would do this. Is it >> possible that you ran into this during development, but it is >> not actually necessary with the final changes? > >> This test is a little fragile, but should be ok. >> >>> It would seem the assessment that serialVersionUID is not needed when >>> using the serial proxy pattern might be misguided (or is this a bug in >>> ObjectInputStream)? >> >> It is debate-able whether this is a test bug or not. But I think >> we should not change the serialVersionUID for this class, whether >> truly Serializable or not. Even though in practice it is not all >> that useful. >> >> Can you please revert the serialization parts of the change, and >> rerun the test? To see if still passes or fails. > > I backed out the serial version UID change and reran the test, same > thing. > > I dug deeper, as this is a bit vexxing... > > While a static field does not in and of itself cause the generated UID > to change, > the private ZERO_LENGTH_ENUM_ARRAY is being accessed from > SerializationProxy > class, thus javac generates a package-private synthetic method > (access$000) which > will be accounted for when generating serialVersionUID. That's > unfortunate. > > I ran a few experiments to confirm this: > > - Adding a package-private static method to EnumSet makes the test fail > - Adding any static field and the test keep passing > > So it seems our options here are limited. > > Thanks! > > /Claes From claes.redestad at oracle.com Fri Jan 22 15:56:54 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 16:56:54 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A24FC6.4060107@Oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> <56A24FC6.4060107@Oracle.com> Message-ID: <56A25146.7020307@oracle.com> Hi Roger, right, I accidentally eliminated the accessor by (re-/)moving the constant which altered the serialization shape, which was the issue here. Widening the field to package-private would have caused the same issue. Thanks! /Claes On 2016-01-22 16:50, Roger Riggs wrote: > Hi Claes, > > I'm not sure it helps much, but making the ZERO_LENGTH_ENUM_ARRAY > package-private > (instead of private) will eliminate the need for the accessor. It is > not mutable so there's no extra risk. > > Roger > > > On 1/22/2016 10:44 AM, Claes Redestad wrote: >> >> >> On 2016-01-22 15:22, Chris Hegarty wrote: >>> Claes, >>> >>> On 22/01/16 12:07, Claes Redestad wrote: >>>> On 2016-01-22 12:56, Claes Redestad wrote: >>>>> On 2016-01-22 12:53, Alan Bateman wrote: >>> >>>>>> On 22/01/2016 11:34, Claes Redestad wrote: >>>>>>> : >>>>>>> During testing I discovered that EnumSet was missing a >>>>>>> serialVersionUID, which needs to be added. >>>>>> I assume this is not needed, it already has: >>>>>> >>>>>> @SuppressWarnings("serial") // No serialVersionUID due to usage of >>>>>> serial proxy pattern >>>>> >>>>> That's odd.. there's a jtreg test that fails if I don't explicitly >>>>> add >>>>> the serialVersionUID (that's where I got the actual UID from). I'll >>>>> investigate. >>>> >>>> Right... java/util/EnumSet/BogusEnumSet.java fails, which is an >>>> explicit test to verify a corrupt EnumSet doesn't deserialize >>>> properly: >>> >>> Hmmm... this would indicate that the changes you have made has >>> affected the serial version UID automatically generated. I'm not >>> sure why removing a 'private static' field would do this. Is it >>> possible that you ran into this during development, but it is >>> not actually necessary with the final changes? >> >>> This test is a little fragile, but should be ok. >>> >>>> It would seem the assessment that serialVersionUID is not needed when >>>> using the serial proxy pattern might be misguided (or is this a bug in >>>> ObjectInputStream)? >>> >>> It is debate-able whether this is a test bug or not. But I think >>> we should not change the serialVersionUID for this class, whether >>> truly Serializable or not. Even though in practice it is not all >>> that useful. >>> >>> Can you please revert the serialization parts of the change, and >>> rerun the test? To see if still passes or fails. >> >> I backed out the serial version UID change and reran the test, same >> thing. >> >> I dug deeper, as this is a bit vexxing... >> >> While a static field does not in and of itself cause the generated >> UID to change, >> the private ZERO_LENGTH_ENUM_ARRAY is being accessed from >> SerializationProxy >> class, thus javac generates a package-private synthetic method >> (access$000) which >> will be accounted for when generating serialVersionUID. That's >> unfortunate. >> >> I ran a few experiments to confirm this: >> >> - Adding a package-private static method to EnumSet makes the test fail >> - Adding any static field and the test keep passing >> >> So it seems our options here are limited. >> >> Thanks! >> >> /Claes > From claes.redestad at oracle.com Fri Jan 22 15:57:21 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 16:57:21 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <64870C9D-FDB3-4739-B89B-E49FAA7293EB@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> <64870C9D-FDB3-4739-B89B-E49FAA7293EB@oracle.com> Message-ID: <56A25161.7040302@oracle.com> On 2016-01-22 16:50, Chris Hegarty wrote: > > +1 to your latest webrev. I just wanted to understand the reason for > the serial version UID change, which you have provided. > > -Chris. Thanks! /Claes From varming at gmail.com Fri Jan 22 16:14:32 2016 From: varming at gmail.com (Carsten Varming) Date: Fri, 22 Jan 2016 11:14:32 -0500 Subject: OpenJDK8: java.util.stream.Stream.onClose Message-ID: Dear core-libs-devs, I was playing around with streams and onClose and ran into an example with odd behavior (well, I found it odd, see example below). I was wondering if you could point me to prior discussions on semantics of intermediate operations and the semantics of Stream.onClose. A bit of code so we have something concrete to discuss: import java.util.stream.Stream; public class Example { public static void main(String[] args) { final Stream foo = Stream.of("1", "2", "3"); final Stream bar = foo.onClose(() -> System.out.println("Closing bar")); try (final Stream s = foo) { s.forEach(System.out::println); } } } When I run the program above I get: 1 2 3 Closing bar Hmm, the onClose call mutated foo. This is surprising to me as when I read [1] it says onClose is an intermediate operation and I quote: "Returns an equivalent stream with an additional close handler.". The documentation does not mention that the stream passed to onClose is mutated. Is this the intended behavior or is it a bug? I hope it is a bug, but I would like to understand the reasons for the current implementation. [1] https://docs.oracle.com/javase/8/docs/api/java/util/stream/BaseStream.html#onClose-java.lang.Runnable- Carsten From rafael.wth at gmail.com Fri Jan 22 16:19:12 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Fri, 22 Jan 2016 17:19:12 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A24DD4.2080705@oracle.com> References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> Message-ID: Hi Vladimir, thank you for your response. While I completely understand your view from a VM implementor's point of view, as a practicioner I would recommend against it. Not being able to instrument lambda expressions puts a severe limitation onto using the instrumentation API alltogether. For example, a monitoring application that promises to record all invocations of a method of a certain interface that only works 95% of the time is not 5% less usefull but might no longer be useful at all. People have build large applications based on the assumption that all user application code can be instrumented and such a regression would hurt them. For example, until today, over 30 people that use my code generation framework reached out to me and reported the reported behavior as a bug in my library and people are only starting to migrate their applications to Java 8. The outcome is simply not intuitive as using a lambda expression over an anonyous inner class should not change an application's behavior. The currently used workaround that people use is to instrument the LambdaMetaFactory class itself to apply the transformer manually when it is being created. This solution is spreading as a standard approach and I am sure that forbidding a redefinition alltogether would only inspire to other workarrounds for being able to migrate running code to the next Java version. Furthermore, I do not think that not being able to patch constant pool indices in the generated code introduces any problems in practice. Most real-world instrumentation only appends new constant pool entries without interfering with the existant pool. Rather, I would like to see an exception being raised if one attempts to change the original constant pool without me being able to consider this from a technical perspective. I think this would serve to be sufficient for most people. I really hope that there is a way to allow for patching anonymously loaded classes even without exposing the constant pool patches. So far, thank you for looking into this. I am sure this a more complex matter than what I am able to comprehend. I appreciate that you are taking the time to consider my opinion! Best regards, Rafael 2016-01-22 16:42 GMT+01:00 Vladimir Ivanov : > Rafael, > > What you are seeing are just consequences. My impression is that VM > anonymous class redefinition/retransformation works mostly by accident. > > The real problem is that current API (both JVMTI and java.lang.instrument) > doesn't serve for them well. > > When VM anonymous class is defined, a user provides both bytecode and CP > patches: > public native Class defineAnonymousClass(Class hostClass, byte[] > data, Object[] cpPatches); > > Unless there's a way to adjust CP patches along with the bytecode, patching > bytecode is very fragile (e.g., no way to add new patches, should keep > patched CP entry indexes intact). > > The root cause is Unsafe.defineAnonymousClass is part of > implementation-specific private API, so no way to expose it a public APIs > (special entry points in JVMTI or java.lang.instrument). > > Also, though the concept of VM anonymous class was a step in the right > direction, it isn't good enough to be standardized. The ultimate goal is to > get lightweight code loading mechanism, but VM anonymous classes are still > loaded with class semantics. > > So, I don't think we can do much w.r.t. VM anonymous classes. I'd prefer JVM > to skip agent notifications or completely forbidding VM anonymous class > retransformation/redefinition, but I haven't thought through compatibility > consequences. > > Best regards, > Vladimir Ivanov > > > On 1/22/16 5:04 PM, Rafael Winterhalter wrote: >> >> Hi Coleen, thanks for looking into this. My original mail was the >> following: >> >> Hello everybody, >> >> classes that are loaded via Unsafe::defineAnonymousClass are not >> transformed by a registered ClassFileTransformer. At the same time, it >> is possible to retransform / redefine such an anonymous classes using >> the instrumentation API. >> >> Here is a rather confusing bug that I encountered when working with an >> internally used Java agent after upgrading a code base to Java 8: >> >> The Java agent attaches at runtime (agentmain). It then registers a >> ClassFileTransformer that also applies for retransformation. >> Afterwards, all loaded classes that fullfil a given condition are >> explicitly registered to be retransformed by the agent. (Doing so, >> anonymous classes are returned by >> Instrumentation::getAllLoadedClasses.) This resulted in the following >> behavior of the agent: >> >> a) If the agent was attached after an anonymous class was loaded, the >> retransformation would apply to an anonymous class. >> b) If the agent was attached "too early", the (non-re-)transformation >> would not apply to an anonymous class. >> >> I wonder if it is intentional that a ClassFileTransformer is not >> applied when an anonymous class is loaded. Personally, I find this >> counter-intuitive, especially when converting anonymous inner classes >> to lambda expressions where many users of instrumentation do not >> forsee the behavioral change. It also puts a very unforseeable limit >> to the instrumentation API. I would therefore like to suggest that >> ClassFileTransformers are also applied to anonymous classes when >> Unsafe::defineAnonymousClass is called just as when going via >> ClassLoader::defineClass. >> >> I can tell that this behavior has not only affected me as I had this >> question comming up by multiple users of my open-source code >> generation library. >> >> What is your view on this? >> >> Thank you for your feedback! >> Best regards, Rafael >> >> 2016-01-22 15:00 GMT+01:00 Coleen Phillimore >> : >>> >>> >>> >>> On 1/22/16 4:11 AM, Andrew Dinn wrote: >>>> >>>> >>>> On 21/01/16 22:14, Rafael Winterhalter wrote: >>>>> >>>>> >>>>> Hi Andrew, >>>>> if there is any update on the matter, I would of course love to hear >>>>> about it. >>>>> Unfortunately, I never received an answer to my question, but maybe I >>>>> picked the wrong list. >>>>> Thank you for your support on this issue! >>>> >>>> >>>> I'm pretty sure this is the right list. But I don't know who cold answer >>>> the question. I know that Coleen Phillmore (added explicitly to CC) >>>> often deals with ClassFileTransformer issues. Coleen, can you answer? >>> >>> >>> >>> Can you send the question again? I didn't see it. I also mostly look at >>> JVM code and rarely deal with the Java side. >>> >>> I'm also adding Dan, Serguei and Markus. >>> >>> Thanks, >>> Coleen >>> >>>> >>>> regards, >>>> >>>> >>>> Andrew Dinn >>>> ----------- >>>> Senior Principal Software Engineer >>>> Red Hat UK Ltd >>>> Registered in UK and Wales under Company Registration No. 3798903 >>>> Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul >>>> Argiry (US) >>> >>> >>> > From aleksey.shipilev at oracle.com Fri Jan 22 17:37:29 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 22 Jan 2016 20:37:29 +0300 Subject: JEP 280 (Indify String Concat) integration In-Reply-To: <56A24059.5030206@gmail.com> References: <56A1F6D8.9070808@oracle.com> <56A24059.5030206@gmail.com> Message-ID: <56A268D9.3040504@oracle.com> On 01/22/2016 05:44 PM, Peter Levart wrote: > If StringConcatException ever gets thrown, will it propagate out of the > concatenation expression execution? If yes, then it should probably be > an unchecked exception, right? StringConcatException is modeled after LambdaConversionException: it covers the linkage invariants, and it is thrown from bootstrap method only. When BSM throws, call site initialization fails with BootstrapMethodError, having this exception as the cause. Therefore, I see no particular reason in making that exception unchecked. Thanks, -Aleksey From paul.sandoz at oracle.com Fri Jan 22 17:44:40 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 22 Jan 2016 18:44:40 +0100 Subject: OpenJDK8: java.util.stream.Stream.onClose In-Reply-To: References: Message-ID: Hi Carsten, If you read the documentation further on down it states: * first exception, since an exception cannot suppress itself.) May * return itself. The same applies to parallel/sequential/unordered. For these cases it would be rather inefficient to create new stages in the pipeline, for essentially functional no-ops regarding actual element processing, and would likely increase the cost when performing the terminal operation. Paul. > On 22 Jan 2016, at 17:14, Carsten Varming wrote: > > Dear core-libs-devs, > > I was playing around with streams and onClose and ran into an example with > odd behavior (well, I found it odd, see example below). I was wondering if > you could point me to prior discussions on semantics of intermediate > operations and the semantics of Stream.onClose. > > A bit of code so we have something concrete to discuss: > > import java.util.stream.Stream; > > public class Example { > public static void main(String[] args) { > final Stream foo = Stream.of("1", "2", "3"); > final Stream bar = foo.onClose(() -> > System.out.println("Closing bar")); > > try (final Stream s = foo) { > s.forEach(System.out::println); > } > } > } > > When I run the program above I get: > 1 > 2 > 3 > Closing bar > > Hmm, the onClose call mutated foo. This is surprising to me as when I read > [1] it says onClose is an intermediate operation and I quote: "Returns an > equivalent stream with an additional close handler.". The documentation > does not mention that the stream passed to onClose is mutated. Is this the > intended behavior or is it a bug? I hope it is a bug, but I would like to > understand the reasons for the current implementation. > > [1] > https://docs.oracle.com/javase/8/docs/api/java/util/stream/BaseStream.html#onClose-java.lang.Runnable- > > Carsten From vladimir.x.ivanov at oracle.com Fri Jan 22 17:47:31 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 22 Jan 2016 20:47:31 +0300 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A09FDD.9080305@redhat.com> <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> Message-ID: <56A26B33.2010308@oracle.com> Rafael, First of all, I'd like to agree on the terminology. There's some confusion there. Anonymous term is ambiguous in Java. There are anonymous classes on language level and there's Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense classes. I prefer to call them VM anonymous classes to avoid confusion. I assume that whenever you use anonymous you assume "VM anonymous". > thank you for your response. While I completely understand your view > from a VM implementor's point of view, as a practicioner I would > recommend against it. Not being able to instrument lambda expressions > puts a severe limitation onto using the instrumentation API > alltogether. For example, a monitoring application that promises to > record all invocations of a method of a certain interface that only > works 95% of the time is not 5% less usefull but might no longer be > useful at all. People have build large applications based on the > assumption that all user application code can be instrumented and such > a regression would hurt them. For example, until today, over 30 people > that use my code generation framework reached out to me and reported > the reported behavior as a bug in my library and people are only > starting to migrate their applications to Java 8. The outcome is > simply not intuitive as using a lambda expression over an anonyous > inner class should not change an application's behavior. Can you elaborate on that point, please? I don't see any problems with instrumenting user code. Javac represents lambda expression body as a private static method of the enclosing class, which can be instrumented. VM anonymous classes are used only as a mechanism to glue functional interfaces and lambda expressions together at runtime. For example: static void f(Runnable r) { r.run(); } f(() -> {}); Test::f (7 bytes) @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) @ 0 Test::lambda$main$0 (1 bytes) inline (hot) Why do you need to instrument Test$$Lambda$1/... and not Test::lambda$main$0? > The currently used workaround that people use is to instrument the > LambdaMetaFactory class itself to apply the transformer manually when > it is being created. This solution is spreading as a standard approach > and I am sure that forbidding a redefinition alltogether would only > inspire to other workarrounds for being able to migrate running code > to the next Java version. It doesn't sound like a workaround, but as a solution for a different problem. If there's a need to gather profile for every lambda expression instantiation site (indy call), then redefining bootstrap method is the right way to go IMO. It allows to intercept and customize indy call site binding. What does sound as a workaround is an attempt to instrument classes produced by LambdaMetaFactory. Right now, it generates a fresh VM anonymous class on every request, but it is an implementation detail and not a requirement. For example, LambdaMetaFactory can reuse wrappers on per-functional interface basis. It would provide significant footprint savings for lambda expression-rich code bases (usually, there are much more instantiations than functional interface flavors). > Furthermore, I do not think that not being able to patch constant pool > indices in the generated code introduces any problems in practice. > Most real-world instrumentation only appends new constant pool entries > without interfering with the existant pool. Rather, I would like to > see an exception being raised if one attempts to change the original > constant pool without me being able to consider this from a technical > perspective. I think this would serve to be sufficient for most > people. If you care only about lambda expressions, then constant pool patching shouldn't be a problem (AFAIR it isn't used for lambda expressions). But in a more generic case (even for java.lang.invoke purposes), the coupling between class file and CP patches is very tight and can be easily broken with benign class file transformations. Considering ASM library, I'm not sure that even simple instrumentations preserve original constant pool structure. Probably, VM can do some structural checks on CP to forbid any modifications except appends, but there are still aliasing problems - sharing doesn't work (even for string constants), so new code shouldn't use original CP entries. I'd prefer to avoid such increase in complexity in VM implementation. > I really hope that there is a way to allow for patching anonymously > loaded classes even without exposing the constant pool patches. From VM implementation perspective I don't see any problems with allowing class file redefinition for VM anonymous classes. What I want to do is to discourage from doing so. Again - it is very fragile and there's little we can do to fix that. > So far, thank you for looking into this. I am sure this a more complex > matter than what I am able to comprehend. I appreciate that you are > taking the time to consider my opinion! Best regards, Vladimir Ivanov From jeremymanson at google.com Fri Jan 22 18:03:10 2016 From: jeremymanson at google.com (Jeremy Manson) Date: Fri, 22 Jan 2016 10:03:10 -0800 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: Couple of thoughts: > The Java Language Specification permits operations to be executed in > orders different than are apparent in program source code, subject to > constraints arising, for example, from the use of locks, volatile fields or > VarHandles. The static methods, fullFence > > , acquireFence > > , releaseFence > > , loadLoadFence > > andstoreStoreFence > , > can also be used to impose constraints. Their specifications are phrased in > terms of the lack of "reorderings" -- observable ordering effects that > might otherwise occur if the fence were not present. > There is no notion of reordering (per se) in the JLS; the fact that reorderings can occur is just implied by the way programs can be observed to behave. So, these fence operations don't map to anything non-implementation dependent. I don't think it would be impossible to fix the spec to define something that works the way you want (similar to the final field semantics), but it isn't in there already. You might want to call that out (either by saying this behavior is best effort implementation-dependent or by fixing the spec). If a VarHandle references a read-only variable (for example a final field) > then write, atomic update and numeric atomic update access modes are not > supported and corresponding methods throw UnsupportedOperationException. Are you sure you want to limit it in this way? There are an awful lot of calls to setAccessible in the world of reflection. And writing to final fields *does* map to something sensible in the spec. In fact, it would be great to have something that wasn't quite such a blunt instrument as setAccessible. getVolatile > public final Object > > getVolatile(Object > > ... args) > Returns the value of a variable, with memory semantics of reading a > volatile variable. Reading *which* volatile variable? You probably mean that all getVolatiles and setVolatiles provide semantics that behave as if the variable being written / read was declared volatile in the first place, but it is worth calling out. Nits: As is usual with virtual methods, source-level calls to access mode methods > compile to an invokevirtual instruction. More unusually, the compiler > must record the actual argument types, and may not perform method > invocation conversions on the arguments. Instead, it must push them on the > stack according to their own unconverted types. The VarHandle object itself > is pushed on the stack before the arguments. The compiler then calls the > VarHandle with a symbolic type descriptor which describes the argument and > return types. This is somewhat oddly worded. The compiler doesn't push arguments on a stack or call anything - it generates instructions that do that. The first time a invokevirtual instruction is executed it is linked, by > symbolically resolving the names in the instruction and verifying that the > method call is statically legal I keep trying not to call this out as a nit, but there should be no comma between "linked" and "by". Jeremy On Thu, Jan 21, 2016 at 2:42 PM, Paul Sandoz wrote: > Hi > > This is a request to review the VarHandles API. The code reviews and > pushes will occur separately, and flow through the hs-comp repo, most > likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just > ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was > previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: > byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to > revisit that functionality w.r.t. alignment and proposed enhancements to > ByteBuffer (in discussion on valhalla-dev). > > Paul. > > > From stuart.marks at oracle.com Fri Jan 22 18:03:57 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 22 Jan 2016 10:03:57 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: <56A133FC.4040806@oracle.com> <56A15D62.6020303@oracle.com> Message-ID: <56A26F0D.7010303@oracle.com> Hi Martin, Thanks for adding the comments and @ignore. On readObject(), ok, you went ahead and rearranged some stuff. You hit a couple of the issues I had spotted, namely the multiple assignment to elementData and the potentially confusing reuse of the name 'elementData'. The other issue was if size is less than zero. This could only occur with a corrupted or tampered serialized data stream. The old code would "successfully" deserialize a dysfunctional ArrayList instance, whereas the modified code will throw NegativeArraySizeException from readObject(). I don't know if that was intentional, but I prefer the new behavior! Strictly speaking I think throwing InvalidObjectException would preferable, but if you want to push what you have, I'm ok with it. s'marks On 1/21/16 6:58 PM, Martin Buchholz wrote: > We have a new webrev. > Bug8146568.java now uses @ignore. > readObject has a minor rewrite, only assigning to elementData once. > (Yes, we can talk more about future improvements to ArrayList) > (maybe MAX_ARRAY_SIZE could have a better name ...) > We have some hopefully clearer internal comments: > > /** > * The maximum size of array to allocate (unless necessary). > * Some VMs reserve some header words in an array. > * Attempts to allocate larger arrays may result in > * OutOfMemoryError: Requested array size exceeds VM limit > */ > private static final int MAX_ARRAY_SIZE = Integer.MAX_VALUE - 8; > > /** > * Increases the capacity to ensure that it can hold at least the > * number of elements specified by the minimum capacity argument. > * > * @param minCapacity the desired minimum capacity > * @throws OutOfMemoryError if minCapacity is less than zero > */ > private Object[] grow(int minCapacity) { > return elementData = Arrays.copyOf(elementData, > newCapacity(minCapacity)); > } > > private Object[] grow() { > return grow(size + 1); > } > > /** > * Returns a capacity at least as large as the given minimum capacity. > * Returns the current capacity increased by 50% if that suffices. > * Will not return a capacity greater than MAX_ARRAY_SIZE unless > * the given minimum capacity is greater than MAX_ARRAY_SIZE. > * > * @param minCapacity the desired minimum capacity > * @throws OutOfMemoryError if minCapacity is less than zero > */ > private int newCapacity(int minCapacity) { > > On Thu, Jan 21, 2016 at 2:36 PM, Stuart Marks wrote: >> >> >> On 1/21/16 1:57 PM, Martin Buchholz wrote: >>>> >>>> One is that in list.addAll(other), the sizes of list and other exceeds >>>> Integer.MAX_VALUE, then grow(int) will be called with a negative value >>>> for >>>> minCapacity. The code *seems* to handle this ok, but the negative >>>> minCapacity value can get pretty deeply into the helper methods before >>>> being >>>> caught. Would it be better to check it at the top of grow(int) and throw >>>> an >>>> exception there? (Probably OOME.) I think it would make the subsequent >>>> code >>>> easier to follow. >>> >>> >>> It's true that the code is rather tricky, "overflow-conscious code". >>> But this is ArrayList, so it seems worth optimizing even for grow. >>> >>> The common case is that we grow by 50% and then if (newCapacity - >>> MAX_ARRAY_SIZE <= 0) we can be sure that newCapacity is not negative. >> >> >> The code may be correct, but I'm concerned about maintenance. If things >> shift around, it might be easy to miss the possibility that negative >> minCapacity could be passed to grow() if overflow had occurred. So perhaps >> at least a comment would be warranted. >> >>>> It looks like there are a variety of ways for minCapacity that is >>>> positive >>>> but greater than MAX_ARRAY_SIZE to reach newCapacity(). If this occurs, >>>> and >>>> other conditions are right, it looks like the code will end up attempting >>>> to >>>> allocate an array greater than MAX_ARRAY_SIZE. >>> >>> >>> If grow(n) is called with MAX_ARRAY_SIZE < n <= MAX_VALUE, then we no >>> choice but to allocate an array of that size! It's only when we use >>> the grow-by-50% strategy that we can change our minds by scaling back. >>> I don't see a bug. >> >> >> Ah, MAX_ARRAY_SIZE applies only to grow-by-50%, not to all array >> allocations. Perhaps it was my mistake for having believed its comment, >> which is >> >> The maximum size of array to allocate. >> >> You know what they say about comments not matching the code.... :-) >> >> I do think this comment needs to be adjusted to say that MAX_ARRAY_SIZE >> applies only to the 50% growth policy. I was certainly misled by it. >> >>>> One style point I noticed (which might be an issue of me not being used >>>> to >>>> it) is the use of an elementData local variable to shadow the elementData >>>> field. This is more-or-less ok in places where elementData is initialized >>>> from this.elementData, but in readObject(), the local elementData is >>>> introduced in a nested scope. This means that elementData has different >>>> meanings in different parts of the method. >>> >>> >>> Yeah, elementData is not great but I couldn't find anything better. >>> "a" is already taken. "snapshot" has the wrong connotations. If you >>> prefer e.g. "elements" I will change it throughout, but in either case >>> a reader needs to understand that "elements" and "elementData" are >>> "almost" the same. >> >> >> I don't think a global change is necessary, as the prevailing style in this >> file is to use the elementData field or to have a local elementData that's >> an alias of the field. I think readObject() is the outlier for using both >> the field and the local variable. But there are several other funny things >> going on here in readObject()... well I won't insist that you address them >> right now, as they're a distraction from this bugfix. So the change as >> you've proposed is fine. >> >> (But let me know if you're interested in discussing readObject() further.) >> >>>> For the test Bug8146568 I think the preferred way to disable a test with >>>> extreme memory requirements is to use @ignore; see >>> >>> >>> I've never liked @ignore in practice, because jtreg is very noisy >>> unless you also say >>> -ignore:quiet >>> (which I always do, but does everyone else?) >> >> >> Yes, I think jtreg's default behavior has taught everyone, including our >> automated systems, to use -ignore:quiet. So I think @ignore is fine. >> >> Thanks. >> >> s'marks From gil at azul.com Fri Jan 22 18:12:45 2016 From: gil at azul.com (Gil Tene) Date: Fri, 22 Jan 2016 18:12:45 +0000 Subject: Ephemerons In-Reply-To: <569BDAF2.2080404@gmail.com> References: <569BDAF2.2080404@gmail.com> Message-ID: <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> Peter, I've been following Ephemerons in other GC'ed environments, and wondering when someone will bring it up for Java. Happy to see attentions given to it. The "conditional weak reference hashmap/table/dictionary" use case seems to be a common primary motivator, and it's a very valid one. Given that we currently do completely concurrent ref processing (in Zing at least) for all reference types, adding Ephemerons to the spec'ed behavior will add some interesting challenges for keeping things concurrent, but I don't think that it is fundamentally undoable (just "hard" and interesting to fully work out). Scanning through the proposal and Java class (mostly JavaDoc spec), I have the following question: Do we really need a separate "ephemerally reachable" strength below week? The was you extended the definition to the weak definition to say "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by traversing a weak reference or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." would [naively] seem to be sufficient to add and fully describe the desired Ephemeron behavior from a reference strength perspective. Specifically, would modifying your implementation such that Ephemeron extended WeakReference (instead of Reference), and it's V value was tracked with "private WeakReference valueRef" (instead of "private V value") [along with the semi-obvious internal changes that would result] provide what is needed to support proper Ephemeron behavior we just added the "?or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." to the spec'ed definition of "weakly reachable" (and modified the associated JVM ref processing accordingly, which you already have to do to support your wider definitions anyway)? ? Gil. > On Jan 17, 2016, at 10:18 AM, Peter Levart wrote: > > Hi, > > Ephemerons are special kind of references described by Barry Hayes [1]. In the simple variant, they contain two "referents". One is called the key, which can be viewed as a "weak" referent in the style of Java reference types (SoftReference, WeakReference, PhantomReference), the other is called the value, which is also treated specially by GC. It's reachability is dependent on the reachability of the key. > > Ephemerons solve the problem seen for example in the java.util.WeakHashMap when a value in this map directly or indirectly refers to it's key. Such entry will never be purged as the value is always reachable from the WeakHashMap instance and through the value, it's key is reachable too. There are other places where Ephemerons could be used - for example in ClassValue API. Try googling for "java ephemeron" (without quotes) to find out other situations where Ephemerons would be beneficial. > > If one was to implement Ephemerons in Java, the main question he would be asking is how Ephemeron(s) were supposed to interact with existing Java reference types. Hayes only defines their behavior in isolation, but Java already has 4 "weak" reference types with different "strengths": SoftReference, WeakReference, FinalReference (for implementing finalization) and PhantomReference. In the prototype I choose to define Ephemerons as a new reference type with it's own "strength" for the key referent positioned between WeakReference and FinalReference. It would be possible to merge it with an existing reference type like WeakReference or position it between SoftReference and WeakReference or even entirely "on the top" of the strength list, but I think it would not be wise to position it below FinalReference or even PhantomReference. What's best is an open question. What is also not so obvious is how to define the reachability of the Ephemeron's value and it's interaction with existing reference types. I think I defined it soundly (see the reachability types in the javadoc of [4]). The reason for defining the reachability of the value to alternate between ephemeraly-reachable and weakly-reachable and not between ephemeraly-reachable and strongly-reachable, while theoretically possible, is the desire to have a separate phase of processing for each reachability strength, like it is done currently, and which doesn't affect the performance of processing of existing reference types. > > Having skimmed through the reference processing code in the VM for a couple of times, I thought it should not be too complicated to implement another type of "weak" reference. Encouraged by how little changes were needed to remove the sun.misc.Cleaner special type of reference [2], I began experimenting and came up with a prototype that seems to work [3]. Luckily most of the logic to process Reference objects in VM is encapsulated in the ReferenceProcessor class and this logic is invoked from various GC implementations of the hotspot. Changes needed are therefore mostly localized to this class. The webrev also contains recent changes from JDK-8143847 [2] that have not yet propagated to the jdk9-dev repo. When they do, I will prepare a rebased webrev without those changes. > > I'm publishing this prototype here to get the answer to the main question: Is there an interest to have Ephemeron reference type in Java? > > Given that there was interest, I would also like to initiate a discussion about: > - What would be the most appropriate "strength" of reachabillity for Epehmeron referents (key and value) and the desired interactions with existing reachabilities. > - The prototype itself. Since I'm new to this part of code (that's my 1st not so shallow dive into the VM), I surely must have missed places that should be changed. Although the prototype seems to work, I have not created extensive tests for the functionality and have not exposed it to all the various GC algorithms and situations yet. I could use some advise on how to exercise all GC algorithm combinations possible in the VM (what flags to use, for example). > > Regards, Peter > > [1] https://static.aminer.org/pdf/PDF/000/522/273/ephemerons_a_new_finalization_mechanism.pdf > [2] https://bugs.openjdk.java.net/browse/JDK-8143847 > [3] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/ > [4] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.01/src/java.base/share/classes/java/lang/ref/Ephemeron.java.html > > P.S. > > To wet some appetite, with the above prototype (applied to current jdk9-dev), it is possible to run the following example: > > import java.lang.ref.Ephemeron; > import java.util.ArrayList; > import java.util.List; > > public class EphemeronTest { > > static class Key { > final int i; > > Key(int i) { > this.i = i; > } > > @Override > public String toString() { > return "k" + i; > } > } > > static class Value { > final Key key; > > Value(Key key) { > this.key = key; > } > > @Override > public String toString() { > return "v(" + key + ")"; > } > } > > static class Eph extends Ephemeron { > public Eph(Key key, Value value) { > super(key, value); > } > > @Override > public String toString() { > return getKey() + "=" + getValue(); > } > } > > static void test(int size, boolean forwardChaining) throws Exception { > System.out.println(); > System.out.println((forwardChaining ? "Forward" : "Backward") + > " chaining of value->key ..."); > System.out.println(); > List ephs = new ArrayList<>(size); > > Key k1 = new Key(1); > { > Key kp = k1; > for (int i = 2; i <= size; i++) { > Key ki = new Key(i); > ephs.add( > forwardChaining > ? new Eph(kp, new Value(ki)) > : new Eph(ki, new Value(kp)) > ); > kp = ki; > } > ephs.add( > forwardChaining > ? new Eph(kp, new Value(k1)) > : new Eph(k1, new Value(kp)) > ); > kp = null; > } > > System.gc(); > System.out.println("1: " + ephs); > k1.getClass(); // reachabilityFence > > k1 = null; > System.gc(); > System.out.println("2: " + ephs); > } > > public static void main(String[] args) throws Exception { > > int size = (args.length < 1) > ? 5 > : Math.max(2, Integer.parseInt(args[0])); > > test(size, true); > test(size, false); > } > } > > > Which prints: > > Forward chaining of value->key ... > > 1: [k1=v(k2), k2=v(k3), k3=v(k4), k4=v(k5), k5=v(k1)] > 2: [null=null, null=null, null=null, null=null, null=null] > > Backward chaining of value->key ... > > 1: [k2=v(k1), k3=v(k2), k4=v(k3), k5=v(k4), k1=v(k5)] > 2: [null=null, null=null, null=null, null=null, null=null] > > > If you compile the JDK with --enable-debug and use the following VM switches when running: -XX:+PrintGCDetails -XX:+TraceReferenceGC, you can also observe the inner workings of the reference processing, including Ephemerons. > From stuart.marks at oracle.com Fri Jan 22 18:43:54 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 22 Jan 2016 10:43:54 -0800 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A24E50.6050705@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> Message-ID: <56A2786A.5090507@oracle.com> On 1/22/16 7:44 AM, Claes Redestad wrote: >>> It would seem the assessment that serialVersionUID is not needed when >>> using the serial proxy pattern might be misguided (or is this a bug in >>> ObjectInputStream)? >> >> It is debate-able whether this is a test bug or not. But I think >> we should not change the serialVersionUID for this class, whether >> truly Serializable or not. Even though in practice it is not all >> that useful. >> >> Can you please revert the serialization parts of the change, and >> rerun the test? To see if still passes or fails. > > I backed out the serial version UID change and reran the test, same thing. > > I dug deeper, as this is a bit vexxing... > > While a static field does not in and of itself cause the generated UID to change, > the private ZERO_LENGTH_ENUM_ARRAY is being accessed from SerializationProxy > class, thus javac generates a package-private synthetic method (access$000) which > will be accounted for when generating serialVersionUID. That's unfortunate. > > I ran a few experiments to confirm this: > > - Adding a package-private static method to EnumSet makes the test fail > - Adding any static field and the test keep passing > > So it seems our options here are limited. I think this is a bug in the test (BogusEnumSet.java). EnumSet, a class in the runtime, should never appear in a serialized stream, thus it shouldn't have to declare a serialVersionUID in order to remain compatible with anything. The test deliberately attempts to create a serialized stream that should never occur in actual use, so the test should be responsible for constructing that stream properly. Unfortunately, the test doesn't actually construct that stream; instead it's a bunch of hex constants. It's tedious but not difficult to patch the constants with the modified value. I've appended a patch below that does this, along with removing EnumSet's serialVersionUID and restoring the @SuppressWarnings. I can post a webrev if this doesn't come through properly. s'marks diff -r c421a3928efc -r 6de800698cf7 src/java.base/share/classes/java/util/EnumMap.java --- a/src/java.base/share/classes/java/util/EnumMap.java Mon Jan 11 10:13:29 2016 -0800 +++ b/src/java.base/share/classes/java/util/EnumMap.java Fri Jan 22 10:42:24 2016 -0800 @@ -25,7 +25,6 @@ package java.util; -import java.util.Map.Entry; import jdk.internal.misc.SharedSecrets; /** @@ -125,8 +124,6 @@ return (V)(value == NULL ? null : value); } - private static final Enum[] ZERO_LENGTH_ENUM_ARRAY = new Enum[0]; - /** * Creates an empty enum map with the specified key type. * diff -r c421a3928efc -r 6de800698cf7 src/java.base/share/classes/java/util/EnumSet.java --- a/src/java.base/share/classes/java/util/EnumSet.java Mon Jan 11 10:13:29 2016 -0800 +++ b/src/java.base/share/classes/java/util/EnumSet.java Fri Jan 22 10:42:24 2016 -0800 @@ -92,8 +92,6 @@ */ final Enum[] universe; - private static Enum[] ZERO_LENGTH_ENUM_ARRAY = new Enum[0]; - EnumSet(ClasselementType, Enum[] universe) { this.elementType = elementType; this.universe = universe; @@ -421,6 +419,9 @@ private static class SerializationProxy > implements java.io.Serializable { + + private static final Enum[] ZERO_LENGTH_ENUM_ARRAY = new Enum[0]; + /** * The element type of this enum set. * diff -r c421a3928efc -r 6de800698cf7 test/java/util/EnumMap/EnumMapBash.java --- a/test/java/util/EnumMap/EnumMapBash.java Mon Jan 11 10:13:29 2016 -0800 +++ b/test/java/util/EnumMap/EnumMapBash.java Fri Jan 22 10:42:24 2016 -0800 @@ -48,8 +48,6 @@ bash(Silly500.class); } - private static Enum[] ZERO_LENGTH_ENUM_ARRAY = new Enum[0]; - static > void bash(Class enumClass) { Enum[] universe = enumClass.getEnumConstants(); diff -r c421a3928efc -r 6de800698cf7 test/java/util/EnumSet/BogusEnumSet.java --- a/test/java/util/EnumSet/BogusEnumSet.java Mon Jan 11 10:13:29 2016 -0800 +++ b/test/java/util/EnumSet/BogusEnumSet.java Fri Jan 22 10:42:24 2016 -0800 @@ -32,6 +32,11 @@ public class BogusEnumSet { public static void main(String[] args) throws Throwable { + // This test depends on the current serialVersionUID of EnumSet, + // which may change if the EnumSet class is modified. + // The current value is 4168005130090799668L = 0x39D7BA9531116234L + // If the value changes, it will have to be patched into the + // serialized byte stream below at the location noted. byte[] serializedForm = { (byte)0xac, (byte)0xed, 0x0, 0x5, 0x73, 0x72, 0x0, 0x18, 0x6a, 0x61, 0x76, 0x61, 0x2e, 0x75, 0x74, 0x69, @@ -40,9 +45,10 @@ 0x7e, (byte)0xb0, (byte)0xd0, 0x7e, 0x2, 0x0, 0x1, 0x4a, 0x0, 0x8, 0x65, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x73, 0x78, 0x72, 0x0, 0x11, 0x6a, 0x61, 0x76, 0x61, 0x2e, 0x75, 0x74, 0x69, - 0x6c, 0x2e, 0x45, 0x6e, 0x75, 0x6d, 0x53, 0x65, 0x74, 0xe, - 0x3, 0x21, 0x6a, (byte)0xcd, (byte)0x8c, 0x29, (byte)0xdd, 0x2, - 0x0, 0x2, 0x4c, 0x0, 0xb, 0x65, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, + 0x6c, 0x2e, 0x45, 0x6e, 0x75, 0x6d, 0x53, 0x65, 0x74, + // EnumSet's serialVersionUID is the following eight bytes (big-endian) + 0x39, (byte)0xd7, (byte)0xba, (byte)0x95, 0x31, 0x11, 0x62, 0x34, + 0x2, 0x0, 0x2, 0x4c, 0x0, 0xb, 0x65, 0x6c, 0x65, 0x6d, 0x65, 0x6e, 0x74, 0x54, 0x79, 0x70, 0x65, 0x74, 0x0, 0x11, 0x4c, 0x6a, 0x61, 0x76, 0x61, 0x2f, 0x6c, 0x61, 0x6e, 0x67, 0x2f, 0x43, 0x6c, 0x61, 0x73, 0x73, 0x3b, 0x5b, 0x0, 0x8, 0x75, 0x6e, 0x69, 0x76, 0x65, 0x72, From martinrb at google.com Fri Jan 22 20:02:44 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 22 Jan 2016 12:02:44 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: <56A26F0D.7010303@oracle.com> References: <56A133FC.4040806@oracle.com> <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> Message-ID: On Fri, Jan 22, 2016 at 10:03 AM, Stuart Marks wrote: > On readObject(), ok, you went ahead and rearranged some stuff. You hit a > couple of the issues I had spotted, namely the multiple assignment to > elementData and the potentially confusing reuse of the name 'elementData'. > > The other issue was if size is less than zero. This could only occur with a > corrupted or tampered serialized data stream. The old code would > "successfully" deserialize a dysfunctional ArrayList instance, whereas the > modified code will throw NegativeArraySizeException from readObject(). > > I don't know if that was intentional, but I prefer the new behavior! > > Strictly speaking I think throwing InvalidObjectException would preferable, > but if you want to push what you have, I'm ok with it. I went "by the book" as you suggested and now throw InvalidObjectException when size < 0. (But I've been saying for a decade: if we're serious about Serialization, it needs to be someone's full time job) I'll commit tomorrow if I don't hear otherwise. From brian.burkhalter at oracle.com Fri Jan 22 20:17:31 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 22 Jan 2016 12:17:31 -0800 Subject: JDK RFR of 8147545: Remove sun.misc.ManagedLocalsThread from java.prefs Message-ID: Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8147545 Patch: http://cr.openjdk.java.net/~bpb/8147545/webrev.00/ Replaced using of ManagedLocalsThread with a Thread created from the new constructor Thread(ThreadGroup,Runnable,String,long,boolean) with the boolean parameter ?inheritThreadLocals? set to false. Thanks, Brian From chris.hegarty at oracle.com Fri Jan 22 20:19:37 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 22 Jan 2016 20:19:37 +0000 Subject: JDK RFR of 8147545: Remove sun.misc.ManagedLocalsThread from java.prefs In-Reply-To: References: Message-ID: <92501848-78D8-4B9E-B37E-BBE73696291B@oracle.com> On 22 Jan 2016, at 20:17, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8147545 > Patch: http://cr.openjdk.java.net/~bpb/8147545/webrev.00/ Looks good to me. Thanks Brian. -Chris. > Replaced using of ManagedLocalsThread with a Thread created from the new constructor > > Thread(ThreadGroup,Runnable,String,long,boolean) > > with the boolean parameter ?inheritThreadLocals? set to false. > > Thanks, > > Brian From yumin.qi at oracle.com Fri Jan 22 20:22:06 2016 From: yumin.qi at oracle.com (Yumin Qi) Date: Fri, 22 Jan 2016 12:22:06 -0800 Subject: RFR: 8147755: ASM should create correct constant tag for invokestatic on handle point to interface static method Message-ID: <56A28F6E.30307@oracle.com> Hi, All Can I have a review for bug: https://bugs.openjdk.java.net/browse/JDK-8147755 Webrev: http://cr.openjdk.java.net/~minqi/8147755/webrev-01/ Summary: When fix bug 8145148 to follow: JVMS-5.4.3.3 Method Resolution: " If C is an interface, method resolution throws an IncompatibleClassChangeError." JVMS-5.4.3.4 Interface Method Resolution: "If C is not an interface, interface method resolution throws an IncompatibleClassChangeError" In defmeth(default method) tests, a MethodHandle pointing to interface static method will violate the spec. Since such a handle in asm will generate a tag of Methodref in constantpool for the call site, with the fix of 8145148, it requires a tag of InterfaceMethodref. Fix by create a new constructor of Handle with extra boolean to indicate if the handle points to interface static method. Tests: A test case and resulted constantpool data attached. The fix will not affect existing code. Also tested with fixed 8145148 and revised defmeth with the new version Handle and passed. Thanks Yumin -------------- next part -------------- import jdk.internal.org.objectweb.asm.*; import java.io.FileOutputStream; import java.lang.reflect.InvocationTargetException; import java.lang.invoke.MethodHandle; import java.lang.invoke.MethodHandles; import java.lang.invoke.MethodType; import static jdk.internal.org.objectweb.asm.Opcodes.*; public class TestStaticInterfaceHandle { // interface I { // static void m() { // System.out.println("Hello from interface I, static m()!"); // } // } final static int version = 52; static byte[] dumpI() { ClassWriter cw = new ClassWriter(0); cw.visit(version, ACC_PUBLIC | ACC_ABSTRACT | ACC_INTERFACE, "I", null, "java/lang/Object", null); { MethodVisitor mv = cw.visitMethod(ACC_PUBLIC | ACC_STATIC, "m", "()V", null, null); mv.visitCode(); mv.visitFieldInsn(GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;"); mv.visitLdcInsn("Hello from interface I, m()!"); mv.visitMethodInsn(INVOKEVIRTUAL, "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false); mv.visitInsn(RETURN); mv.visitMaxs(3, 1); mv.visitEnd(); } cw.visitEnd(); return cw.toByteArray(); } static byte[] dumpTestIm() { ClassWriter cw = new ClassWriter(0); cw.visit(version, ACC_PUBLIC | ACC_SUPER, "TestIm", null, "java/lang/Object", null); Handle handle = new Handle(Opcodes.H_INVOKESTATIC, "I", "m", "()V", true/* interface static version*/); { MethodVisitor mv = cw.visitMethod(ACC_PUBLIC, "", "()V", null, null); mv.visitCode(); mv.visitVarInsn(ALOAD, 0); mv.visitMethodInsn(INVOKESPECIAL, "java/lang/Object", "", "()V", false); mv.visitInsn(RETURN); mv.visitMaxs(1, 1); mv.visitEnd(); } { MethodVisitor mv = cw.visitMethod(ACC_PUBLIC | ACC_STATIC, "staticCallIm", "()V", null, null); mv.visitCode(); mv.visitFieldInsn(GETSTATIC, "java/lang/System", "out", "Ljava/io/PrintStream;"); mv.visitLdcInsn("Calling static I.m():"); mv.visitMethodInsn(INVOKEVIRTUAL, "java/io/PrintStream", "println", "(Ljava/lang/String;)V", false/*intf*/); mv.visitLdcInsn(handle); mv.visitMethodInsn(INVOKEVIRTUAL, "java/lang/invoke/MethodHandle", "invoke", "()V", false); mv.visitInsn(RETURN); mv.visitMaxs(3, 1); mv.visitEnd(); } cw.visitEnd(); return cw.toByteArray(); } static class CL extends ClassLoader { @Override protected Class findClass(String name) throws ClassNotFoundException { byte[] classBytes = null; switch (name) { case "TestIm" : classBytes = dumpTestIm(); break; case "I" : classBytes = dumpI(); break; default : throw new ClassNotFoundException(name); } return defineClass(name, classBytes, 0, classBytes.length); } } public static void main(String[] args) throws Throwable { try (FileOutputStream fos = new FileOutputStream("I.class")) { fos.write(dumpI()); } try (FileOutputStream fos = new FileOutputStream("TestIm.class")) { fos.write(dumpTestIm()); } String cName = "TestIm"; String mName = "staticCallIm"; Class cls = (new CL()).loadClass(cName); System.out.println("Test " + cName + "." + mName + ":"); try { cls.getMethod(mName).invoke(cls.newInstance()); } catch (Throwable e) { System.out.println("FAILED"); throw e; } } } Classfile /scratch/yqi/ws/test/java/8147419/TestIm.class Last modified Jan 20, 2016; size 413 bytes MD5 checksum cafde171a1e44138b00e389d8bb944a1 public class TestIm minor version: 0 major version: 52 flags: ACC_PUBLIC, ACC_SUPER Constant pool: #1 = Utf8 TestIm #2 = Class #1 // TestIm #3 = Utf8 java/lang/Object #4 = Class #3 // java/lang/Object #5 = Utf8 #6 = Utf8 ()V #7 = NameAndType #5:#6 // "":()V #8 = Methodref #4.#7 // java/lang/Object."":()V #9 = Utf8 staticCallIm #10 = Utf8 java/lang/System #11 = Class #10 // java/lang/System #12 = Utf8 out #13 = Utf8 Ljava/io/PrintStream; #14 = NameAndType #12:#13 // out:Ljava/io/PrintStream; #15 = Fieldref #11.#14 // java/lang/System.out:Ljava/io/PrintStream; #16 = Utf8 Calling static I.m(): #17 = String #16 // Calling static I.m(): #18 = Utf8 java/io/PrintStream #19 = Class #18 // java/io/PrintStream #20 = Utf8 println #21 = Utf8 (Ljava/lang/String;)V #22 = NameAndType #20:#21 // println:(Ljava/lang/String;)V #23 = Methodref #19.#22 // java/io/PrintStream.println:(Ljava/lang/String;)V #24 = Utf8 I #25 = Class #24 // I #26 = Utf8 m #27 = NameAndType #26:#6 // m:()V #28 = InterfaceMethodref #25.#27 // I.m:()V #29 = MethodHandle #6:#28 // invokestatic I.m:()V #30 = Utf8 java/lang/invoke/MethodHandle #31 = Class #30 // java/lang/invoke/MethodHandle #32 = Utf8 invoke #33 = NameAndType #32:#6 // invoke:()V #34 = Methodref #31.#33 // java/lang/invoke/MethodHandle.invoke:()V #35 = Utf8 Code { public TestIm(); descriptor: ()V flags: ACC_PUBLIC Code: stack=1, locals=1, args_size=1 0: aload_0 1: invokespecial #8 // Method java/lang/Object."":()V 4: return public static void staticCallIm(); descriptor: ()V flags: ACC_PUBLIC, ACC_STATIC Code: stack=3, locals=1, args_size=0 0: getstatic #15 // Field java/lang/System.out:Ljava/io/PrintStream; 3: ldc #17 // String Calling static I.m(): 5: invokevirtual #23 // Method java/io/PrintStream.println:(Ljava/lang/String;)V 8: ldc #29 // MethodHandle invokestatic I.m:()V 10: invokevirtual #34 // Method java/lang/invoke/MethodHandle.invoke:()V 13: return } From claes.redestad at oracle.com Fri Jan 22 20:30:51 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 22 Jan 2016 21:30:51 +0100 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A2786A.5090507@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> <56A2786A.5090507@oracle.com> Message-ID: <56A2917B.6040205@oracle.com> On 2016-01-22 19:43, Stuart Marks wrote: > > > On 1/22/16 7:44 AM, Claes Redestad wrote: >>>> It would seem the assessment that serialVersionUID is not needed when >>>> using the serial proxy pattern might be misguided (or is this a bug in >>>> ObjectInputStream)? >>> >>> It is debate-able whether this is a test bug or not. But I think >>> we should not change the serialVersionUID for this class, whether >>> truly Serializable or not. Even though in practice it is not all >>> that useful. >>> >>> Can you please revert the serialization parts of the change, and >>> rerun the test? To see if still passes or fails. >> >> I backed out the serial version UID change and reran the test, same >> thing. >> >> I dug deeper, as this is a bit vexxing... >> >> While a static field does not in and of itself cause the generated >> UID to change, >> the private ZERO_LENGTH_ENUM_ARRAY is being accessed from >> SerializationProxy >> class, thus javac generates a package-private synthetic method >> (access$000) which >> will be accounted for when generating serialVersionUID. That's >> unfortunate. >> >> I ran a few experiments to confirm this: >> >> - Adding a package-private static method to EnumSet makes the test fail >> - Adding any static field and the test keep passing >> >> So it seems our options here are limited. > > I think this is a bug in the test (BogusEnumSet.java). > > EnumSet, a class in the runtime, should never appear in a serialized > stream, thus it shouldn't have to declare a serialVersionUID in order > to remain compatible with anything. > > The test deliberately attempts to create a serialized stream that > should never occur in actual use, so the test should be responsible > for constructing that stream properly. Unfortunately, the test doesn't > actually construct that stream; instead it's a bunch of hex constants. > > It's tedious but not difficult to patch the constants with the > modified value. I've appended a patch below that does this, along with > removing EnumSet's serialVersionUID and restoring the @SuppressWarnings. > > I can post a webrev if this doesn't come through properly. > > s'marks > Thanks Stuart! I've posted a webrev including your changes here: http://cr.openjdk.java.net/~redestad/8148044/webrev.03/ If there's a preference for this version I'll push it after the weekend. Thanks! /Claes From ecki at zusammenkunft.net Fri Jan 22 20:53:10 2016 From: ecki at zusammenkunft.net (ecki at zusammenkunft.net) Date: Fri, 22 Jan 2016 21:53:10 +0100 Subject: JDK RFR of 8147545: Remove sun.misc.ManagedLocalsThread from java.prefs In-Reply-To: <92501848-78D8-4B9E-B37E-BBE73696291B@oracle.com> References: <92501848-78D8-4B9E-B37E-BBE73696291B@oracle.com> Message-ID: Hello, I would name it more like "Preference Timer Shutdown Hook" and "Preference Dispatch Thread" and "Preference Flush Thread" (i.e. include the subsystem) + new Thread(null, null, "Sync Timer Thread", 0, false) Gruss Bernd -- http://bernd.eckenfels.net -----Original Message----- From: Chris Hegarty To: Brian Burkhalter Cc: core-libs-dev Sent: Fr., 22 Jan. 2016 21:38 Subject: Re: JDK RFR of 8147545: Remove sun.misc.ManagedLocalsThread from java.prefs On 22 Jan 2016, at 20:17, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8147545 > Patch: http://cr.openjdk.java.net/~bpb/8147545/webrev.00/ Looks good to me. Thanks Brian. -Chris. > Replaced using of ManagedLocalsThread with a Thread created from the new constructor > > Thread(ThreadGroup,Runnable,String,long,boolean) > > with the boolean parameter ?inheritThreadLocals? set to false. > > Thanks, > > Brian From brian.burkhalter at oracle.com Fri Jan 22 20:56:28 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 22 Jan 2016 12:56:28 -0800 Subject: JDK RFR of 8147545: Remove sun.misc.ManagedLocalsThread from java.prefs In-Reply-To: References: <92501848-78D8-4B9E-B37E-BBE73696291B@oracle.com> Message-ID: <5E1BDF8E-7D72-4FB8-BD78-6087B85F0AE6@oracle.com> Good suggestions but I already pushed it as-is. I can file a new issue for this ? Thanks, Brian On Jan 22, 2016, at 12:53 PM, ecki at zusammenkunft.net wrote: > I would name it more like "Preference Timer Shutdown Hook" and "Preference Dispatch Thread" and "Preference Flush Thread" (i.e. include the subsystem) > > + new Thread(null, null, "Sync Timer Thread", 0, false) From stuart.marks at oracle.com Fri Jan 22 21:37:22 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 22 Jan 2016 13:37:22 -0800 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A2917B.6040205@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> <56A2786A.5090507@oracle.com> <56A2917B.6040205@oracle.com> Message-ID: <56A2A112.9010200@oracle.com> On 1/22/16 12:30 PM, Claes Redestad wrote: > Thanks Stuart! > > I've posted a webrev including your changes here: > http://cr.openjdk.java.net/~redestad/8148044/webrev.03/ > > If there's a preference for this version I'll push it after the weekend. This looks good to me, thanks for updating it. I definitely prefer this version of the EnumSet and BogusEnumSet test changes. Perhaps ideally BogusEnumSet would generate its serialized stream in order to avoid this problem if EnumSet were to change again, but at least now finding and patching the right bytes manually isn't too difficult. s'marks From stuart.marks at oracle.com Fri Jan 22 22:30:53 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Fri, 22 Jan 2016 14:30:53 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: <56A133FC.4040806@oracle.com> <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> Message-ID: <56A2AD9D.70802@oracle.com> On 1/22/16 12:02 PM, Martin Buchholz wrote: > I went "by the book" as you suggested and now throw > InvalidObjectException when size < 0. > (But I've been saying for a decade: if we're serious about > Serialization, it needs to be someone's full time job) "Admiral, if we go by the book, years could turn into decades." Thanks for the update; this looks great to me. s'marks From peter.levart at gmail.com Fri Jan 22 22:49:16 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 22 Jan 2016 23:49:16 +0100 Subject: Ephemerons In-Reply-To: <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> Message-ID: <56A2B1EC.6050907@gmail.com> Hi Gil, Thanks for taking a look at the Ephemerons for Java. It's great to have a big mind joining the discussion. On 01/22/2016 07:12 PM, Gil Tene wrote: > Peter, > > I've been following Ephemerons in other GC'ed environments, and wondering when someone will bring it up for Java. Happy to see attentions given to it. The "conditional weak reference hashmap/table/dictionary" use case seems to be a common primary motivator, and it's a very valid one. Given that we currently do completely concurrent ref processing (in Zing at least) for all reference types, adding Ephemerons to the spec'ed behavior will add some interesting challenges for keeping things concurrent, but I don't think that it is fundamentally undoable (just "hard" and interesting to fully work out). The algorithm for processing ephemerons is not much different from that for processing normal references. It's just that it is iterative until it converges to a stable state. If you allow mutator threads to at least in some phases execute concurrently with normal reference processing then I suspect it should be possible to do that for ephemerons too, but I don't have a clue what tricks you perform to be able to allow mutator threads to be concurrent with reference processing. > Scanning through the proposal and Java class (mostly JavaDoc spec), I have the following question: > > Do we really need a separate "ephemerally reachable" strength below week? The was you extended the definition to the weak definition to say "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by traversing a weak reference or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." would [naively] seem to be sufficient to add and fully describe the desired Ephemeron behavior from a reference strength perspective. Ephemeron always touches definitions of at least two consecutive strengths of reachabilities. The prototype says: *
  • An object is weakly reachable if it is neither * strongly nor softly reachable but can be reached by traversing a * weak reference or by traversing an ephemeron through it's value while * the ephemeron's key is at least weakly reachable. *
  • An object is ephemerally reachable if it is neither * strongly, softly nor weakly reachable but can be reached by traversing an * ephemeron through it's key or by traversing an ephemeron through it's value * while it's key is at most ephemerally reachable. When the ephemerons that * refer to ephemerally reachable key object are cleared, the key object becomes * eligible for finalization. But Ephemeron does not need a special reachability strength. It could be merged with WeakReference as far as the reachability of it's key is concerned. In that case it would touch at least the definitions of softly-reachable and weakly-reachable: *
  • An object is softly reachable if it is not strongly * reachable but can be reached by traversing a soft reference or * by traversing an ephemeron through it's value while * the ephemeron's key is at least softly reachable. *
  • An object is weakly reachable if it is neither * strongly nor softly reachable but can be reached by traversing a * weak reference (including ephemeron through it's key) or by traversing * an ephemeron through it's value while the ephemeron's key is at most * weakly reachable. When the weak references to a weakly-reachable * object are cleared (which includes ephemerons to a weakly-reachable key), * the object becomes eligible for finalization. Presently the hotspot handles different strengths of references by maintaining a distinct set of discovered references for each type and then processing these sets in sequence from the strongest to the weakest. If Ephemerons and WeakReferences were kept in the same discovered set, they would present the same reachability strength for their referent (called also the key in case of ephemeron). > Specifically, would modifying your implementation such that Ephemeron extended WeakReference (instead of Reference), and it's V value was tracked with "private WeakReference valueRef" (instead of "private V value") [along with the semi-obvious internal changes that would result] provide what is needed to support proper Ephemeron behavior we just added the "?or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." to the spec'ed definition of "weakly reachable" (and modified the associated JVM ref processing accordingly, which you already have to do to support your wider definitions anyway)? In case the strength of ephemeron's key was the same as that of WeakReference's referent, it would make sense for Ephemeron to extend WeakReference, since it would then just be a special kind of WeakReference. But I don't think that changing "private V value" into "private WeakReference valueRef" would buy anything in terms of processing algorithm simplicity. At least in hotspot it would increase the code complexity, since marking the ephemeron's value 'alive' as a consequence of finding it's key marked alive during iteration over the list of ephemerons and removing the current ephemeron from the pending list, would also require removing of the valuRef WeakReference from the pending list at the same time but that would require finding it in the list 1st, since it's a single-linked-list or removing it at next iteration which would increase the number of iterations through the pending set before the state converges... There's no problem in making the VM to treat the "private V value" specially. In HotSpot it is only required to update the OopMap for the Ephemeron class to exclude the value field - analogue to how this is done for Reference fields. There's no need for indirection through another WeakReference to get special treatment for value (exclusion from traversals). Regards, Peter ? Gil. >> On Jan 17, 2016, at 10:18 AM, Peter Levart wrote: >> >> Hi, >> >> Ephemerons are special kind of references described by Barry Hayes [1]. In the simple variant, they contain two "referents". One is called the key, which can be viewed as a "weak" referent in the style of Java reference types (SoftReference, WeakReference, PhantomReference), the other is called the value, which is also treated specially by GC. It's reachability is dependent on the reachability of the key. >> >> Ephemerons solve the problem seen for example in the java.util.WeakHashMap when a value in this map directly or indirectly refers to it's key. Such entry will never be purged as the value is always reachable from the WeakHashMap instance and through the value, it's key is reachable too. There are other places where Ephemerons could be used - for example in ClassValue API. Try googling for "java ephemeron" (without quotes) to find out other situations where Ephemerons would be beneficial. >> >> If one was to implement Ephemerons in Java, the main question he would be asking is how Ephemeron(s) were supposed to interact with existing Java reference types. Hayes only defines their behavior in isolation, but Java already has 4 "weak" reference types with different "strengths": SoftReference, WeakReference, FinalReference (for implementing finalization) and PhantomReference. In the prototype I choose to define Ephemerons as a new reference type with it's own "strength" for the key referent positioned between WeakReference and FinalReference. It would be possible to merge it with an existing reference type like WeakReference or position it between SoftReference and WeakReference or even entirely "on the top" of the strength list, but I think it would not be wise to position it below FinalReference or even PhantomReference. What's best is an open question. What is also not so obvious is how to define the reachability of the Ephemeron's value and it's interaction with existing reference types. I think I defined it soundly (see the reachability types in the javadoc of [4]). The reason for defining the reachability of the value to alternate between ephemeraly-reachable and weakly-reachable and not between ephemeraly-reachable and strongly-reachable, while theoretically possible, is the desire to have a separate phase of processing for each reachability strength, like it is done currently, and which doesn't affect the performance of processing of existing reference types. >> >> Having skimmed through the reference processing code in the VM for a couple of times, I thought it should not be too complicated to implement another type of "weak" reference. Encouraged by how little changes were needed to remove the sun.misc.Cleaner special type of reference [2], I began experimenting and came up with a prototype that seems to work [3]. Luckily most of the logic to process Reference objects in VM is encapsulated in the ReferenceProcessor class and this logic is invoked from various GC implementations of the hotspot. Changes needed are therefore mostly localized to this class. The webrev also contains recent changes from JDK-8143847 [2] that have not yet propagated to the jdk9-dev repo. When they do, I will prepare a rebased webrev without those changes. >> >> I'm publishing this prototype here to get the answer to the main question: Is there an interest to have Ephemeron reference type in Java? >> >> Given that there was interest, I would also like to initiate a discussion about: >> - What would be the most appropriate "strength" of reachabillity for Epehmeron referents (key and value) and the desired interactions with existing reachabilities. >> - The prototype itself. Since I'm new to this part of code (that's my 1st not so shallow dive into the VM), I surely must have missed places that should be changed. Although the prototype seems to work, I have not created extensive tests for the functionality and have not exposed it to all the various GC algorithms and situations yet. I could use some advise on how to exercise all GC algorithm combinations possible in the VM (what flags to use, for example). >> >> Regards, Peter >> >> [1] https://static.aminer.org/pdf/PDF/000/522/273/ephemerons_a_new_finalization_mechanism.pdf >> [2] https://bugs.openjdk.java.net/browse/JDK-8143847 >> [3] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/ >> [4] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.01/src/java.base/share/classes/java/lang/ref/Ephemeron.java.html >> >> P.S. >> >> To wet some appetite, with the above prototype (applied to current jdk9-dev), it is possible to run the following example: >> >> import java.lang.ref.Ephemeron; >> import java.util.ArrayList; >> import java.util.List; >> >> public class EphemeronTest { >> >> static class Key { >> final int i; >> >> Key(int i) { >> this.i = i; >> } >> >> @Override >> public String toString() { >> return "k" + i; >> } >> } >> >> static class Value { >> final Key key; >> >> Value(Key key) { >> this.key = key; >> } >> >> @Override >> public String toString() { >> return "v(" + key + ")"; >> } >> } >> >> static class Eph extends Ephemeron { >> public Eph(Key key, Value value) { >> super(key, value); >> } >> >> @Override >> public String toString() { >> return getKey() + "=" + getValue(); >> } >> } >> >> static void test(int size, boolean forwardChaining) throws Exception { >> System.out.println(); >> System.out.println((forwardChaining ? "Forward" : "Backward") + >> " chaining of value->key ..."); >> System.out.println(); >> List ephs = new ArrayList<>(size); >> >> Key k1 = new Key(1); >> { >> Key kp = k1; >> for (int i = 2; i <= size; i++) { >> Key ki = new Key(i); >> ephs.add( >> forwardChaining >> ? new Eph(kp, new Value(ki)) >> : new Eph(ki, new Value(kp)) >> ); >> kp = ki; >> } >> ephs.add( >> forwardChaining >> ? new Eph(kp, new Value(k1)) >> : new Eph(k1, new Value(kp)) >> ); >> kp = null; >> } >> >> System.gc(); >> System.out.println("1: " + ephs); >> k1.getClass(); // reachabilityFence >> >> k1 = null; >> System.gc(); >> System.out.println("2: " + ephs); >> } >> >> public static void main(String[] args) throws Exception { >> >> int size = (args.length < 1) >> ? 5 >> : Math.max(2, Integer.parseInt(args[0])); >> >> test(size, true); >> test(size, false); >> } >> } >> >> >> Which prints: >> >> Forward chaining of value->key ... >> >> 1: [k1=v(k2), k2=v(k3), k3=v(k4), k4=v(k5), k5=v(k1)] >> 2: [null=null, null=null, null=null, null=null, null=null] >> >> Backward chaining of value->key ... >> >> 1: [k2=v(k1), k3=v(k2), k4=v(k3), k5=v(k4), k1=v(k5)] >> 2: [null=null, null=null, null=null, null=null, null=null] >> >> >> If you compile the JDK with --enable-debug and use the following VM switches when running: -XX:+PrintGCDetails -XX:+TraceReferenceGC, you can also observe the inner workings of the reference processing, including Ephemerons. >> From steve.drach at oracle.com Fri Jan 22 23:10:29 2016 From: steve.drach at oracle.com (Steve Drach) Date: Fri, 22 Jan 2016 15:10:29 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56A1035A.9020805@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> Message-ID: <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> Hi Alan, et. al., I?ve released a new webrev that addresses all the issues you raised. http://cr.openjdk.java.net/~sdrach/8132734/webrev.03/index.html Specifically: > For Release then I have to admit that I dislike _9 and wonder if other options were considered? javax.lang.model.SourceVersion uses the RELEASE_xx convention for example. Changed to VERSION_9, i.e. Release.VERSION_9 > > Also I wonder about Release.ROOT and whether Release.UNVERSIONED was considered? In general the phrase "root entry" in the javadoc makes me think the root or top-most directory. An alternative that might be clearer is to say "unversioned entry" and define that term clearly in the class description. Changed to BASE, i.e. Release.BASE > > The javadoc for Release.RUNTIME looks like it will have a javadoc link to jdk.Version but that is a JDK-specific API. Could the text starting "The effective runtime ..." move to an @implNote? Done > > I assume @since will be added to Release before this is pushed. Done > > The new JarFile ctor doesn't seem to handle the case that version is null when multi release is forced. Also I assume it should specify @throws SecurityException. Yes, both done and added more @throws clauses > > Could the legacy JarFile ctor be changed to this(file, verify, mode, Release.ROOT) to avoid duplication? Done > > I don't have time to do a detailed pass over the updated tests but I wonder if SimpleHttpServer is really a candidate to put in the testlibrary tree. It looks like it is very specific to multi-release JARs and so I would expect to be co-located with those tests rather than being a hazard in the testlibrary tree. Moved SimpleHttpServer into the test that uses it, MultiReleaseJarHttpProperties > > A small comment is that it would be good to fix the really long lines before these changes are pushed. This will help future side-by-side reviews where it would be otherwise annoying to have to do horizontal scrolling to view diffs. I think I fixed this From gil at azul.com Sat Jan 23 04:13:17 2016 From: gil at azul.com (Gil Tene) Date: Sat, 23 Jan 2016 04:13:17 +0000 Subject: Ephemerons In-Reply-To: <56A2B1EC.6050907@gmail.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> Message-ID: > On Jan 22, 2016, at 2:49 PM, Peter Levart wrote: > > Hi Gil, > > Thanks for taking a look at the Ephemerons for Java. It's great to have a big mind joining the discussion. > > On 01/22/2016 07:12 PM, Gil Tene wrote: >> Peter, >> >> I've been following Ephemerons in other GC'ed environments, and wondering when someone will bring it up for Java. Happy to see attentions given to it. The "conditional weak reference hashmap/table/dictionary" use case seems to be a common primary motivator, and it's a very valid one. Given that we currently do completely concurrent ref processing (in Zing at least) for all reference types, adding Ephemerons to the spec'ed behavior will add some interesting challenges for keeping things concurrent, but I don't think that it is fundamentally undoable (just "hard" and interesting to fully work out). > > The algorithm for processing ephemerons is not much different from that for processing normal references. It's just that it is iterative until it converges to a stable state. If you allow mutator threads to at least in some phases execute concurrently with normal reference processing then I suspect it should be possible to do that for ephemerons too, but I don't have a clue what tricks you perform to be able to allow mutator threads to be concurrent with reference processing. > >> Scanning through the proposal and Java class (mostly JavaDoc spec), I have the following question: >> >> Do we really need a separate "ephemerally reachable" strength below week? The was you extended the definition to the weak definition to say "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by traversing a weak reference or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." would [naively] seem to be sufficient to add and fully describe the desired Ephemeron behavior from a reference strength perspective. > > Ephemeron always touches definitions of at least two consecutive strengths of reachabilities. The prototype says: > > *
  • An object is weakly reachable if it is neither > * strongly nor softly reachable but can be reached by traversing a > * weak reference or by traversing an ephemeron through it's value while > * the ephemeron's key is at least weakly reachable. > > *
  • An object is ephemerally reachable if it is neither > * strongly, softly nor weakly reachable but can be reached by traversing an > * ephemeron through it's key or by traversing an ephemeron through it's value > * while it's key is at most ephemerally reachable. When the ephemerons that > * refer to ephemerally reachable key object are cleared, the key object becomes > * eligible for finalization. Looking into this a bit more, I don't think the above is quite right. Specifically, If an ephemeron's key is either strongly of softly reachable, you want the value to remain appropriately strongly/softly reachable. Without this quality, Ephemeron value referents can (and will) be prematurely collected and finalized while the keys are not. This (IMO) needed quality not provided by the behavior you specify? For a correctly specified behavior, I think all strengths (from strong down) need to be affected by key/value Ephemeron relationships, but without adding an "ephemerally reachable" strength. E.g. I think you fundamentally need something like this: - "An object is strongly reachable if it can be reached by (a) some thread without traversing any reference objects, or by (b) traversing the value of an Ephemeron whose key is strongly reachable. A newly-created object is strongly reachable by the thread that created it" - "An object is softly reachable if it is not strongly reachable but can be reached by (a) traversing a soft reference or by (b) traversing the value of an Ephemeron whose key is softly reachable. - "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by (a) traversing a weak reference or by (b) traversing the value of an ephemeron whose key is weakly reachable. > > But Ephemeron does not need a special reachability strength. It could be merged with WeakReference as far as the reachability of it's key is concerned. In that case it would touch at least the definitions of softly-reachable and weakly-reachable: > > *
  • An object is softly reachable if it is not strongly > * reachable but can be reached by traversing a soft reference or > * by traversing an ephemeron through it's value while > * the ephemeron's key is at least softly reachable. > > *
  • An object is weakly reachable if it is neither > * strongly nor softly reachable but can be reached by traversing a > * weak reference (including ephemeron through it's key) or by traversing > * an ephemeron through it's value while the ephemeron's key is at most > * weakly reachable. When the weak references to a weakly-reachable > * object are cleared (which includes ephemerons to a weakly-reachable key), > * the object becomes eligible for finalization. > > > Presently the hotspot handles different strengths of references by maintaining a distinct set of discovered references for each type and then processing these sets in sequence from the strongest to the weakest. If Ephemerons and WeakReferences were kept in the same discovered set, they would present the same reachability strength for their referent (called also the key in case of ephemeron). > >> Specifically, would modifying your implementation such that Ephemeron extended WeakReference (instead of Reference), and it's V value was tracked with "private WeakReference valueRef" (instead of "private V value") [along with the semi-obvious internal changes that would result] provide what is needed to support proper Ephemeron behavior we just added the "?or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." to the spec'ed definition of "weakly reachable" (and modified the associated JVM ref processing accordingly, which you already have to do to support your wider definitions anyway)? > > In case the strength of ephemeron's key was the same as that of WeakReference's referent, it would make sense for Ephemeron to extend WeakReference, since it would then just be a special kind of WeakReference. But I don't think that changing "private V value" into "private WeakReference valueRef" would buy anything in terms of processing algorithm simplicity. At least in hotspot it would increase the code complexity, since marking the ephemeron's value 'alive' as a consequence of finding it's key marked alive during iteration over the list of ephemerons and removing the current ephemeron from the pending list, would also require removing of the valuRef WeakReference from the pending list at the same time but that would require finding it in the list 1st, since it's a single-linked-list or removing it at next iteration which would increase the number of iterations through the pending set before the state converges... > > There's no problem in making the VM to treat the "private V value" specially. In HotSpot it is only required to update the OopMap for the Ephemeron class to exclude the value field - analogue to how this is done for Reference fields. There's no need for indirection through another WeakReference to get special treatment for value (exclusion from traversals). I agree that the value fields does not need to be a Weakref, and the a "private V value; /* Treated specially by GC */" can be used (much like Reference's declaration of the referent field). So I take back my suggestion on that. I still think that Ephemeron should extend WeakReference, since that places already established rules and expectation on (a) when it will be enqueued, (b) when the collector will clear it (when the the collector encounters the key being weakly reachable), and (c) that clearing of all Ephemeron *and* WeakReference instances who share an identical key value is done atomically, along with (d) all weak references to to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. These last (c, d) parts are critically captured since an Ephemeron *is a* WeakReference, and the statement in WeakReference that says that "? it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references." has a clear application. Here are some suggested edits to the JavaDoc to go with this suggested spec'ed behavior: /** * Ephemeron objects are a special kind of WeakReference objects, which * hold two referents (a key referent and a value referent) and do not prevent their * referents from being made finalizable, finalized, and then reclaimed. * In addition to the key referent, which adheres to the referent behavior of a * WeakReference, an ephemeron also holds a value referent whose reachabiliy * strength is affected by the reachability strength of the key referent: * The value referent of an Ephemeron instance is considered: * (a) strongly reachable if the key referent of the same Ephemeron * object is strongly reachable, or if the value referent is otherwise strongly reachable. * (b) softly reachable if it is not strongly reachable, and (i) the key referent of * the same Ephemeron object is softly reachable, or (ii) if the value referent is otherwise * softly reachable. * (c) weakly reachable if it is not strongly or softly reachable, and (i) the key referent of * the same Ephemeron object is weakly reachable, or (ii) if the value referent is otherwise * weakly reachable. *

    When the collector clears an Ephemeron object instance (according to the rules * expressed for clearing WeakReference object instances), the Ephemeron instance's * key referent value referent are simultaneously and atomically cleared. *

    By convenience, the Ephemeron's referent is also called the key, and can be * obtained either by invoking {@link #get} or {@link #getKey} while the value * can be obtained by invoking {@link #getValue} method. *... > > Regards, Peter > > > > ? Gil. >> >>> On Jan 17, 2016, at 10:18 AM, Peter Levart wrote: >>> >>> Hi, >>> >>> Ephemerons are special kind of references described by Barry Hayes [1]. In the simple variant, they contain two "referents". One is called the key, which can be viewed as a "weak" referent in the style of Java reference types (SoftReference, WeakReference, PhantomReference), the other is called the value, which is also treated specially by GC. It's reachability is dependent on the reachability of the key. >>> >>> Ephemerons solve the problem seen for example in the java.util.WeakHashMap when a value in this map directly or indirectly refers to it's key. Such entry will never be purged as the value is always reachable from the WeakHashMap instance and through the value, it's key is reachable too. There are other places where Ephemerons could be used - for example in ClassValue API. Try googling for "java ephemeron" (without quotes) to find out other situations where Ephemerons would be beneficial. >>> >>> If one was to implement Ephemerons in Java, the main question he would be asking is how Ephemeron(s) were supposed to interact with existing Java reference types. Hayes only defines their behavior in isolation, but Java already has 4 "weak" reference types with different "strengths": SoftReference, WeakReference, FinalReference (for implementing finalization) and PhantomReference. In the prototype I choose to define Ephemerons as a new reference type with it's own "strength" for the key referent positioned between WeakReference and FinalReference. It would be possible to merge it with an existing reference type like WeakReference or position it between SoftReference and WeakReference or even entirely "on the top" of the strength list, but I think it would not be wise to position it below FinalReference or even PhantomReference. What's best is an open question. What is also not so obvious is how to define the reachability of the Ephemeron's value and it's interaction with exis >>> ting refe >>> rence types. I think I defined it soundly (see the reachability types in the javadoc of [4]). The reason for defining the reachability of the value to alternate between ephemeraly-reachable and weakly-reachable and not between ephemeraly-reachable and strongly-reachable, while theoretically possible, is the desire to have a separate phase of processing for each reachability strength, like it is done currently, and which doesn't affect the performance of processing of existing reference types. >>> >>> Having skimmed through the reference processing code in the VM for a couple of times, I thought it should not be too complicated to implement another type of "weak" reference. Encouraged by how little changes were needed to remove the sun.misc.Cleaner special type of reference [2], I began experimenting and came up with a prototype that seems to work [3]. Luckily most of the logic to process Reference objects in VM is encapsulated in the ReferenceProcessor class and this logic is invoked from various GC implementations of the hotspot. Changes needed are therefore mostly localized to this class. The webrev also contains recent changes from JDK-8143847 [2] that have not yet propagated to the jdk9-dev repo. When they do, I will prepare a rebased webrev without those changes. >>> >>> I'm publishing this prototype here to get the answer to the main question: Is there an interest to have Ephemeron reference type in Java? >>> >>> Given that there was interest, I would also like to initiate a discussion about: >>> - What would be the most appropriate "strength" of reachabillity for Epehmeron referents (key and value) and the desired interactions with existing reachabilities. >>> - The prototype itself. Since I'm new to this part of code (that's my 1st not so shallow dive into the VM), I surely must have missed places that should be changed. Although the prototype seems to work, I have not created extensive tests for the functionality and have not exposed it to all the various GC algorithms and situations yet. I could use some advise on how to exercise all GC algorithm combinations possible in the VM (what flags to use, for example). >>> >>> Regards, Peter >>> >>> [1] https://static.aminer.org/pdf/PDF/000/522/273/ephemerons_a_new_finalization_mechanism.pdf >>> [2] https://bugs.openjdk.java.net/browse/JDK-8143847 >>> [3] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/ >>> [4] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.01/src/java.base/share/classes/java/lang/ref/Ephemeron.java.html >>> >>> P.S. >>> >>> To wet some appetite, with the above prototype (applied to current jdk9-dev), it is possible to run the following example: >>> >>> import java.lang.ref.Ephemeron; >>> import java.util.ArrayList; >>> import java.util.List; >>> >>> public class EphemeronTest { >>> >>> static class Key { >>> final int i; >>> >>> Key(int i) { >>> this.i = i; >>> } >>> >>> @Override >>> public String toString() { >>> return "k" + i; >>> } >>> } >>> >>> static class Value { >>> final Key key; >>> >>> Value(Key key) { >>> this.key = key; >>> } >>> >>> @Override >>> public String toString() { >>> return "v(" + key + ")"; >>> } >>> } >>> >>> static class Eph extends Ephemeron { >>> public Eph(Key key, Value value) { >>> super(key, value); >>> } >>> >>> @Override >>> public String toString() { >>> return getKey() + "=" + getValue(); >>> } >>> } >>> >>> static void test(int size, boolean forwardChaining) throws Exception { >>> System.out.println(); >>> System.out.println((forwardChaining ? "Forward" : "Backward") + >>> " chaining of value->key ..."); >>> System.out.println(); >>> List ephs = new ArrayList<>(size); >>> >>> Key k1 = new Key(1); >>> { >>> Key kp = k1; >>> for (int i = 2; i <= size; i++) { >>> Key ki = new Key(i); >>> ephs.add( >>> forwardChaining >>> ? new Eph(kp, new Value(ki)) >>> : new Eph(ki, new Value(kp)) >>> ); >>> kp = ki; >>> } >>> ephs.add( >>> forwardChaining >>> ? new Eph(kp, new Value(k1)) >>> : new Eph(k1, new Value(kp)) >>> ); >>> kp = null; >>> } >>> >>> System.gc(); >>> System.out.println("1: " + ephs); >>> k1.getClass(); // reachabilityFence >>> >>> k1 = null; >>> System.gc(); >>> System.out.println("2: " + ephs); >>> } >>> >>> public static void main(String[] args) throws Exception { >>> >>> int size = (args.length < 1) >>> ? 5 >>> : Math.max(2, Integer.parseInt(args[0])); >>> >>> test(size, true); >>> test(size, false); >>> } >>> } >>> >>> >>> Which prints: >>> >>> Forward chaining of value->key ... >>> >>> 1: [k1=v(k2), k2=v(k3), k3=v(k4), k4=v(k5), k5=v(k1)] >>> 2: [null=null, null=null, null=null, null=null, null=null] >>> >>> Backward chaining of value->key ... >>> >>> 1: [k2=v(k1), k3=v(k2), k4=v(k3), k5=v(k4), k1=v(k5)] >>> 2: [null=null, null=null, null=null, null=null, null=null] >>> >>> >>> If you compile the JDK with --enable-debug and use the following VM switches when running: -XX:+PrintGCDetails -XX:+TraceReferenceGC, you can also observe the inner workings of the reference processing, including Ephemerons. >>> > From gil at azul.com Sat Jan 23 07:47:57 2016 From: gil at azul.com (Gil Tene) Date: Sat, 23 Jan 2016 07:47:57 +0000 Subject: Ephemerons In-Reply-To: <56A2B1EC.6050907@gmail.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> Message-ID: <2FC77775-5155-4FDC-A007-E90DA893ACA4@azul.com> > On Jan 22, 2016, at 2:49 PM, Peter Levart wrote: > > Hi Gil, > > Thanks for taking a look at the Ephemerons for Java. It's great to have a big mind joining the discussion. > > On 01/22/2016 07:12 PM, Gil Tene wrote: >> Peter, >> >> I've been following Ephemerons in other GC'ed environments, and wondering when someone will bring it up for Java. Happy to see attentions given to it. The "conditional weak reference hashmap/table/dictionary" use case seems to be a common primary motivator, and it's a very valid one. Given that we currently do completely concurrent ref processing (in Zing at least) for all reference types, adding Ephemerons to the spec'ed behavior will add some interesting challenges for keeping things concurrent, but I don't think that it is fundamentally undoable (just "hard" and interesting to fully work out). > > The algorithm for processing ephemerons is not much different from that for processing normal references. It's just that it is iterative until it converges to a stable state. If you allow mutator threads to at least in some phases execute concurrently with normal reference processing then I suspect it should be possible to do that for ephemerons too, but I don't have a clue what tricks you perform to be able to allow mutator threads to be concurrent with reference processing. The subject of Concurrent Reference Processing has not been widely written about. While we've been doing it in our production collectors (PGC, C4 in our Vega and Zing products) since roughly 2007, we had not published papers about the mechanisms involved. Partially because it seemed to be an "obscure" area of GC. I believe there are versions of Metronome that also dealt with non-STW ref processing, but without publishing the mechanisms either. In fact, the first publication I am aware of that deals with the subject of Concurrent (aka "on the fly") collection and reference types is "Reference Object Processing in On-The-Fly Garbage Collection" by Ugawa, Jones, and Ritson (published in 2014 at ISMM'14, and can be found here , with a presentation here ). I believe that I can say (with some authority) that there are sound and reliable ways to achieve fully concurrent (as in Zero STW) reference processing for the non-strong reference types in Java today (soft, weak, phantom, final). But that doing so requires some interesting state machine modeling and careful "throwing of gates" to achieve atomic transitions in behavior without requiring STW processing of references (which is probably a must), and without even requiring an STW transition point (which is a nice bonus for the perfectionists among us, but probably acceptable to do without if the transition included no processing of references). The mechanisms we use rely on an ability to carefully "throw a gate" at a point in the tracing where strong ref processing (including all soft refs that have been policy-chosen to be strengthened in this cycle) is known to be complete, and after which references to non-strongly-marked objects can safely return null (and before which they return a strong reference). Ephemerons will almost certainly mess with how that gate currently works... So while I believe that dealing with Ephemerons in a STW ref processing world is fairly straight forward, adding the Ephemeron behavior into a concurrent ref processing world will "mess" with the delicate transitions and invariants that make the current mechanisms work. Still, I'm fairly confident that we could still find a way to deal with that. But I don't have one formulated fully at the moment, and don't know of anyone else who has... > >> Scanning through the proposal and Java class (mostly JavaDoc spec), I have the following question: >> >> Do we really need a separate "ephemerally reachable" strength below week? The was you extended the definition to the weak definition to say "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by traversing a weak reference or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." would [naively] seem to be sufficient to add and fully describe the desired Ephemeron behavior from a reference strength perspective. > > Ephemeron always touches definitions of at least two consecutive strengths of reachabilities. The prototype says: > > *

  • An object is weakly reachable if it is neither > * strongly nor softly reachable but can be reached by traversing a > * weak reference or by traversing an ephemeron through it's value while > * the ephemeron's key is at least weakly reachable. > > *
  • An object is ephemerally reachable if it is neither > * strongly, softly nor weakly reachable but can be reached by traversing an > * ephemeron through it's key or by traversing an ephemeron through it's value > * while it's key is at most ephemerally reachable. When the ephemerons that > * refer to ephemerally reachable key object are cleared, the key object becomes > * eligible for finalization. > > > But Ephemeron does not need a special reachability strength. It could be merged with WeakReference as far as the reachability of it's key is concerned. In that case it would touch at least the definitions of softly-reachable and weakly-reachable: > > *
  • An object is softly reachable if it is not strongly > * reachable but can be reached by traversing a soft reference or > * by traversing an ephemeron through it's value while > * the ephemeron's key is at least softly reachable. > > *
  • An object is weakly reachable if it is neither > * strongly nor softly reachable but can be reached by traversing a > * weak reference (including ephemeron through it's key) or by traversing > * an ephemeron through it's value while the ephemeron's key is at most > * weakly reachable. When the weak references to a weakly-reachable > * object are cleared (which includes ephemerons to a weakly-reachable key), > * the object becomes eligible for finalization. > > > Presently the hotspot handles different strengths of references by maintaining a distinct set of discovered references for each type and then processing these sets in sequence from the strongest to the weakest. If Ephemerons and WeakReferences were kept in the same discovered set, they would present the same reachability strength for their referent (called also the key in case of ephemeron). > >> Specifically, would modifying your implementation such that Ephemeron extended WeakReference (instead of Reference), and it's V value was tracked with "private WeakReference valueRef" (instead of "private V value") [along with the semi-obvious internal changes that would result] provide what is needed to support proper Ephemeron behavior we just added the "?or by traversing an ephemeron through it's value while the ephemeron's key is at least weakly reachable." to the spec'ed definition of "weakly reachable" (and modified the associated JVM ref processing accordingly, which you already have to do to support your wider definitions anyway)? > > In case the strength of ephemeron's key was the same as that of WeakReference's referent, it would make sense for Ephemeron to extend WeakReference, since it would then just be a special kind of WeakReference. But I don't think that changing "private V value" into "private WeakReference valueRef" would buy anything in terms of processing algorithm simplicity. At least in hotspot it would increase the code complexity, since marking the ephemeron's value 'alive' as a consequence of finding it's key marked alive during iteration over the list of ephemerons and removing the current ephemeron from the pending list, would also require removing of the valuRef WeakReference from the pending list at the same time but that would require finding it in the list 1st, since it's a single-linked-list or removing it at next iteration which would increase the number of iterations through the pending set before the state converges... > > There's no problem in making the VM to treat the "private V value" specially. In HotSpot it is only required to update the OopMap for the Ephemeron class to exclude the value field - analogue to how this is done for Reference fields. There's no need for indirection through another WeakReference to get special treatment for value (exclusion from traversals). > > Regards, Peter > > > > ? Gil. >> >>> On Jan 17, 2016, at 10:18 AM, Peter Levart wrote: >>> >>> Hi, >>> >>> Ephemerons are special kind of references described by Barry Hayes [1]. In the simple variant, they contain two "referents". One is called the key, which can be viewed as a "weak" referent in the style of Java reference types (SoftReference, WeakReference, PhantomReference), the other is called the value, which is also treated specially by GC. It's reachability is dependent on the reachability of the key. >>> >>> Ephemerons solve the problem seen for example in the java.util.WeakHashMap when a value in this map directly or indirectly refers to it's key. Such entry will never be purged as the value is always reachable from the WeakHashMap instance and through the value, it's key is reachable too. There are other places where Ephemerons could be used - for example in ClassValue API. Try googling for "java ephemeron" (without quotes) to find out other situations where Ephemerons would be beneficial. >>> >>> If one was to implement Ephemerons in Java, the main question he would be asking is how Ephemeron(s) were supposed to interact with existing Java reference types. Hayes only defines their behavior in isolation, but Java already has 4 "weak" reference types with different "strengths": SoftReference, WeakReference, FinalReference (for implementing finalization) and PhantomReference. In the prototype I choose to define Ephemerons as a new reference type with it's own "strength" for the key referent positioned between WeakReference and FinalReference. It would be possible to merge it with an existing reference type like WeakReference or position it between SoftReference and WeakReference or even entirely "on the top" of the strength list, but I think it would not be wise to position it below FinalReference or even PhantomReference. What's best is an open question. What is also not so obvious is how to define the reachability of the Ephemeron's value and it's interaction with exis >>> ting refe >>> rence types. I think I defined it soundly (see the reachability types in the javadoc of [4]). The reason for defining the reachability of the value to alternate between ephemeraly-reachable and weakly-reachable and not between ephemeraly-reachable and strongly-reachable, while theoretically possible, is the desire to have a separate phase of processing for each reachability strength, like it is done currently, and which doesn't affect the performance of processing of existing reference types. >>> >>> Having skimmed through the reference processing code in the VM for a couple of times, I thought it should not be too complicated to implement another type of "weak" reference. Encouraged by how little changes were needed to remove the sun.misc.Cleaner special type of reference [2], I began experimenting and came up with a prototype that seems to work [3]. Luckily most of the logic to process Reference objects in VM is encapsulated in the ReferenceProcessor class and this logic is invoked from various GC implementations of the hotspot. Changes needed are therefore mostly localized to this class. The webrev also contains recent changes from JDK-8143847 [2] that have not yet propagated to the jdk9-dev repo. When they do, I will prepare a rebased webrev without those changes. >>> >>> I'm publishing this prototype here to get the answer to the main question: Is there an interest to have Ephemeron reference type in Java? >>> >>> Given that there was interest, I would also like to initiate a discussion about: >>> - What would be the most appropriate "strength" of reachabillity for Epehmeron referents (key and value) and the desired interactions with existing reachabilities. >>> - The prototype itself. Since I'm new to this part of code (that's my 1st not so shallow dive into the VM), I surely must have missed places that should be changed. Although the prototype seems to work, I have not created extensive tests for the functionality and have not exposed it to all the various GC algorithms and situations yet. I could use some advise on how to exercise all GC algorithm combinations possible in the VM (what flags to use, for example). >>> >>> Regards, Peter >>> >>> [1] https://static.aminer.org/pdf/PDF/000/522/273/ephemerons_a_new_finalization_mechanism.pdf >>> [2] https://bugs.openjdk.java.net/browse/JDK-8143847 >>> [3] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/ >>> [4] http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.01/src/java.base/share/classes/java/lang/ref/Ephemeron.java.html >>> >>> P.S. >>> >>> To wet some appetite, with the above prototype (applied to current jdk9-dev), it is possible to run the following example: >>> >>> import java.lang.ref.Ephemeron; >>> import java.util.ArrayList; >>> import java.util.List; >>> >>> public class EphemeronTest { >>> >>> static class Key { >>> final int i; >>> >>> Key(int i) { >>> this.i = i; >>> } >>> >>> @Override >>> public String toString() { >>> return "k" + i; >>> } >>> } >>> >>> static class Value { >>> final Key key; >>> >>> Value(Key key) { >>> this.key = key; >>> } >>> >>> @Override >>> public String toString() { >>> return "v(" + key + ")"; >>> } >>> } >>> >>> static class Eph extends Ephemeron { >>> public Eph(Key key, Value value) { >>> super(key, value); >>> } >>> >>> @Override >>> public String toString() { >>> return getKey() + "=" + getValue(); >>> } >>> } >>> >>> static void test(int size, boolean forwardChaining) throws Exception { >>> System.out.println(); >>> System.out.println((forwardChaining ? "Forward" : "Backward") + >>> " chaining of value->key ..."); >>> System.out.println(); >>> List ephs = new ArrayList<>(size); >>> >>> Key k1 = new Key(1); >>> { >>> Key kp = k1; >>> for (int i = 2; i <= size; i++) { >>> Key ki = new Key(i); >>> ephs.add( >>> forwardChaining >>> ? new Eph(kp, new Value(ki)) >>> : new Eph(ki, new Value(kp)) >>> ); >>> kp = ki; >>> } >>> ephs.add( >>> forwardChaining >>> ? new Eph(kp, new Value(k1)) >>> : new Eph(k1, new Value(kp)) >>> ); >>> kp = null; >>> } >>> >>> System.gc(); >>> System.out.println("1: " + ephs); >>> k1.getClass(); // reachabilityFence >>> >>> k1 = null; >>> System.gc(); >>> System.out.println("2: " + ephs); >>> } >>> >>> public static void main(String[] args) throws Exception { >>> >>> int size = (args.length < 1) >>> ? 5 >>> : Math.max(2, Integer.parseInt(args[0])); >>> >>> test(size, true); >>> test(size, false); >>> } >>> } >>> >>> >>> Which prints: >>> >>> Forward chaining of value->key ... >>> >>> 1: [k1=v(k2), k2=v(k3), k3=v(k4), k4=v(k5), k5=v(k1)] >>> 2: [null=null, null=null, null=null, null=null, null=null] >>> >>> Backward chaining of value->key ... >>> >>> 1: [k2=v(k1), k3=v(k2), k4=v(k3), k5=v(k4), k1=v(k5)] >>> 2: [null=null, null=null, null=null, null=null, null=null] >>> >>> >>> If you compile the JDK with --enable-debug and use the following VM switches when running: -XX:+PrintGCDetails -XX:+TraceReferenceGC, you can also observe the inner workings of the reference processing, including Ephemerons. >>> > From timo.kinnunen at gmail.com Sat Jan 23 09:50:09 2016 From: timo.kinnunen at gmail.com (Timo Kinnunen) Date: Sat, 23 Jan 2016 10:50:09 +0100 Subject: OpenJDK8: java.util.stream.Stream.onClose In-Reply-To: References: Message-ID: <56a34cdf.460f1c0a.03c7.3dbd@mx.google.com> Hi, The pipeline stages don?t really need to mutate themselves (as they can mutate the Stream if needed). So they should be shareable between Streams, alleviating some of cost of creating new copies of a Stream, right? -- Have a nice day, Timo Sent from Mail for Windows 10 From: Paul Sandoz Sent: Friday, January 22, 2016 18:45 Cc: core-libs-dev Libs Subject: Re: OpenJDK8: java.util.stream.Stream.onClose Hi Carsten, If you read the documentation further on down it states: * first exception, since an exception cannot suppress itself.) May * return itself. The same applies to parallel/sequential/unordered. For these cases it would be rather inefficient to create new stages in the pipeline, for essentially functional no-ops regarding actual element processing, and would likely increase the cost when performing the terminal operation. Paul. > On 22 Jan 2016, at 17:14, Carsten Varming wrote: > > Dear core-libs-devs, > > I was playing around with streams and onClose and ran into an example with > odd behavior (well, I found it odd, see example below). I was wondering if > you could point me to prior discussions on semantics of intermediate > operations and the semantics of Stream.onClose. > > A bit of code so we have something concrete to discuss: > > import java.util.stream.Stream; > > public class Example { > public static void main(String[] args) { > final Stream foo = Stream.of("1", "2", "3"); > final Stream bar = foo.onClose(() -> > System.out.println("Closing bar")); > > try (final Stream s = foo) { > s.forEach(System.out::println); > } > } > } > > When I run the program above I get: > 1 > 2 > 3 > Closing bar > > Hmm, the onClose call mutated foo. This is surprising to me as when I read > [1] it says onClose is an intermediate operation and I quote: "Returns an > equivalent stream with an additional close handler.". The documentation > does not mention that the stream passed to onClose is mutated. Is this the > intended behavior or is it a bug? I hope it is a bug, but I would like to > understand the reasons for the current implementation. > > [1] > https://docs.oracle.com/javase/8/docs/api/java/util/stream/BaseStream.html#onClose-java.lang.Runnable- > > Carsten From amaembo at gmail.com Sat Jan 23 10:38:20 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sat, 23 Jan 2016 16:38:20 +0600 Subject: RFR 8147505: Clarification of BaseStream.onClose() behavior In-Reply-To: <1616962894.20160116091332@gmail.com> References: <1666489469.20160115103543@gmail.com> <68E50067-D3FB-414F-80AD-40F188DAE5FC@oracle.com> <1616962894.20160116091332@gmail.com> Message-ID: <0886057.20160123163820@gmail.com> Hello! Here's a webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8147505/r1/ I just added a check in AbstractPipeline.onClose and the corresponding unit-test. To me it seems that no documentation corrections are necessary, because "onClose" is already documented that it's an intermediate operation: http://download.java.net/jdk9/docs/api/java/util/stream/BaseStream.html#onClose-java.lang.Runnable- Also it's already documented: > A stream should be operated on (invoking an intermediate or terminal > stream operation) only once. This rules out, for example, "forked" > streams, where the same source feeds two or more pipelines, or > multiple traversals of the same stream. A stream implementation may > throw IllegalStateException if it detects that the stream is being > reused. http://download.java.net/jdk9/docs/api/java/util/stream/Stream.html So this already implies that invoking intermediate operation "onClose" after stream terminal operation is not allowed and may throw IllegalStateException. Please review! With best regards, Tagir Valeev. TFV> Seems that my original letter sounds confusing due to my bad English. TFV> Every occurrence of "idempotent" should be read as "not idempotent". TFV> Sorry for this! PS>> We did not bother to throw ISEs from parallel/sequence/onClose PS>> primarily because they return ?this? and they are kind of PS>> harmless, but the undefined behaviour you point out could be shut PS>> down without much concern for backwards compatibility. We can PS>> modify them to check whether they have been linked or consumed PS>> (while not updating the previous stage). PS>> Wanna log an issue? TFV> Sure: TFV> https://bugs.openjdk.java.net/browse/JDK-8147505 TFV> Thanks, TFV> Tagir Valeev. PS>> Thanks, PS>> Paul. >>> On 15 Jan 2016, at 05:35, Tagir F. Valeev wrote: >>> >>> Hello! >>> >>> Current documentation for BaseStream.onClose() does not specify >>> explicitly how the stream would behave if additional close handlers >>> are registered after the stream is consumed or even closed. The >>> implementation actually allows registering the additional close >>> handlers after consumption or closure and close() method is >>> idempotent: newly registered handlers are perfectly called: >>> >>> Stream s = Stream.of("content"); >>> s = s.onClose(() -> System.out.println("A")); >>> s.forEach(System.out::println); >>> s = s.onClose(() -> System.out.println("B")); >>> s.close(); // prints A and B >>> s = s.onClose(() -> System.out.println("C")); >>> s.close(); // prints C >>> >>> (removing "s =" produces the same result). >>> >>> I think if such behavior is intended, it should be explicitly noted in >>> the specification, as third-party implementations of BaseStream >>> interface should provide consistent behavior. Or at least it should be >>> noted that some BaseStream implementations may have idempotent close() >>> method, so the derived AutoCloseable objects (which own the BaseStream >>> object) should be aware about this behavior and provide idempotent >>> close() method as well. Without knowing this one may write: >>> >>> class MyObject implements AutoCloseable { >>> private BaseStream s; >>> >>> public MyObject(BaseStream s) { >>> this.s = s; >>> } >>> >>> @Override >>> public void close() throws Exception { >>> if(s != null) { >>> s.close(); >>> s = null; >>> } >>> } >>> } >>> >>> Such code closes the stream only once, so newly registered handlers >>> will not be called if MyObject::close is called the second time. >>> >>> However, to my opinion the current behavior is weird and should be >>> changed in order not to allow registering new close handles (throwing >>> IllegalStateException) when the stream is already closed, or even >>> better when the stream is linked/consumed. As far as I know currently >>> in JDK close handlers are registered only for non-consumed stream, so >>> such change would not break existing JDK code (though may break some >>> strange third party code). It should be noted that AutoCloseable >>> interface discourages idempotent close() methods (though not forbids >>> them). >>> >>> What do you think? >>> >>> With best regards, >>> Tagir Valeev. >>> From amaembo at gmail.com Sat Jan 23 12:43:32 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sat, 23 Jan 2016 18:43:32 +0600 Subject: RFR 8148115: Stream.findFirst for unordered source optimization In-Reply-To: <30A68664-D525-41BF-9688-FAB7E1F6944A@oracle.com> References: <1152739344.20160122121906@gmail.com> <30A68664-D525-41BF-9688-FAB7E1F6944A@oracle.com> Message-ID: <916767911.20160123184332@gmail.com> Hello, Paul! Thank you for review. I implemented it according to your suggestions, here's issue: https://bugs.openjdk.java.net/browse/JDK-8148115 And webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8148115/r1/ This patch required some changes in existing tests as FindFirstOpTest asserted that findFirst() result must be equal to .iterator().next() even if the source is HashSet. I extracted the relevant part of FindAnyOpTest into assertContains(Optional, Iterator) method and placed it into LambdaTestHelpers. The corresponding changes in primitive tests are unnecessary, because it seems that primitive data providers never produce unordered streams. I can try to add unordered primitive sources and correct these tests as well if it's necessary. With best regards, Tagir Valeev. PS> Hi Tagir, PS> Thanks for looking at this. PS> I would prefer if a boolean was passed into the task constructor e.g.: PS> @Override PS> public O evaluateParallel(PipelineHelper helper, PS> Spliterator spliterator) { PS> // This takes into account the upstream ops flags and the terminal PS> // op flags and therefore takes into account findFirst or findAny PS> boolean mustFindFirst = PS> StreamOpFlag.ORDERED.isKnown(helper.getStreamAndOpFlags()); PS> return new FindTask<>(this, mustFindFirst, helper, spliterator).invoke(); PS> } PS> so we keep the flag evaluation logic within the evaluate method. PS> Then change FindOp.mustFindFirst to be an int of the opFlags calculated in the constructor: PS> this.opFlags = StreamOpFlag.IS_SHORT_CIRCUIT | (mustFindFirst ? 0 : StreamOpFlag.NOT_ORDERED); PS> Paul. >> On 22 Jan 2016, at 07:19, Tagir F. Valeev wrote: >> >> Hello! >> >> Seems that currently Stream.findFirst is not optimized for unordered >> source. I think it should work as findAny in this case. Here's a small >> patch which fixes this: >> >> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/find_patch.txt >> >> Simple JMH test: >> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/FindTest.java >> >> Original: >> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_orig.txt >> # JMH 1.11.2 (released 85 days ago) >> # VM version: JDK 9-ea, VM 9-ea+99-2015-12-23-183325.javare.4146.nc >> Benchmark Mode Cnt Score Error Units >> FindTest.findAny avgt 30 12439,631 ? 1787,866 us/op >> FindTest.findAnyUnordered avgt 30 12923,080 ? 1072,537 us/op >> FindTest.findFirst avgt 30 48047,467 ? 2713,489 us/op >> FindTest.findFirstUnordered avgt 30 52648,893 ? 3934,682 us/op >> >> Patched: >> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_patched.txt >> Benchmark Mode Cnt Score Error Units >> FindTest.findAny avgt 30 11312,238 ? 386,627 us/op >> FindTest.findAnyUnordered avgt 30 12136,953 ? 1536,817 us/op >> FindTest.findFirst avgt 30 47517,776 ? 2844,607 us/op >> FindTest.findFirstUnordered avgt 30 13147,492 ? 1140,592 us/op >> >> If you think it's a reasonable thing to patch, I can log an issue, >> generate webrev and check whether jtreg tests pass. >> >> With best regards, >> Tagir Valeev. >> From forax at univ-mlv.fr Sat Jan 23 12:55:42 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Sat, 23 Jan 2016 13:55:42 +0100 (CET) Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A26B33.2010308@oracle.com> References: <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> Message-ID: <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> I agree with Vladimir, You should not be able to transform/redefine a VM anonymous class because the transformer will certainly mess up with the order of the constant pool entries. Slightly off-topic, about ASM, when you create a ClassWriter [1], you can pass a ClassReader of an existing class, in that case ASM copy the constant pool from the class reader to the class writer so the constant pool is preserved. R?mi [1] http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 ----- Mail original ----- > De: "Vladimir Ivanov" > ?: "Rafael Winterhalter" > Cc: "Coleen Phillimore" , core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com > Spitsyn" , "Daniel Daugherty" > Envoy?: Vendredi 22 Janvier 2016 18:47:31 > Objet: Re: ClassFileTransformer does not apply to anonymous classes > > Rafael, > > First of all, I'd like to agree on the terminology. There's some > confusion there. Anonymous term is ambiguous in Java. There are > anonymous classes on language level and there's > Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense > classes. I prefer to call them VM anonymous classes to avoid confusion. > > I assume that whenever you use anonymous you assume "VM anonymous". > > > thank you for your response. While I completely understand your view > > from a VM implementor's point of view, as a practicioner I would > > recommend against it. Not being able to instrument lambda expressions > > puts a severe limitation onto using the instrumentation API > > alltogether. For example, a monitoring application that promises to > > record all invocations of a method of a certain interface that only > > works 95% of the time is not 5% less usefull but might no longer be > > useful at all. People have build large applications based on the > > assumption that all user application code can be instrumented and such > > a regression would hurt them. For example, until today, over 30 people > > that use my code generation framework reached out to me and reported > > the reported behavior as a bug in my library and people are only > > starting to migrate their applications to Java 8. The outcome is > > simply not intuitive as using a lambda expression over an anonyous > > inner class should not change an application's behavior. > Can you elaborate on that point, please? > > I don't see any problems with instrumenting user code. Javac represents > lambda expression body as a private static method of the enclosing > class, which can be instrumented. VM anonymous classes are used only as > a mechanism to glue functional interfaces and lambda expressions > together at runtime. > > For example: > static void f(Runnable r) { r.run(); } > f(() -> {}); > > Test::f (7 bytes) > @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) > @ 0 Test::lambda$main$0 (1 bytes) inline (hot) > > Why do you need to instrument Test$$Lambda$1/... and not > Test::lambda$main$0? > > > The currently used workaround that people use is to instrument the > > LambdaMetaFactory class itself to apply the transformer manually when > > it is being created. This solution is spreading as a standard approach > > and I am sure that forbidding a redefinition alltogether would only > > inspire to other workarrounds for being able to migrate running code > > to the next Java version. > It doesn't sound like a workaround, but as a solution for a different > problem. > > If there's a need to gather profile for every lambda expression > instantiation site (indy call), then redefining bootstrap method is the > right way to go IMO. It allows to intercept and customize indy call site > binding. > > What does sound as a workaround is an attempt to instrument classes > produced by LambdaMetaFactory. Right now, it generates a fresh VM > anonymous class on every request, but it is an implementation detail and > not a requirement. > > For example, LambdaMetaFactory can reuse wrappers on per-functional > interface basis. It would provide significant footprint savings for > lambda expression-rich code bases (usually, there are much more > instantiations than functional interface flavors). > > > Furthermore, I do not think that not being able to patch constant pool > > indices in the generated code introduces any problems in practice. > > Most real-world instrumentation only appends new constant pool entries > > without interfering with the existant pool. Rather, I would like to > > see an exception being raised if one attempts to change the original > > constant pool without me being able to consider this from a technical > > perspective. I think this would serve to be sufficient for most > > people. > If you care only about lambda expressions, then constant pool patching > shouldn't be a problem (AFAIR it isn't used for lambda expressions). But > in a more generic case (even for java.lang.invoke purposes), the > coupling between class file and CP patches is very tight and can be > easily broken with benign class file transformations. Considering ASM > library, I'm not sure that even simple instrumentations preserve > original constant pool structure. > > Probably, VM can do some structural checks on CP to forbid any > modifications except appends, but there are still aliasing problems - > sharing doesn't work (even for string constants), so new code shouldn't > use original CP entries. > > I'd prefer to avoid such increase in complexity in VM implementation. > > > I really hope that there is a way to allow for patching anonymously > > loaded classes even without exposing the constant pool patches. > From VM implementation perspective I don't see any problems with > allowing class file redefinition for VM anonymous classes. What I want > to do is to discourage from doing so. Again - it is very fragile and > there's little we can do to fix that. > > > So far, thank you for looking into this. I am sure this a more complex > > matter than what I am able to comprehend. I appreciate that you are > > taking the time to consider my opinion! > > Best regards, > Vladimir Ivanov > From peter.levart at gmail.com Sat Jan 23 13:14:28 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 23 Jan 2016 14:14:28 +0100 Subject: Ephemerons In-Reply-To: References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> Message-ID: <56A37CB4.3000407@gmail.com> Hi Gil, it's good to have this discussion. See comments inline... On 01/23/2016 05:13 AM, Gil Tene wrote: .... >> On Jan 22, 2016, at 2:49 PM, Peter Levart > > wrote: >> >> Ephemeron always touches definitions of at least two consecutive >> strengths of reachabilities. The prototype says: >> >> *
  • An object is weakly reachable if it is neither >> * strongly nor softly reachable but can be reached by traversing a >> * weak reference or by traversing an ephemeron through it's value while >> * the ephemeron's key is at least weakly reachable. >> >> *
  • An object is ephemerally reachable if it is neither >> * strongly, softly nor weakly reachable but can be reached by >> traversing an >> * ephemeron through it's key or by traversing an ephemeron through >> it's value >> * while it's key is at most ephemerally reachable. When the >> ephemerons that >> * refer to ephemerally reachable key object are cleared, the key >> object becomes >> * eligible for finalization. > > Looking into this a bit more, I don't think the above is quite right. > Specifically, If an ephemeron's key is either strongly of softly > reachable, you want the value to remain appropriately strongly/softly > reachable. Without this quality, Ephemeron value referents can (and > will) be prematurely collected and finalized while the keys are not. > This (IMO) needed quality not provided by the behavior you specify? This is not quite true. While ephemeron's value is weakly or even ephemerally-reachable, it is not finalizable, because ephemeraly-reachable is stronger than finaly-reachable. After ephemeron's key becomes ephemeraly-reachable, the ephemeron is cleared by GC which sets it's key *and* value to null atomically. The life of key and value at that moment becomes untangled. Either of them can have a finalizer or not and both of them will eventually be collected if not revived by their finalize() methods. But it can never happen that ephemeron's value is finalized or collected while it's key is still reachable through the ephemeron (while the ephemeron is not cleared yet). But I agree that it would be desirable for ephemeron's value to follow the reachability of it's key. In above specification, if the key is strongly reachable, the value is weakly reachable, so any WeakReferences or SoftReferences pointing at the Ephemeron's value can already be cleared while the key is still strongly reachable. This is arguably no different than current specification of Soft vs. Weak references. A SoftReference can already be cleared while its referent is still reachable through a WeakReference, but for Ephemeron's value this might be confusing. The easier to understand conceptual model for Ephemerons might be a pair of (WeakReference, WeakReference) where the key has a virtual strong reference to the value. And this is what we get if we say that reachability of the value follows reachability of the key. > > For a correctly specified behavior, I think all strengths (from strong > down) need to be affected by key/value Ephemeron relationships, but > without adding an "ephemerally reachable" strength. E.g. I think you > fundamentally need something like this: > > - "An object is strongly reachable if it can be reached by > (a) some thread without traversing any reference objects, or by (b) > traversing the value of an Ephemeron whose key is strongly reachable. > A newly-created object is strongly reachable by the thread that > created it" > > - "An object is softly reachable if it is not > strongly reachable but can be reached by (a) traversing a soft > reference or by (b) traversing the value of an Ephemeron whose key is > softly reachable. > > - "An object is weakly reachable if it is neither strongly > nor softly reachable but can be reached by (a) traversing a weak > reference or by (b) traversing the value of an ephemeron whose key is > weakly reachable. ...and that's where we stop, because when we make Ephemeron just a special kind of WeakReference, the next thing that happens is: *

    Suppose that the garbage collector determines at a certain point in time * that an object is weakly * reachable. At that time it will atomically clear all weak references to * that object and all weak references to any other weakly-reachable objects * from which that object is reachable through a chain of strong and soft * references. At the same time it will declare all of the formerly * weakly-reachable objects to be finalizable. At the same time or at some * later time it will enqueue those newly-cleared weak references that are * registered with reference queues. ...where "clearing of the WeakReference" means reseting the key *and* value to null in case it is an Ephemeron; and "all weak references to some object" means Ephemerons that have that object as a key (but not those that only have it as a value!) in case of ephemerons ... > I still think that Ephemeron should extend WeakReference, > since that places already established rules and expectation on (a) > when it will be enqueued, (b) when the collector will clear it (when > the the collector encounters the key being weakly reachable), and > (c) that clearing of all Ephemeron *and* WeakReference instances who > share an identical key value is done atomically, along with (d) all > weak references to to any other weakly-reachable objects from which > that object is reachable through a chain of strong and soft > references. These last (c, d) parts are critically captured since an > Ephemeron *is a* WeakReference, and the statement in WeakReference > that says that "? it will atomically clear all weak references to that > object and all weak references to any other weakly-reachable objects > from which that object is reachable through a chain of strong and soft > references." has a clear application. > > Here are some suggested edits to the JavaDoc to go with this suggested > spec'ed behavior: > /** > * Ephemeron objects are a special kind of WeakReference > objects, which > * hold two referents (a key referent and a value referent) and do > not prevent their > * referents from being made finalizable, finalized, and then reclaimed. > * In addition to the key referent, which adheres to the referent > behavior of a > * WeakReference, an ephemeron also holds a value referent whose > reachabiliy > * strength is affected by the reachability strength of the key > referent: > * The value referent of an Ephemeron instance is considered: > * (a) strongly reachable if the key referent of the same Ephemeron > * object is strongly reachable, or if the value referent is > otherwise strongly reachable. > * (b) softly reachable if it is not strongly reachable, and (i) the > key referent of > * the same Ephemeron object is softly reachable, or (ii) if the > value referent is otherwise > * softly reachable. > * (c) weakly reachable if it is not strongly or softly reachable, > and (i) the key referent of > * the same Ephemeron object is weakly reachable, or (ii) if the > value referent is otherwise > * weakly reachable. > *

    When the collector clears an Ephemeron object instance > (according to the rules > * expressed for clearing WeakReference object instances), the > Ephemeron instance's > * key referent value referent are simultaneously and atomically cleared. > *

    By convenience, the Ephemeron's referent is also called the > key, and can be > * obtained either by invoking {@link #get} or {@link #getKey} while > the value > * can be obtained by invoking {@link #getValue} method. > *... Thanks, this is very nice. I do like this behavior more. Let me see what it takes to implement this strategy... Regards, Peter From chris.hegarty at oracle.com Sat Jan 23 16:16:51 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sat, 23 Jan 2016 16:16:51 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref Message-ID: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> sun.misc.Cleaner was previously listed as a critical internal API in JEP 260 [1], but on further investigation it has been moved to an open issue, for the following reasons: 1) its primary use in the JDK is within NIO direct buffers to release native memory. The base module cannot have a dependency on the jdk.unsupported module so will need to be updated to use an alternative cleaner, 2) the usage of Cleaner outside the JDK, as determined by corpus analysis, has largely been observed to hack into private fields of the internal NIO direct buffer classes to explicitly release native memory. As stated in 1), the type of the cleaner used by NIO direct buffers will have to change. Given this, and the fact that JDK 9 has a new general purposed cleaner API, java.lang.ref.Cleaner [2] the value of keep sun.misc.Cleaner is questionable. This issue proposes to simply move Cleaner into an internal non-exported package in the base module so that it can be used by the NIO direct buffers classes, and small number of other places. Webrev: http://cr.openjdk.java.net/~chegar/8148117/ If, at some point in the future, it is determined that sun.misc.Cleaner is in fact a critical internal API ( with static usage ), then it can be reinstated as a subtype of jdk.internal.ref.Cleaner, providing the same public API. Note: some popular open source libraries that hack into the internal private cleaner field of direct buffers will have to have their code updated, if they wish to continue to do this. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8132928 [1] https://bugs.openjdk.java.net/browse/JDK-8138696 From Alan.Bateman at oracle.com Sat Jan 23 16:36:45 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 23 Jan 2016 16:36:45 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> Message-ID: <56A3AC1D.50003@oracle.com> On 23/01/2016 16:16, Chris Hegarty wrote: > : > > Webrev: > http://cr.openjdk.java.net/~chegar/8148117/ > > This has to move to your patch looks okay. You might need to update the TEST.groups to ensure that the existOnThrow tests will be run by jdk_core. -Alan. From Alan.Bateman at oracle.com Sat Jan 23 16:39:05 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 23 Jan 2016 16:39:05 +0000 Subject: RFR (XS): 8148044: Remove Enum[0] constants from EnumSet and EnumMap In-Reply-To: <56A2917B.6040205@oracle.com> References: <56A213AD.9090605@oracle.com> <56A21822.2000203@oracle.com> <56A218FF.6070002@oracle.com> <56A21B81.7090800@oracle.com> <56A23B0D.2050407@oracle.com> <56A24E50.6050705@oracle.com> <56A2786A.5090507@oracle.com> <56A2917B.6040205@oracle.com> Message-ID: <56A3ACA9.3060902@oracle.com> On 22/01/2016 20:30, Claes Redestad wrote: > : > > I've posted a webrev including your changes here: > http://cr.openjdk.java.net/~redestad/8148044/webrev.03/ > > If there's a preference for this version I'll push it after the weekend. This version looks good to me too. I wonder if it's worth putting the instructions for generated serializedForm into BogusEnumSet given that you had to re-generate it. -Alan From gil at azul.com Sat Jan 23 18:25:51 2016 From: gil at azul.com (Gil Tene) Date: Sat, 23 Jan 2016 18:25:51 +0000 Subject: Ephemerons In-Reply-To: <56A37CB4.3000407@gmail.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> <56A37CB4.3000407@gmail.com> Message-ID: <8154B857-931F-4BED-872A-36892E050CCF@azul.com> > On Jan 23, 2016, at 5:14 AM, Peter Levart wrote: > > Hi Gil, it's good to have this discussion. See comments inline... > > On 01/23/2016 05:13 AM, Gil Tene wrote: > .... >>> On Jan 22, 2016, at 2:49 PM, Peter Levart < peter.levart at gmail.com > wrote: >>> >>> Ephemeron always touches definitions of at least two consecutive strengths of reachabilities. The prototype says: >>> >>> *

  • An object is weakly reachable if it is neither >>> * strongly nor softly reachable but can be reached by traversing a >>> * weak reference or by traversing an ephemeron through it's value while >>> * the ephemeron's key is at least weakly reachable. >>> >>> *
  • An object is ephemerally reachable if it is neither >>> * strongly, softly nor weakly reachable but can be reached by traversing an >>> * ephemeron through it's key or by traversing an ephemeron through it's value >>> * while it's key is at most ephemerally reachable. When the ephemerons that >>> * refer to ephemerally reachable key object are cleared, the key object becomes >>> * eligible for finalization. >> >> Looking into this a bit more, I don't think the above is quite right. Specifically, If an ephemeron's key is either strongly of softly reachable, you want the value to remain appropriately strongly/softly reachable. Without this quality, Ephemeron value referents can (and will) be prematurely collected and finalized while the keys are not. This (IMO) needed quality not provided by the behavior you specify? > > This is not quite true. While ephemeron's value is weakly or even ephemerally-reachable, it is not finalizable, because ephemeraly-reachable is stronger than finaly-reachable. After ephemeron's key becomes ephemeraly-reachable, the ephemeron is cleared by GC which sets it's key *and* value to null atomically. The life of key and value at that moment becomes untangled. Either of them can have a finalizer or not and both of them will eventually be collected if not revived by their finalize() methods. But it can never happen that ephemeron's value is finalized or collected while it's key is still reachable through the ephemeron (while the ephemeron is not cleared yet). > > But I agree that it would be desirable for ephemeron's value to follow the reachability of it's key. In above specification, if the key is strongly reachable, the value is weakly reachable, so any WeakReferences or SoftReferences pointing at the Ephemeron's value can already be cleared while the key is still strongly reachable. This is arguably no different than current specification of Soft vs. Weak references. A SoftReference can already be cleared while its referent is still reachable through a WeakReference, We seem to agree about the cleaner behavior specification (in both of our texts below), so the these next paragraphs are really about arguing for why this is an important design choice if/when adding Ephemerons to Java: It is true the [current] spec allows for soft references to an object to be cleared while weak references to the same object are not: the "determines" in "Suppose that the garbage collector determines at a certain point in time hat an object is RRRR reachable..." part [for RRRR = {soft, weak}] does not have to happen at the same "certain point in time". However, to my knowledge all current implementations present as if this determination is happening at the same "point in time" for all weakly and softly reachable objects combined. Specifically [in implementations]: if soft reachability is determined for an object at some point in time, then weak reachability for that object is determined at the same point in time. And the weak reachability determination for an object depends on whether the collector chose to clear existing soft references to that object at that same point in time, with the appearance of the choice to clear (or not to clear) soft references to a given object atomically affecting the determination of it's weak reachability. Since the collector is *required* to act on a weak determination when it is made, while it *may* act on a soft determination when it is made, making the combined determination at the same "point in time" eliminates an obviously confusing situation that is not prohibited by the spec: if the determination for weak and soft reachability was not done at the same point in time, then an object that was softly reachable and had it's soft references cleared and queued could later become strongly reachable, and even softly reachable again. When reference processing is done as a STW thing, this "combined determination" effect is a trivial side-effect of STW. When it is done concurrently (or incrementally?), implementations still work to maintain the appearance of combined atomic determination of soft and weak reachability. I know ours does. In our case, we do it because we had no desire to be the ones to argue "I know that all implementations did this atomically because they were STW, but the spec allows us to add this bug to your program?". So in actual implementations (to my knowledge), finalization is currently the only mechanism that can create this "strange situation" where an object was no longer strongly reachable, had actions triggered as a result from loss of strong reachability (i.e. actually observed by the program as "known to not be strongly reachable"), and later became strongly reachable again. E.g. a finalizer can propagate a strong reference to a previously non-strongly reachable object ('this' in the finalizer, or anything that 'this' transitively refers and was not otherwise reachable when the finalizer was called).. This is one of those "undesired" things that the introduction of Reference types was meant to deal with (Reference types were introduced in 1.2, after finalization was unfortunately already included and spec'ed. And phantom refs were meant to allow for a cleaner form that could replace finalization). And while the specifications of SoftReference and WeakReference do not prohibit it, implementations are not required to allow it, and in practice non of them do (I think), as doing so would most likely expose some "interesting" spec-allowed-but-extremely-surprising things/bugs that none of us want to have to defend... In this context, it would be a "highly undesirable" design choice to introduce Ephemerons in a way that would them to return a strong reference to an object that has previously been determined to no longer be strongly reachable. Structuring the spec to prohibit this is a better design choice. To highlight the design choice here, let me describe a specific problem scenario for which the previous (above) spec would cause "re-strengthening" behavior that would break assumptions that are allowed under the current spec: in the above/previously specified behavior an object V that is known to have no finalizers, but has e.g. 3 WeakReference objects that refer to it, can become weakly reachable while both a key referent object K in some ephemeron E with a value referent of V remain strongly reachable. At such a point (V is weakly reachable, K and E are strongly reachable), the collector may determine weak reachability for V, [atomically] clear all weak references to V, and enqueue those weak reference objects on their respective queues. While V is still ephemerally reachable under your previous definition, there are no references to it anywhere other than in ephemeron value referent fields, and weak references that did refer to it have been cleared and queued. Since the ephemeron is still there, and the key is still there, and the ephemeron has not been cleared, an Ephemeron.getValue() call would create a strong reference to an object that was previously determined to not be weakly reachable. Re-creating a strong reference to V after the point where weak references to V were cleared and the weak refs to it were enqueued would be "surprising" to current weak reference based code (the only thing that could cause this under the current spec would be a finalizer), so allowing that (jn the spec) is likely to break all kinds of logic that depends on currently spec'ed weak reference behaviors. The spec'ed behavior we seem to be agreeing on (below) would prohibit this loophole and would [I think] maintain any reachability-based expectations that current weak-ref based logic can make under the current spec. Maintaining this continuity is an important design choice for adding Ephemerons into the current set of Reference behaviors. And since I suspect that all implementations will continue to choose to do the "determination" of soft and weak reachability at the same "point in time", this will fit well with how people would build this stuff anyway. Separate note: It would be separately interesting to consider narrowing the SoftRef spec to require JVM implementations to atomically clear all soft *and* weak references to an object at the same time. I.e. if the garbage collector chooses to clear a soft reference to an object that would become weakly reachable as a result, then all weak references to that object must be [atomically] cleared at the same time. Since I suspect that all current JVM implementations actually adhere to this stronger requirement already, this would not "hurt" anything or require extra work to comply with. [Anyone from Metronome or some other non-STW reference processing implementations want to chime in?]. > but for Ephemeron's value this might be confusing. The easier to understand conceptual model for Ephemerons might be a pair of (WeakReference, WeakReference) where the key has a virtual strong reference to the value. And this is what we get if we say that reachability of the value follows reachability of the key. > >> >> >> For a correctly specified behavior, I think all strengths (from strong down) need to be affected by key/value Ephemeron relationships, but without adding an "ephemerally reachable" strength. E.g. I think you fundamentally need something like this: >> >> - "An object is strongly reachable if it can be reached by (a) some thread without traversing any reference objects, or by (b) traversing the value of an Ephemeron whose key is strongly reachable. A newly-created object is strongly reachable by the thread that created it" >> >> - "An object is softly reachable if it is not strongly reachable but can be reached by (a) traversing a soft reference or by (b) traversing the value of an Ephemeron whose key is softly reachable. >> >> - "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by (a) traversing a weak reference or by (b) traversing the value of an ephemeron whose key is weakly reachable. > > ...and that's where we stop, because when we make Ephemeron just a special kind of WeakReference, the next thing that happens is: > > *

    Suppose that the garbage collector determines at a certain point in time > * that an object is weakly > * reachable. At that time it will atomically clear all weak references to > * that object and all weak references to any other weakly-reachable objects > * from which that object is reachable through a chain of strong and soft > * references. At the same time it will declare all of the formerly > * weakly-reachable objects to be finalizable. At the same time or at some > * later time it will enqueue those newly-cleared weak references that are > * registered with reference queues. > > ...where "clearing of the WeakReference" means reseting the key *and* value to null in case it is an Ephemeron; and > "all weak references to some object" means Ephemerons that have that object as a key (but not those that only have it as a value!) in case of ephemerons > > ... >> I still think that Ephemeron should extend WeakReference, since that places already established rules and expectation on (a) when it will be enqueued, (b) when the collector will clear it (when the the collector encounters the key being weakly reachable), and (c) that clearing of all Ephemeron *and* WeakReference instances who share an identical key value is done atomically, along with (d) all weak references to to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. These last (c, d) parts are critically captured since an Ephemeron *is a* WeakReference, and the statement in WeakReference that says that "? it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references." has a clear application. >> >> Here are some suggested edits to the JavaDoc to go with this suggested spec'ed behavior: >> /** >> * Ephemeron objects are a special kind of WeakReference objects, which >> * hold two referents (a key referent and a value referent) and do not prevent their >> * referents from being made finalizable, finalized, and then reclaimed. >> * In addition to the key referent, which adheres to the referent behavior of a >> * WeakReference, an ephemeron also holds a value referent whose reachabiliy >> * strength is affected by the reachability strength of the key referent: >> * The value referent of an Ephemeron instance is considered: >> * (a) strongly reachable if the key referent of the same Ephemeron >> * object is strongly reachable, or if the value referent is otherwise strongly reachable. >> * (b) softly reachable if it is not strongly reachable, and (i) the key referent of >> * the same Ephemeron object is softly reachable, or (ii) if the value referent is otherwise >> * softly reachable. >> * (c) weakly reachable if it is not strongly or softly reachable, and (i) the key referent of >> * the same Ephemeron object is weakly reachable, or (ii) if the value referent is otherwise >> * weakly reachable. >> *

    When the collector clears an Ephemeron object instance (according to the rules >> * expressed for clearing WeakReference object instances), the Ephemeron instance's >> * key referent value referent are simultaneously and atomically cleared. >> *

    By convenience, the Ephemeron's referent is also called the key, and can be >> * obtained either by invoking {@link #get} or {@link #getKey} while the value >> * can be obtained by invoking {@link #getValue} method. >> *... > > > Thanks, this is very nice. I do like this behavior more. > > Let me see what it takes to implement this strategy... > > Regards, Peter > From uschindler at apache.org Sat Jan 23 19:23:03 2016 From: uschindler at apache.org (Uwe Schindler) Date: Sat, 23 Jan 2016 20:23:03 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> Message-ID: <003701d15613$7ce20690$76a613b0$@apache.org> (reposting with correct sender address) Hi, > Note: some popular open source libraries that hack into the internal > private cleaner field of direct buffers will have to have their code > updated, if they wish to continue to do this. I followed parts of this issue, but as representative from Apache Lucene, which really really needs to un-mmap MappedByteBuffers (otherwise Lucene gets unuseable with JDK 9), I have no idea what the fix for us should be? What is the replacement for the following code (see marked BufferCleaner impl, which is our abstraction): https://goo.gl/DGYZZj The main reason why sun.misc.Cleaner was on the critical API list is NOT the case that somebody wanted to implement their own cleaner as replacement for finalization. The main reason (and this is *not* solved by the new APIs in java.lang.ref.Cleaner) is to *force* unmapping of direct and mmapped byte buffers. The unmapping of mmapped buffers is really needed and must be in Java 9, otherwise projects like Lucene will not work with Java 9, or may be slow as hell (if they use non-mmapped I/O) or otherwise run out of disk space (because mmapped files use disk space as they are never ever freed by the GC - see below). On Windows, in addition, files that are mmapped cannot be closed. These problem are the worst desaster that can happen to Lucene - a desaster like the big Hotspot issues back in 2011. With this patch the above code part would no longer work, because you can still make the getCleaner method available with setAccessible in Java 9 Jigsaw, but invoking the clean() method on the cleaner will horribly fail, because the cleaner is part of an internal package. So you have 2 possibilities: - Re-add sun.misc.Cleaner with a limited API (only the public clean() method) and let DirectBuffer return an instance of this sun.misc.Cleaner public API, so it is accessible without breaking encapsulation by adding extra exports - Or fix the full issue: Add "official" support to unmmap mmapped files. Really! You have to do this, otherwise it's a desaster (see below for the "why"). Mark Reinhold: I will as every year contact you on the FOSDEM dinner about sun.misc.Cleaner and unmapping DirectBuffers :-) Be sure to have a heated discussion! FYI: The reason why unmapping mmapped buffers is really needed is the following: MappedByteBuffer/DirectBuffers are very small objects (GC wise). Those are created by Lucene on mapping huuuuuuge files (several tens or hundreds of Gigabytes for some large indexes). Those objects live for longer time (up to several hours). But from time to time, Lucene releases the buffers and maps new ones (when updates occur on disk). Because those small - only a few bytes for the Garbage Collector - buffer instances lived so long, there is no reason to free them asap by GC. In fact, they are never freed if your application runs for several days (as Lucene/SOlr/Elasticsearch servers are living). So the small 20 byte objects hang forever on gigabytes of data! No way to realease that! So Lucene/Solr/Elasticserach's misuse of sun.misc.Cleaner is just to force the DirectBuffers to free the huge amounts of resources they hold. The new API of java.lang.ref.Cleaner does not help at all with this. You are just hiding the only possible workaround! Sorry for the bad news, but there needs to be a decision! We are great supporters of Jigsaw, but if we have to disable Jigsaw encapsulation, just to get access to jdk.internal.**.Cleaner, you are destroying the additional encapsulation for us! Please add a new issue to either revert to have sun.misc.Cleaner available just to forcefully free mmapped files, or add an official API to allow unmapping (including Hotspot changes to prevent SIGSEGV/SIGBUS). Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Chris Hegarty > Sent: Saturday, January 23, 2016 5:17 PM > To: core-libs-dev > Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref > > sun.misc.Cleaner was previously listed as a critical internal API in JEP > 260 [1], but on further investigation it has been moved to an open issue, > for the following reasons: > 1) its primary use in the JDK is within NIO direct buffers to release > native memory. The base module cannot have a dependency on the > jdk.unsupported module so will need to be updated to use an > alternative cleaner, > 2) the usage of Cleaner outside the JDK, as determined by corpus > analysis, has largely been observed to hack into private fields of > the internal NIO direct buffer classes to explicitly release native > memory. > > As stated in 1), the type of the cleaner used by NIO direct buffers will > have to change. Given this, and the fact that JDK 9 has a new general > purposed cleaner API, java.lang.ref.Cleaner [2] the value of keep > sun.misc.Cleaner is questionable. > > This issue proposes to simply move Cleaner into an internal non-exported > package in the base module so that it can be used by the NIO direct > buffers classes, and small number of other places. > > Webrev: > http://cr.openjdk.java.net/~chegar/8148117/ > > > If, at some point in the future, it is determined that sun.misc.Cleaner > is in fact a critical internal API ( with static usage ), then it can be > reinstated as a subtype of jdk.internal.ref.Cleaner, providing the same > public API. > > Note: some popular open source libraries that hack into the internal > private cleaner field of direct buffers will have to have their code > updated, if they wish to continue to do this. > > -Chris. > > [1] https://bugs.openjdk.java.net/browse/JDK-8132928 > [1] https://bugs.openjdk.java.net/browse/JDK-8138696 From aph at redhat.com Sat Jan 23 19:32:31 2016 From: aph at redhat.com (Andrew Haley) Date: Sat, 23 Jan 2016 19:32:31 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <003701d15613$7ce20690$76a613b0$@apache.org> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> Message-ID: <56A3D54F.1070300@redhat.com> Here's a question for you: could you tolerate a closeable mapped ByteBuffer which was somewhat slower to access? It is hard to make a secure closeable mapped ByteBuffer: the question is how small the overhead of closeability can be made. But if the answer is "We can't tolerate *any* additional overhead, no matter how small" then there is no motivation to do any experiments. Andrew. From uschindler at apache.org Sat Jan 23 20:01:05 2016 From: uschindler at apache.org (Uwe Schindler) Date: Sat, 23 Jan 2016 21:01:05 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A3D54F.1070300@redhat.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> Message-ID: <004301d15618$cd340f80$679c2e80$@apache.org> Hi, > Here's a question for you: could you tolerate a closeable mapped > ByteBuffer which was somewhat slower to access? It is hard to make a > secure closeable mapped ByteBuffer: the question is how small the > overhead of closeability can be made. Lucene uses the ByteBuffer like a conventional byte[] arrays and expects as close performance to that as possible. In Java 7 and Java 8 there is for sure some large overhead because of missing Hotspot optimizations. In Java 9 we found out, that recent builds are now almost as fast as byte[] access. Completely destroying that improvement would be no good. :-( We are also looking into using VarHandles instead of ByteBuffers directly (using the new MethodHandles methods to get a VarHandle on top of any ByteBuffer, although this does not really fit our use-case completely, because we cannot use views like long[] on top of a ByteBuffer, because we sometimes also need unaligned accesses). We will try to check this out as soon as first builds with VarHandles are available. Of course this would also not help, if a Closeable ByteBuffer would also slowdown the VarHandle accesses. > But if the answer is "We can't tolerate *any* additional overhead, no > matter how small" then there is no motivation to do any experiments. It depends how small! If the speed is still somewhere between Java 8 ByteBuffer performance and the recent Hotspot improvements in Java 9, I agree with trying it out. But some volatile memory access on every access is a no-go. The code around ByteBufferIndexInput in Lucene is the most performance-critical critical code, because on every search query or sorting there is all the work happening in there (millions of iterations with positional ByteBuffer.get* calls). As ByteBuffers are limited to 2 GiB, we also need lots of hairy code to work around that limitation! If you look at ByteBufferIndexInput's code you will see that we simply do stuff like trying to read from one bytebuffer and only if we catch an BufferUnderflowException we fall back to handling buffer switches: Instead of checking bounds on every access, we have fallback code only happening on exceptions. E.g. if you are 3 bytes before end of one buffer slice and read a long, it will throw BufferUnderflow. When this happens the code will fall back to read byte by byte from 2 different buffers and reassemble the long): https://goo.gl/g1jKcm Stuff like sorting will make heavy use of the ByteBufferIndexInput's (RandomAccessIndexInput interface's) methods like readXxx(long position) for stuff like quicksort with really hundreds of millions of items! > Andrew. Thanks, Uwe From aph at redhat.com Sat Jan 23 20:31:28 2016 From: aph at redhat.com (Andrew Haley) Date: Sat, 23 Jan 2016 20:31:28 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <004301d15618$cd340f80$679c2e80$@apache.org> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> Message-ID: <56A3E320.2070004@redhat.com> On 23/01/16 20:01, Uwe Schindler wrote: > It depends how small! If the speed is still somewhere between Java 8 > ByteBuffer performance and the recent Hotspot improvements in Java > 9, I agree with trying it out. But some volatile memory access on > every access is a no-go. The code around ByteBufferIndexInput in > Lucene is the most performance-critical critical code, because on > every search query or sorting there is all the work happening in > there (millions of iterations with positional ByteBuffer.get* > calls). As ByteBuffers are limited to 2 GiB, we also need lots of > hairy code to work around that limitation! Yes, I see that code. It would be helpful if there were a self-contained but realistic benchmark using that code. That way, some simple experiments would allow changes to be measured. > If you look at ByteBufferIndexInput's code you will see that we > simply do stuff like trying to read from one bytebuffer and only if > we catch an BufferUnderflowException we fall back to handling buffer > switches: Instead of checking bounds on every access, we have > fallback code only happening on exceptions. E.g. if you are 3 bytes > before end of one buffer slice and read a long, it will throw > BufferUnderflow. When this happens the code will fall back to read > byte by byte from 2 different buffers and reassemble the long): I'm surprised you don't see painful deoptimization traps when that happens. I suppose it's rare enough that you don't care. There's a new group of methods in JDK9 called Objects.checkIndex() which are intended to provide a very efficient way to do bounds checks. It might be interesting to see if they work well with ByteBufferIndexInput: that's an important use case. BTW, does anyone here know why we don't have humongous ByteBuffers with a long index? Andrew. From Alan.Bateman at oracle.com Sat Jan 23 21:03:05 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 23 Jan 2016 21:03:05 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <003701d15613$7ce20690$76a613b0$@apache.org> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> Message-ID: <56A3EA89.7080804@oracle.com> On 23/01/2016 19:23, Uwe Schindler wrote: > : > > What is the replacement for the following code (see marked BufferCleaner impl, which is our abstraction): https://goo.gl/DGYZZj > > The main reason why sun.misc.Cleaner was on the critical API list is NOT the case that somebody wanted to implement their own cleaner as replacement for finalization. The main reason (and this is *not* solved by the new APIs in java.lang.ref.Cleaner) is to *force* unmapping of direct and mmapped byte buffers. JEP 260 was updated recently to move this to an open issue and I think the text captures the issue correctly. Part of it that we can't have types in java.base depending on types in other modules and this is why Buffers can't have fields of type sun.misc.Cleaner or any type in sun.misc. As has come several times, there are huge security and reliability issues that arise when releasing or unmapping memory that is still accessible via reachable objects. Proposed solutions over the years have traded performance or were limited to platforms where we could atomically remap. JDK 9 may be the time to try yet again or else just introduce a JDK-specific API to unmap that comes with a warning in 96pt font. It might have to be disabled completely when running with a security manager. Alternatively the burden of the current hack could be reduced by having Cleaner implement Runnable with a run method that invokes clean. That would avoid needing to cast to a JDK-internal type and so avoiding needing to use -XaddExports to break encapsulation. -Alan. From roger.riggs at oracle.com Sat Jan 23 21:40:25 2016 From: roger.riggs at oracle.com (Roger Riggs) Date: Sat, 23 Jan 2016 16:40:25 -0500 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> Message-ID: <56A3F349.5070500@oracle.com> Hi, Looks good. I would want to verify that it is defined such that if it later needs to be abstracted up to ChronoLocalDate and apply to compatible Chronologies and respective local dates such as JapaneseDate there is no conflict. It should not be an issue since Period implements ChronoPeriod. In the respective implementations, the estimation/computation of the number of steps would need to depend on the Chronology's definition of months. I can sponsor this enhancement. Thanks, Roger On 1/20/16 10:15 AM, Stephen Colebourne wrote: > I'm happy with the logic and specification of this proposal. I think it > will be a useful addition. > > I'll let the Oracle team chime in to do a further review. > > thanks > Stephen > > > > On 16 January 2016 at 13:31, Tagir F. Valeev wrote: > >> Hello! >> >> Thanks for review! Here's the updated patch: >> http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ >> >> SC> The docs say that if the end date is before the start date, the >> SC> stream is empty. While I can see just about why LongStream.range() >> SC> works that way, I don't think this API should. The minimum is an >> SC> exception, but it would be easy to support negative in the >> SC> days-only case or the months-only case. The problem is where there >> SC> are both months/years and days and those should be exceptions. >> >> Now datesUntil(endExclusive) throws an IllegalArgumentException if end >> date is before start date. >> >> datesUntil(endExclusive, step) supports negative periods. It throws >> IllegalArgumentException if: >> - step is zero >> - step.toTotalMonths() and step.getDays() both non-zero and have >> opposite sign >> - step is negative and end date is after start date >> - step is positive and end date is before start date >> >> Otherwise it works correctly: you can use >> LocalDate.of(2016, 1, 1) >> .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); >> >> Steps like Period.of(-1, -1, -1) are also supported. >> >> SC> The single-arg implementation uses plusDays() with an >> SC> incrementing number. When the performance patch goes in, the >> SC> proposed streaming implementation will be optimal for small >> SC> streams but less optimal for large ones. The fastest way to loop >> SC> over a list of dates would be to manually generate them by >> SC> incrementing the day until it exceeds the length of month, and so >> SC> on. However, this would be serial. >> >> As I understand, plusDays performance patch is already pushed. >> >> It's possible to write custom Spliterator which would use >> previous.plusDays(1) in tryAdvance() and jump (via >> LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is >> requested. However this adds at least one additional class and more >> complexity. I guess, such optimization can be considered later as >> separate issue when API is stabilized. >> >> SC> As such, I think the best way to write this, taking account of >> SC> how plusDays() is implemented, is as follows: >> >> SC> LongStream.range(start.toEpochDay(), >> SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); >> >> Ok, now it's done this way. >> >> SC> In the period-based method, it would be best to capture the case >> SC> where totalMonths == 0 and days > 0 and forward to another method >> SC> that ignores months. That private method can share implementation >> SC> with the public single-arg method (passing in 1). >> >> This case still more complex than one-day case as it requires division >> and multiplication. Thus I'd keep these case separately. However I >> simplified "months = 0" path using ofEpochDay, now it should be >> faster. >> >> SC> The docs for the period-based method should specify the strategy >> SC> that produces the results (in abstract terms). This needs to cover >> SC> that the result is equivalent to always adding to the start period >> SC> a multiple of the period. >> >> I added some clarifications, please check. >> >> SC> Some nits: >> SC> I prefer to avoid @link in the first sentence. Just using 'stream' >> would be sufficient. >> >> Done. >> >> SC> The first sentence should be a summary. In this case it probably has a >> bit too much detail. >> >> Done. >> >> SC> The @return has 'values' on a new line when it could be on the same >> line. >> >> I set now line length = 100 characters in my IDE. Is it ok? >> >> SC> If statements need braces. >> >> Done. >> >> With best regards, >> Tagir Valeev. >> >> From uschindler at apache.org Sat Jan 23 21:57:06 2016 From: uschindler at apache.org (Uwe Schindler) Date: Sat, 23 Jan 2016 22:57:06 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A3EA89.7080804@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3EA89.7080804@oracle.com> Message-ID: <006401d15629$0294f760$07bee620$@apache.org> Hi, Alan Bateman [mailto:Alan.Bateman at oracle.com] wrote: > On 23/01/2016 19:23, Uwe Schindler wrote: > > : > > > > What is the replacement for the following code (see marked BufferCleaner > impl, which is our abstraction): https://goo.gl/DGYZZj > > > > The main reason why sun.misc.Cleaner was on the critical API list is NOT the > case that somebody wanted to implement their own cleaner as replacement > for finalization. The main reason (and this is *not* solved by the new APIs in > java.lang.ref.Cleaner) is to *force* unmapping of direct and mmapped byte > buffers. > JEP 260 was updated recently to move this to an open issue and I think > the text captures the issue correctly. Part of it that we can't have > types in java.base depending on types in other modules and this is why > Buffers can't have fields of type sun.misc.Cleaner or any type in sun.misc. > > As has come several times, there are huge security and reliability > issues that arise when releasing or unmapping memory that is still > accessible via reachable objects. Proposed solutions over the years have > traded performance or were limited to platforms where we could > atomically remap. JDK 9 may be the time to try yet again or else just > introduce a JDK-specific API to unmap that comes with a warning in 96pt > font. It might have to be disabled completely when running with a > security manager. Alternatively the burden of the current hack could be > reduced by having Cleaner implement Runnable with a run method that > invokes clean. That would avoid needing to cast to a JDK-internal type > and so avoiding needing to use -XaddExports to break encapsulation. We know this problem. Because of this, e.g. Elasticsearch runs with a security manager and only allows lucene-core.jar access to do this reflection on sun.misc package (using the correct permission, only given to JAR file). I was about to suggest the same thing: Maybe add a public API to byte buffer, that is documented to throw SecurityException. Add a new Runtime/WhateverPermission that can be enabled to work around that (e.g., Elasticsearch would do this and allow this Permission only to lucene-core.jar). In addition add red and blinking warnings to this method :-) In any case, a possible solution was proposed by Andrew Haley (he had some ideas in the past, I think we brought this up the last time a few months ago) with a small slowdown, but still providing better ByteBuffer performance than in Java 7 or Java 8. I would hope for a solution that installs a SIGSEGV signal handler in the JDK, that translates any completely illegal access that would otherwise lead to a crash of the JDK to some Java Exception. The security issues you also mention do not apply for us (e.g., reading memory from another web application in the same JVM), so a new permission for SecurityManager would also help here. Uwe From uschindler at apache.org Sat Jan 23 22:07:01 2016 From: uschindler at apache.org (Uwe Schindler) Date: Sat, 23 Jan 2016 23:07:01 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A3E320.2070004@redhat.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> Message-ID: <006501d1562a$64ee77a0$2ecb66e0$@apache.org> Hi Andrew, Andrew Haley [mailto:aph at redhat.com] wrote: > On 23/01/16 20:01, Uwe Schindler wrote: > > > It depends how small! If the speed is still somewhere between Java 8 > > ByteBuffer performance and the recent Hotspot improvements in Java > > 9, I agree with trying it out. But some volatile memory access on > > every access is a no-go. The code around ByteBufferIndexInput in > > Lucene is the most performance-critical critical code, because on > > every search query or sorting there is all the work happening in > > there (millions of iterations with positional ByteBuffer.get* > > calls). As ByteBuffers are limited to 2 GiB, we also need lots of > > hairy code to work around that limitation! > > Yes, I see that code. It would be helpful if there were a > self-contained but realistic benchmark using that code. That way, > some simple experiments would allow changes to be measured. Unfortunately I am a bit occupied with preparing for FOSDEM and also we switched Lucene/Solr to GIT today, so I cannot write a benchmark today. I know that Robert Muir (I CC'ed him) did a lot of performance testing with the ByteBufferIndexInput, maybe we can work both on some isolated benchmark, that you can use for testing. But this may take a while. > > If you look at ByteBufferIndexInput's code you will see that we > > simply do stuff like trying to read from one bytebuffer and only if > > we catch an BufferUnderflowException we fall back to handling buffer > > switches: Instead of checking bounds on every access, we have > > fallback code only happening on exceptions. E.g. if you are 3 bytes > > before end of one buffer slice and read a long, it will throw > > BufferUnderflow. When this happens the code will fall back to read > > byte by byte from 2 different buffers and reassemble the long): > > I'm surprised you don't see painful deoptimization traps when that > happens. I suppose it's rare enough that you don't care. Exactly! Of course the accesses to the ByteBufferIndexInput are not really completely random. You always have accesses concentrated on places close (otherwise this would produce lots of swapping and I/O). As the size of the ByteBuffer (the chunkSizePower in this code) is 1 GiB (2^30) by default, the probability to catch that exception and cause deoptimization is very low. By the way it is not 2 GiB, because signed integers and maximum array size in Java don't allow to reach 2^31 (only 2^21 - 1, damn!). So we chunk the file into 1 GiB large ByteBuffers. > There's a > new group of methods in JDK9 called Objects.checkIndex() which are > intended to provide a very efficient way to do bounds checks. It might > be interesting to see if they work well with ByteBufferIndexInput: > that's an important use case. We can check for this. The main idea in this code is to not do similar bounds checks both in ByteBuffer and outside. After switching to the try/catch we improved the whole situation. We can try with Java 9, but using Objects.checkIndex() can at earliest be used once Lucene is on Java 9 (or with Multi-Version-JAR files with ByteBufferIndexInput alternate impl for Java 9+). > BTW, does anyone here know why we don't have humongous ByteBuffers > with a long index? I think 64 bit arrays were proposed for Java 10, so are ByteBuffers with longs (or VarHandles using longs). This is also an old and long going discussion. Uwe > Andrew. Thanks, Uwe From chris.hegarty at oracle.com Sat Jan 23 22:52:18 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sat, 23 Jan 2016 22:52:18 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <006401d15629$0294f760$07bee620$@apache.org> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3EA89.7080804@oracle.com> <006401d15629$0294f760$07bee620$@apache.org> Message-ID: <7FA611E5-C261-4809-9F3D-6009B43144F3@oracle.com> Just catching up? I see no objection to the proposed patch, but the obvious concern ( that is highlighted in JEP 260 ), about the digging into the private internal details of NIO buffers. I think Alan summarized the issues around providing a public API for releasing/unmapping native memory very well, and if this does not make it for JDK 9, then Alan?s alternative ( implements Runnable ) may be a better interim solution than the current use of reflection. -Chris. > On 23 Jan 2016, at 21:57, Uwe Schindler wrote: > > Hi, > > Alan Bateman [mailto:Alan.Bateman at oracle.com] wrote: >> On 23/01/2016 19:23, Uwe Schindler wrote: >>> : >>> >>> What is the replacement for the following code (see marked BufferCleaner >> impl, which is our abstraction): https://goo.gl/DGYZZj >>> >>> The main reason why sun.misc.Cleaner was on the critical API list is NOT the >> case that somebody wanted to implement their own cleaner as replacement >> for finalization. The main reason (and this is *not* solved by the new APIs in >> java.lang.ref.Cleaner) is to *force* unmapping of direct and mmapped byte >> buffers. >> JEP 260 was updated recently to move this to an open issue and I think >> the text captures the issue correctly. Part of it that we can't have >> types in java.base depending on types in other modules and this is why >> Buffers can't have fields of type sun.misc.Cleaner or any type in sun.misc. >> >> As has come several times, there are huge security and reliability >> issues that arise when releasing or unmapping memory that is still >> accessible via reachable objects. Proposed solutions over the years have >> traded performance or were limited to platforms where we could >> atomically remap. JDK 9 may be the time to try yet again or else just >> introduce a JDK-specific API to unmap that comes with a warning in 96pt >> font. It might have to be disabled completely when running with a >> security manager. Alternatively the burden of the current hack could be >> reduced by having Cleaner implement Runnable with a run method that >> invokes clean. That would avoid needing to cast to a JDK-internal type >> and so avoiding needing to use -XaddExports to break encapsulation. > > We know this problem. Because of this, e.g. Elasticsearch runs with a security manager and only allows lucene-core.jar access to do this reflection on sun.misc package (using the correct permission, only given to JAR file). > > I was about to suggest the same thing: Maybe add a public API to byte buffer, that is documented to throw SecurityException. Add a new Runtime/WhateverPermission that can be enabled to work around that (e.g., Elasticsearch would do this and allow this Permission only to lucene-core.jar). In addition add red and blinking warnings to this method :-) > > In any case, a possible solution was proposed by Andrew Haley (he had some ideas in the past, I think we brought this up the last time a few months ago) with a small slowdown, but still providing better ByteBuffer performance than in Java 7 or Java 8. > > I would hope for a solution that installs a SIGSEGV signal handler in the JDK, that translates any completely illegal access that would otherwise lead to a crash of the JDK to some Java Exception. The security issues you also mention do not apply for us (e.g., reading memory from another web application in the same JVM), so a new permission for SecurityManager would also help here. > > Uwe > From peter.levart at gmail.com Sun Jan 24 10:52:57 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 24 Jan 2016 11:52:57 +0100 Subject: Ephemerons In-Reply-To: <8154B857-931F-4BED-872A-36892E050CCF@azul.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> <56A37CB4.3000407@gmail.com> <8154B857-931F-4BED-872A-36892E050CCF@azul.com> Message-ID: <56A4AD09.1010901@gmail.com> Hi Gil, I totally agree with your assessment. We should not introduce another way of reviving the almost collectable objects and I fully support tightening the specification so that soft and weak references to the same referent and to other referents from which this referent is reachable are required to be cleared together atomically. I modified the prototype to (hopefully) adhere to this new Ephemeron specification that Gil and I agreed upon. Anyone interested in experimenting can find it here: http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.02/ http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.hotspot.02/ It is rebased to current tip of jdk9-dev repositories (after the bulk of merges for jdk-9+102), but still contains the change to remove the Cleaner reference type as it has not yet managed to get in... I have also added a test that is a start for verifying the functionality. Regards, Peter On 01/23/2016 07:25 PM, Gil Tene wrote: > >> On Jan 23, 2016, at 5:14 AM, Peter Levart > > wrote: >> >> Hi Gil, it's good to have this discussion. See comments inline... >> >> On 01/23/2016 05:13 AM, Gil Tene wrote: >> .... >>>> On Jan 22, 2016, at 2:49 PM, Peter Levart >>>> wrote: >>>> >>>> Ephemeron always touches definitions of at least two consecutive >>>> strengths of reachabilities. The prototype says: >>>> >>>> *

  • An object is weakly reachable if it is neither >>>> * strongly nor softly reachable but can be reached by traversing a >>>> * weak reference or by traversing an ephemeron through it's value >>>> while >>>> * the ephemeron's key is at least weakly reachable. >>>> >>>> *
  • An object is ephemerally reachable if it is neither >>>> * strongly, softly nor weakly reachable but can be reached by >>>> traversing an >>>> * ephemeron through it's key or by traversing an ephemeron through >>>> it's value >>>> * while it's key is at most ephemerally reachable. When the >>>> ephemerons that >>>> * refer to ephemerally reachable key object are cleared, the key >>>> object becomes >>>> * eligible for finalization. >>> >>> Looking into this a bit more, I don't think the above is quite >>> right. Specifically, If an ephemeron's key is either strongly of >>> softly reachable, you want the value to remain appropriately >>> strongly/softly reachable. Without this quality, Ephemeron value >>> referents can (and will) be prematurely collected and finalized >>> while the keys are not. This (IMO) needed quality not provided by >>> the behavior you specify? >> >> This is not quite true. While ephemeron's value is weakly or even >> ephemerally-reachable, it is not finalizable, because >> ephemeraly-reachable is stronger than finaly-reachable. After >> ephemeron's key becomes ephemeraly-reachable, the ephemeron is >> cleared by GC which sets it's key *and* value to null atomically. The >> life of key and value at that moment becomes untangled. Either of >> them can have a finalizer or not and both of them will eventually be >> collected if not revived by their finalize() methods. But it can >> never happen that ephemeron's value is finalized or collected while >> it's key is still reachable through the ephemeron (while the >> ephemeron is not cleared yet). >> >> But I agree that it would be desirable for ephemeron's value to >> follow the reachability of it's key. In above specification, if the >> key is strongly reachable, the value is weakly reachable, so any >> WeakReferences or SoftReferences pointing at the Ephemeron's value >> can already be cleared while the key is still strongly reachable. >> This is arguably no different than current specification of Soft vs. >> Weak references. A SoftReference can already be cleared while its >> referent is still reachable through a WeakReference, > > We seem to agree about the cleaner behavior specification (in both of > our texts below), so the these next paragraphs are really about > arguing for why this is an important design choice if/when adding > Ephemerons to Java: > > It is true the [current] spec allows for soft references to an object > to be cleared while weak references to the same object are not: the > "determines" in "Suppose that the garbage collector determines at a > certain point in time hat an object is RRRR reachable..." part > [for RRRR = {soft, weak}] does not have to happen at the same "certain > point in time". > > However, to my knowledge all current implementations present as if > this determination is happening at the same "point in time" for all > weakly and softly reachable objects combined. Specifically [in > implementations]: if soft reachability is determined for an object at > some point in time, then weak reachability for that object is > determined at the same point in time. And the weak reachability > determination for an object depends on whether the collector chose to > clear existing soft references to that object at that same point in > time, with the appearance of the choice to clear (or not to clear) > soft references to a given object atomically affecting the > determination of it's weak reachability. Since the collector is > *required* to act on a weak determination when it is made, while it > *may* act on a soft determination when it is made, making the combined > determination at the same "point in time" eliminates an obviously > confusing situation that is not prohibited by the spec: if the > determination for weak and soft reachability was not done at the same > point in time, then an object that was softly reachable and had it's > soft references cleared and queued could later become strongly > reachable, and even softly reachable again. When reference processing > is done as a STW thing, this "combined determination" effect is a > trivial side-effect of STW. When it is done concurrently (or > incrementally?), implementations still work to maintain the appearance > of combined atomic determination of soft and weak reachability. I know > ours does. In our case, we do it because we had no desire to be the > ones to argue "I know that all implementations did this atomically > because they were STW, but the spec allows us to add this bug to your > program?". > > So in actual implementations (to my knowledge), finalization is > currently the only mechanism that can create this "strange situation" > where an object was no longer strongly reachable, had actions > triggered as a result from loss of strong reachability (i.e. actually > observed by the program as "known to not be strongly reachable"), and > later became strongly reachable again. E.g. a finalizer can propagate > a strong reference to a previously non-strongly reachable object > ('this' in the finalizer, or anything that 'this' transitively refers > and was not otherwise reachable when the finalizer was called).. This > is one of those "undesired" things that the introduction of Reference > types was meant to deal with (Reference types were introduced in 1.2, > after finalization was unfortunately already included and spec'ed. And > phantom refs were meant to allow for a cleaner form that could replace > finalization). And while the specifications of SoftReference and > WeakReference do not prohibit it, implementations are not required to > allow it, and in practice non of them do (I think), as doing so would > most likely expose some "interesting" > spec-allowed-but-extremely-surprising things/bugs that none of us want > to have to defend... > > In this context, it would be a "highly undesirable" design choice to > introduce Ephemerons in a way that would them to return a strong > reference to an object that has previously been determined to no > longer be strongly reachable. Structuring the spec to prohibit this is > a better design choice. > > To highlight the design choice here, let me describe a specific > problem scenario for which the previous (above) spec would cause > "re-strengthening" behavior that would break assumptions that are > allowed under the current spec: in the above/previously specified > behavior an object V that is known to have no finalizers, but has e.g. > 3 WeakReference objects that refer to it, can become weakly reachable > while both a key referent object K in some ephemeron E with a value > referent of V remain strongly reachable. At such a point (V is weakly > reachable, K and E are strongly reachable), the collector may > determine weak reachability for V, [atomically] clear all weak > references to V, and enqueue those weak reference objects on their > respective queues. While V is still ephemerally reachable under your > previous definition, there are no references to it anywhere other than > in ephemeron value referent fields, and weak references that did refer > to it have been cleared and queued. Since the ephemeron is still > there, and the key is still there, and the ephemeron has not been > cleared, an Ephemeron.getValue() call would create a strong reference > to an object that was previously determined to not be weakly > reachable. Re-creating a strong reference to V after the point where > weak references to V were cleared and the weak refs to it were > enqueued would be "surprising" to current weak reference based code > (the only thing that could cause this under the current spec would be > a finalizer), so allowing that (jn the spec) is likely to break all > kinds of logic that depends on currently spec'ed weak reference behaviors. > > The spec'ed behavior we seem to be agreeing on (below) would prohibit > this loophole and would [I think] maintain any reachability-based > expectations that current weak-ref based logic can make under the > current spec. Maintaining this continuity is an important design > choice for adding Ephemerons into the current set of Reference behaviors. > > And since I suspect that all implementations will continue to choose > to do the "determination" of soft and weak reachability at the same > "point in time", this will fit well with how people would build this > stuff anyway. > > Separate note: It would be separately interesting to consider > narrowing the SoftRef spec to require JVM implementations to > atomically clear all soft *and* weak references to an object at the > same time. I.e. if the garbage collector chooses to clear a soft > reference to an object that would become weakly reachable as a result, > then all weak references to that object must be [atomically] cleared > at the same time. Since I suspect that all current JVM implementations > actually adhere to this stronger requirement already, this would not > "hurt" anything or require extra work to comply with. [Anyone from > Metronome or some other non-STW reference processing implementations > want to chime in?]. > >> but for Ephemeron's value this might be confusing. The easier to >> understand conceptual model for Ephemerons might be a pair of >> (WeakReference, WeakReference) where the key has a virtual >> strong reference to the value. And this is what we get if we say that >> reachability of the value follows reachability of the key. >> >>> >>> For a correctly specified behavior, I think all strengths (from >>> strong down) need to be affected by key/value Ephemeron >>> relationships, but without adding an "ephemerally reachable" >>> strength. E.g. I think you fundamentally need something like this: >>> >>> - "An object is strongly reachable if it can be reached by >>> (a) some thread without traversing any reference objects, or by (b) >>> traversing the value of an Ephemeron whose key is strongly >>> reachable. A newly-created object is strongly reachable by the >>> thread that created it" >>> >>> - "An object is softly reachable if it is not >>> strongly reachable but can be reached by (a) traversing a soft >>> reference or by (b) traversing the value of an Ephemeron whose key >>> is softly reachable. >>> >>> - "An object is weakly reachable if it is neither strongly >>> nor softly reachable but can be reached by (a) traversing a weak >>> reference or by (b) traversing the value of an ephemeron whose key >>> is weakly reachable. >> >> ...and that's where we stop, because when we make Ephemeron just a >> special kind of WeakReference, the next thing that happens is: >> >> *

    Suppose that the garbage collector determines at a certain >> point in time >> * that an object is weakly >> * reachable. At that time it will atomically clear all weak >> references to >> * that object and all weak references to any other weakly-reachable >> objects >> * from which that object is reachable through a chain of strong and soft >> * references. At the same time it will declare all of the formerly >> * weakly-reachable objects to be finalizable. At the same time or >> at some >> * later time it will enqueue those newly-cleared weak references >> that are >> * registered with reference queues. >> >> ...where "clearing of the WeakReference" means reseting the key *and* >> value to null in case it is an Ephemeron; and >> "all weak references to some object" means Ephemerons that have that >> object as a key (but not those that only have it as a value!) in case >> of ephemerons >> >> ... >>> I still think that Ephemeron should extend WeakReference, >>> since that places already established rules and expectation on (a) >>> when it will be enqueued, (b) when the collector will clear it (when >>> the the collector encounters the key being weakly reachable), >>> and (c) that clearing of all Ephemeron *and* WeakReference instances >>> who share an identical key value is done atomically, along with (d) >>> all weak references to to any other weakly-reachable objects from >>> which that object is reachable through a chain of strong and soft >>> references. These last (c, d) parts are critically captured since an >>> Ephemeron *is a* WeakReference, and the statement in WeakReference >>> that says that "? it will atomically clear all weak references to >>> that object and all weak references to any other weakly-reachable >>> objects from which that object is reachable through a chain of >>> strong and soft references." has a clear application. >>> >>> Here are some suggested edits to the JavaDoc to go with this >>> suggested spec'ed behavior: >>> /** >>> * Ephemeron objects are a special kind of WeakReference >>> objects, which >>> * hold two referents (a key referent and a value referent) and do >>> not prevent their >>> * referents from being made finalizable, finalized, and then >>> reclaimed. >>> * In addition to the key referent, which adheres to the referent >>> behavior of a >>> * WeakReference, an ephemeron also holds a value referent whose >>> reachabiliy >>> * strength is affected by the reachability strength of the key >>> referent: >>> * The value referent of an Ephemeron instance is considered: >>> * (a) strongly reachable if the key referent of the same Ephemeron >>> * object is strongly reachable, or if the value referent is >>> otherwise strongly reachable. >>> * (b) softly reachable if it is not strongly reachable, and (i) >>> the key referent of >>> * the same Ephemeron object is softly reachable, or (ii) if the >>> value referent is otherwise >>> * softly reachable. >>> * (c) weakly reachable if it is not strongly or softly reachable, >>> and (i) the key referent of >>> * the same Ephemeron object is weakly reachable, or (ii) if the >>> value referent is otherwise >>> * weakly reachable. >>> *

    When the collector clears an Ephemeron object instance >>> (according to the rules >>> * expressed for clearing WeakReference object instances), the >>> Ephemeron instance's >>> * key referent value referent are simultaneously and atomically >>> cleared. >>> *

    By convenience, the Ephemeron's referent is also called the >>> key, and can be >>> * obtained either by invoking {@link #get} or {@link #getKey} >>> while the value >>> * can be obtained by invoking {@link #getValue} method. >>> *... >> >> >> Thanks, this is very nice. I do like this behavior more. >> >> Let me see what it takes to implement this strategy... >> >> Regards, Peter >> > From amaembo at gmail.com Sun Jan 24 11:36:51 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sun, 24 Jan 2016 17:36:51 +0600 Subject: Stream.limit parallel ordered performance Message-ID: <919184407.20160124173651@gmail.com> Hello! I'm investigating Stream.limit() performance for parallel ordered streams when no SUBSIZED optimization is performed. Consider the following code, for example: AtomicInteger counter = new AtomicInteger(); int[] result = IntStream.range(0, 1_000_000).parallel().filter(x -> true) .peek(x -> counter.incrementAndGet()).limit(10).toArray(); System.out.println(Arrays.toString(result)); System.out.println(counter.get()); How much the counter.get() would print? It changes from launch to launch, but usually within 375000..625000. This is just insane. On my 4-core system parallel stream creates 16 individual tasks. I expect that every individual task should consume no more than 10 elements, so in total no more than 160 elements should be consumed in this case. Here's a patch which addresses this issue: http://cr.openjdk.java.net/~tvaleev/patches/limit/limit-patch.txt In the limit case non-root leaf tasks may switch to copyIntoWithCancel to control the count of consumed elements and do not consume more than necessary. This change seems to fix the issue addressed in comment (at least partially): // @@@ OOMEs will occur for LongStream.range(0, Long.MAX_VALUE).filter(i -> true).limit(n) // regardless of the value of n // Need to adjust the target size of splitting for the // SliceTask from say (size / k) to say min(size / k, 1 << 14) // This will limit the size of the buffers created at the leaf nodes // cancellation will be more aggressive cancelling later tasks // if the target slice size has been reached from a given task, // cancellation should also clear local results if any I checked with the following code: for(int n : new int[] {10, 100, 1000, 5000, 10000, 50000, 100000, 500000, 1000000}) { System.out.println(n); long[] arr = LongStream.range(0, Long.MAX_VALUE).filter(i -> true).parallel().limit(n).toArray(); long[] ref = LongStream.range(0, n).toArray(); System.out.println(Arrays.equals(arr, ref)); } It works correctly after applying my patch (while dies with OOME without patch, as comment suggests). Currently existing unit tests also pass with my patch. However I'm very new in the internals of parallel stream processing, so it's possible that I'm missing something. Please review! If this looks reasonable I will log an issue and write new test cases. Thank you in advance, With best regards, Tagir Valeev. From forax at univ-mlv.fr Sun Jan 24 15:36:55 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Sun, 24 Jan 2016 16:36:55 +0100 (CET) Subject: RFR: 8147755: ASM should create correct constant tag for invokestatic on handle point to interface static method In-Reply-To: <56A28F6E.30307@oracle.com> References: <56A28F6E.30307@oracle.com> Message-ID: <774410875.2198558.1453649815643.JavaMail.zimbra@u-pem.fr> Hi Yumin, I'm one of the developers of ASM, I've pushed a new versions of ASM (the future 5.1) in the trunk that should let you create constant method handles on an interface, when i have updated ASM for Java 8, i've overlooked that case, sorry for that. The code is mostly similar to your patch but it does the full roundtrip, being able not only to encode constant method handles on interface but also to decode them. Kumar is in CC, i believe, he is the one that sync the private copy of ASM used by the JDK with the SVN trunk. regards, R?mi ----- Mail original ----- > De: "Yumin Qi" > ?: core-libs-dev at openjdk.java.net, "CHRIS.HEGARTY" , "harold seigel" > , "Lois Foltan" , "Vladimir Ivanov" > Envoy?: Vendredi 22 Janvier 2016 21:22:06 > Objet: RFR: 8147755: ASM should create correct constant tag for invokestatic on handle point to interface static > method > > Hi, All > > Can I have a review for > bug: https://bugs.openjdk.java.net/browse/JDK-8147755 > Webrev: http://cr.openjdk.java.net/~minqi/8147755/webrev-01/ > > Summary: When fix bug 8145148 to follow: > > JVMS-5.4.3.3 Method Resolution: > " If C is an interface, method resolution throws an > IncompatibleClassChangeError." > JVMS-5.4.3.4 Interface Method Resolution: > "If C is not an interface, interface method resolution throws an > IncompatibleClassChangeError" > > In defmeth(default method) tests, a MethodHandle pointing to > interface static method will violate the spec. Since such a handle in > asm will generate a tag of Methodref in constantpool for the call site, > with the fix of 8145148, it requires a tag of InterfaceMethodref. > > Fix by create a new constructor of Handle with extra boolean to > indicate if the handle points to interface static method. > > Tests: A test case and resulted constantpool data attached. > The fix will not affect existing code. > Also tested with fixed 8145148 and revised defmeth > with the new version Handle and passed. > > Thanks > Yumin > > > From uschindler at apache.org Sun Jan 24 15:39:16 2016 From: uschindler at apache.org (Uwe Schindler) Date: Sun, 24 Jan 2016 16:39:16 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <7FA611E5-C261-4809-9F3D-6009B43144F3@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3EA89.7080804@oracle.com> <006401d15629$0294f760$07bee620$@apache.org> <7FA611E5-C261-4809-9F3D-6009B43144F3@oracle.com> Message-ID: <004801d156bd$6459a7e0$2d0cf7a0$@apache.org> Hi all, hi Chris, Thanks for the catch up! As we are continuously testing EA builds of OpenJDK with Apache Lucene, we will see the issues caused by this commit asap. So I would maybe delay our testing a bit, until the proposed changes of Alan are also committed and part of EA builds. Is there already an issue about that? I opened an issue in the Apache Lucene tracker: https://issues.apache.org/jira/browse/LUCENE-6989 I hope, we will find a good solution! I had some other thoughts about the issue: Basically the forceful unmapping is only needed because the Grabage Collector never ever tries to finalize and GC the very small MappedByteBuffer instances, especially if they were living for longer time. But they are sitting on heavy resources, so maybe a solution to solve this would be: - Add some special annotation in the JDK that is respected by all Garbage Collectors and makes them prefer those "heavy" instances. Something like @jdk.internal.Heavy - Maybe add some close() method to MappedByteBuffer, but just make it a candidate for earlier Grabage Collection instead of doing risky unmapping. A solution may be to add it to some special GC queue or whatever. I have no idea if that?s possible, just tell GC: "I am done with this object, discard it asap if its unreachable!" Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: Chris Hegarty [mailto:chris.hegarty at oracle.com] > Sent: Saturday, January 23, 2016 11:52 PM > To: Uwe Schindler > Cc: Alan Bateman ; core-libs-dev dev at openjdk.java.net> > Subject: Re: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref > > Just catching up? > > I see no objection to the proposed patch, but the obvious concern ( that > is highlighted in JEP 260 ), about the digging into the private internal > details of NIO buffers. > > I think Alan summarized the issues around providing a public API for > releasing/unmapping native memory very well, and if this does not > make it for JDK 9, then Alan?s alternative ( implements Runnable ) may > be a better interim solution than the current use of reflection. > > -Chris. > > > On 23 Jan 2016, at 21:57, Uwe Schindler wrote: > > > > Hi, > > > > Alan Bateman [mailto:Alan.Bateman at oracle.com] wrote: > >> On 23/01/2016 19:23, Uwe Schindler wrote: > >>> : > >>> > >>> What is the replacement for the following code (see marked > BufferCleaner > >> impl, which is our abstraction): https://goo.gl/DGYZZj > >>> > >>> The main reason why sun.misc.Cleaner was on the critical API list is NOT > the > >> case that somebody wanted to implement their own cleaner as > replacement > >> for finalization. The main reason (and this is *not* solved by the new APIs > in > >> java.lang.ref.Cleaner) is to *force* unmapping of direct and mmapped > byte > >> buffers. > >> JEP 260 was updated recently to move this to an open issue and I think > >> the text captures the issue correctly. Part of it that we can't have > >> types in java.base depending on types in other modules and this is why > >> Buffers can't have fields of type sun.misc.Cleaner or any type in sun.misc. > >> > >> As has come several times, there are huge security and reliability > >> issues that arise when releasing or unmapping memory that is still > >> accessible via reachable objects. Proposed solutions over the years have > >> traded performance or were limited to platforms where we could > >> atomically remap. JDK 9 may be the time to try yet again or else just > >> introduce a JDK-specific API to unmap that comes with a warning in 96pt > >> font. It might have to be disabled completely when running with a > >> security manager. Alternatively the burden of the current hack could be > >> reduced by having Cleaner implement Runnable with a run method that > >> invokes clean. That would avoid needing to cast to a JDK-internal type > >> and so avoiding needing to use -XaddExports to break encapsulation. > > > > We know this problem. Because of this, e.g. Elasticsearch runs with a > security manager and only allows lucene-core.jar access to do this reflection > on sun.misc package (using the correct permission, only given to JAR file). > > > > I was about to suggest the same thing: Maybe add a public API to byte > buffer, that is documented to throw SecurityException. Add a new > Runtime/WhateverPermission that can be enabled to work around that (e.g., > Elasticsearch would do this and allow this Permission only to lucene-core.jar). > In addition add red and blinking warnings to this method :-) > > > > In any case, a possible solution was proposed by Andrew Haley (he had > some ideas in the past, I think we brought this up the last time a few months > ago) with a small slowdown, but still providing better ByteBuffer > performance than in Java 7 or Java 8. > > > > I would hope for a solution that installs a SIGSEGV signal handler in the JDK, > that translates any completely illegal access that would otherwise lead to a > crash of the JDK to some Java Exception. The security issues you also > mention do not apply for us (e.g., reading memory from another web > application in the same JVM), so a new permission for SecurityManager > would also help here. > > > > Uwe > > From amaembo at gmail.com Sun Jan 24 16:04:54 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sun, 24 Jan 2016 22:04:54 +0600 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <56A3F349.5070500@oracle.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> Message-ID: <19110132004.20160124220454@gmail.com> Hello! Thanks for sponsoring! If something is still necessary from my side regarding this issue, feel free to ask. With best regards, Tagir Valeev. RR> Hi, RR> Looks good. RR> I would want to verify that it is defined such that if it later needs to be RR> abstracted up to ChronoLocalDate and apply to compatible Chronologies RR> and respective local dates such as JapaneseDate there is no conflict. RR> It should not be an issue since Period implements ChronoPeriod. RR> In the respective implementations, the estimation/computation of the RR> number of steps RR> would need to depend on the Chronology's definition of months. RR> I can sponsor this enhancement. RR> Thanks, Roger RR> On 1/20/16 10:15 AM, Stephen Colebourne wrote: >> I'm happy with the logic and specification of this proposal. I think it >> will be a useful addition. >> >> I'll let the Oracle team chime in to do a further review. >> >> thanks >> Stephen >> >> >> >> On 16 January 2016 at 13:31, Tagir F. Valeev wrote: >> >>> Hello! >>> >>> Thanks for review! Here's the updated patch: >>> http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ >>> >>> SC> The docs say that if the end date is before the start date, the >>> SC> stream is empty. While I can see just about why LongStream.range() >>> SC> works that way, I don't think this API should. The minimum is an >>> SC> exception, but it would be easy to support negative in the >>> SC> days-only case or the months-only case. The problem is where there >>> SC> are both months/years and days and those should be exceptions. >>> >>> Now datesUntil(endExclusive) throws an IllegalArgumentException if end >>> date is before start date. >>> >>> datesUntil(endExclusive, step) supports negative periods. It throws >>> IllegalArgumentException if: >>> - step is zero >>> - step.toTotalMonths() and step.getDays() both non-zero and have >>> opposite sign >>> - step is negative and end date is after start date >>> - step is positive and end date is before start date >>> >>> Otherwise it works correctly: you can use >>> LocalDate.of(2016, 1, 1) >>> .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); >>> >>> Steps like Period.of(-1, -1, -1) are also supported. >>> >>> SC> The single-arg implementation uses plusDays() with an >>> SC> incrementing number. When the performance patch goes in, the >>> SC> proposed streaming implementation will be optimal for small >>> SC> streams but less optimal for large ones. The fastest way to loop >>> SC> over a list of dates would be to manually generate them by >>> SC> incrementing the day until it exceeds the length of month, and so >>> SC> on. However, this would be serial. >>> >>> As I understand, plusDays performance patch is already pushed. >>> >>> It's possible to write custom Spliterator which would use >>> previous.plusDays(1) in tryAdvance() and jump (via >>> LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is >>> requested. However this adds at least one additional class and more >>> complexity. I guess, such optimization can be considered later as >>> separate issue when API is stabilized. >>> >>> SC> As such, I think the best way to write this, taking account of >>> SC> how plusDays() is implemented, is as follows: >>> >>> SC> LongStream.range(start.toEpochDay(), >>> SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); >>> >>> Ok, now it's done this way. >>> >>> SC> In the period-based method, it would be best to capture the case >>> SC> where totalMonths == 0 and days > 0 and forward to another method >>> SC> that ignores months. That private method can share implementation >>> SC> with the public single-arg method (passing in 1). >>> >>> This case still more complex than one-day case as it requires division >>> and multiplication. Thus I'd keep these case separately. However I >>> simplified "months = 0" path using ofEpochDay, now it should be >>> faster. >>> >>> SC> The docs for the period-based method should specify the strategy >>> SC> that produces the results (in abstract terms). This needs to cover >>> SC> that the result is equivalent to always adding to the start period >>> SC> a multiple of the period. >>> >>> I added some clarifications, please check. >>> >>> SC> Some nits: >>> SC> I prefer to avoid @link in the first sentence. Just using 'stream' >>> would be sufficient. >>> >>> Done. >>> >>> SC> The first sentence should be a summary. In this case it probably has a >>> bit too much detail. >>> >>> Done. >>> >>> SC> The @return has 'values' on a new line when it could be on the same >>> line. >>> >>> I set now line length = 100 characters in my IDE. Is it ok? >>> >>> SC> If statements need braces. >>> >>> Done. >>> >>> With best regards, >>> Tagir Valeev. >>> >>> From peter.levart at gmail.com Sun Jan 24 17:10:32 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 24 Jan 2016 18:10:32 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A3E320.2070004@redhat.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> Message-ID: <56A50588.20000@gmail.com> Hi, I had an idea recently on how to expedite the collection of an object. It is simple - just don't let it live long. Here's a concept prototype: http://cr.openjdk.java.net/~plevart/misc/CloseableMemory/CloseableMemory.java The overhead of the check in access methods (getByte()/setByte()) amounts to one volatile read of an oop variable that changes once per say 5 to 10 seconds. That's the period a special guard object is alive. It's reachability is tracked by the GC and extends to the end of each access method (using Reference.reachabilityFence). Every few seconds, the guard object is changed with new fresh one so that the chance of the guard and its tracking Cleaner being promoted to old generation is very low. Could something like that enable a low-overhead CloseableMappedByteBuffer? Regards, Peter On 01/23/2016 09:31 PM, Andrew Haley wrote: > On 23/01/16 20:01, Uwe Schindler wrote: > >> It depends how small! If the speed is still somewhere between Java 8 >> ByteBuffer performance and the recent Hotspot improvements in Java >> 9, I agree with trying it out. But some volatile memory access on >> every access is a no-go. The code around ByteBufferIndexInput in >> Lucene is the most performance-critical critical code, because on >> every search query or sorting there is all the work happening in >> there (millions of iterations with positional ByteBuffer.get* >> calls). As ByteBuffers are limited to 2 GiB, we also need lots of >> hairy code to work around that limitation! > Yes, I see that code. It would be helpful if there were a > self-contained but realistic benchmark using that code. That way, > some simple experiments would allow changes to be measured. > >> If you look at ByteBufferIndexInput's code you will see that we >> simply do stuff like trying to read from one bytebuffer and only if >> we catch an BufferUnderflowException we fall back to handling buffer >> switches: Instead of checking bounds on every access, we have >> fallback code only happening on exceptions. E.g. if you are 3 bytes >> before end of one buffer slice and read a long, it will throw >> BufferUnderflow. When this happens the code will fall back to read >> byte by byte from 2 different buffers and reassemble the long): > I'm surprised you don't see painful deoptimization traps when that > happens. I suppose it's rare enough that you don't care. There's a > new group of methods in JDK9 called Objects.checkIndex() which are > intended to provide a very efficient way to do bounds checks. It might > be interesting to see if they work well with ByteBufferIndexInput: > that's an important use case. > > BTW, does anyone here know why we don't have humongous ByteBuffers > with a long index? > > Andrew. From gil at azul.com Sun Jan 24 17:43:01 2016 From: gil at azul.com (Gil Tene) Date: Sun, 24 Jan 2016 17:43:01 +0000 Subject: Ephemerons In-Reply-To: <56A4AD09.1010901@gmail.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> <56A37CB4.3000407@gmail.com> <8154B857-931F-4BED-872A-36892E050CCF@azul.com> <56A4AD09.1010901@gmail.com> Message-ID: <5AFD362E-43AD-4234-8D6E-158AB1BA5340@azul.com> A note on implementation possibilities: If I read the implementation correctly, a "weakness" of the current implementation approach for making sure value-referents (and their transitively reachable) objects are kept alive if key referents are alive is that it requires multiple passes through the discovered Ephemeron list, with the passes terminating when the list stabilizes. While I think that this is sound (in the sense that it will work), it carries a potentially high cost when large sets of Ephemerons exist in the heap. E.g. if the Ephemerons are linked in a k-v list (where the value referent of one ephemeron is the key of another, in a chain), as in your code example, there is an N^2 scanning thing going on. And e.g. if a large set of ephemeron keys become weakly reachable in a single cycle (e.g. a large cache was discarded) while other ephemeron participate in some linked list relationship, the entire list of [stably] weak-keyed ephemerons has to be traversed in each pass (in case one of them has become live in a previous pass). I'd worry that these computational complexity issues could become prohibitive enough in GC cost that you there would be significant resistance to their adoption. Note that in comparison (to my understanding), current ref processing work involved in GC handling soft/weak/final/phantom refs remains linear to the number of refs, and does not have an O(N^2) component. I believe that there is a relatively simple way to bring Ephemeron processing to O(N) by establishing reverse mapping during the scan (the below description assumes STW during the scan): 1. Start ref processing with no reverse mapping table established. 2. During ref processing, establish an EphemeronKeyReverseMapping table (logically a Map>) which would map individual heap addresses to ephemerons who's key referent points to those addresses. 3. Note that since each heap address can show up in multiple key referents, the map needs to return a (potentially empty) list of Ephemerons who's keys refer to the address. 4. Specifically, starting with an empty list, and for each discovered Ephemeron, add a reverse-mapping entry to the EphemeronKeyReverseMapping, mapping from the key referent address to the Ephemeron. 5. During Ephemeron processing (under the case where the referent is found to be alive and the ephemeron then needs to keep the value referent alive) mark down the value referent path using a special ephemeron_keep_alive OopClosure (or a mode flag that affects the normal keep_alive behavior) which, when reaching a not-yet-marked-live object [in addition to marking it live and traversing it as keep_alive would normally do] would look up the object's address in the EphemeronKeyReverseMapping to get a list of Ephemerons to traverse, and traverse each of the mapped Ephemerons' value referent with the same ephemeron_keep_alive closure. Note: doing reverse-mapping lookups on each not-yet-marked object in a keep_alive closure will add cost, which is why this ephemeron_keep_alive pass should probably be done after regular keep_alive passes have had their chance to mark objects live. This way only paths that become newly-live via ephemeron processing are subject to the extra reverse-mapping-lookip cost. While I haven't been poking at this too long to see if it has holes, I think it can produce a reliable result, and is O(N) on the count of Ephemerons. ? Gil. > On Jan 24, 2016, at 2:52 AM, Peter Levart wrote: > > Hi Gil, > > I totally agree with your assessment. We should not introduce another way of reviving the almost collectable objects and I fully support tightening the specification so that soft and weak references to the same referent and to other referents from which this referent is reachable are required to be cleared together atomically. > > I modified the prototype to (hopefully) adhere to this new Ephemeron specification that Gil and I agreed upon. Anyone interested in experimenting can find it here: > > http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.02/ > http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.hotspot.02/ > > It is rebased to current tip of jdk9-dev repositories (after the bulk of merges for jdk-9+102), but still contains the change to remove the Cleaner reference type as it has not yet managed to get in... > > I have also added a test that is a start for verifying the functionality. > > Regards, Peter > > On 01/23/2016 07:25 PM, Gil Tene wrote: >> >>> On Jan 23, 2016, at 5:14 AM, Peter Levart < peter.levart at gmail.com > wrote: >>> >>> Hi Gil, it's good to have this discussion. See comments inline... >>> >>> On 01/23/2016 05:13 AM, Gil Tene wrote: >>> .... >>>>> On Jan 22, 2016, at 2:49 PM, Peter Levart > wrote: >>>>> >>>>> Ephemeron always touches definitions of at least two consecutive strengths of reachabilities. The prototype says: >>>>> >>>>> *

  • An object is weakly reachable if it is neither >>>>> * strongly nor softly reachable but can be reached by traversing a >>>>> * weak reference or by traversing an ephemeron through it's value while >>>>> * the ephemeron's key is at least weakly reachable. >>>>> >>>>> *
  • An object is ephemerally reachable if it is neither >>>>> * strongly, softly nor weakly reachable but can be reached by traversing an >>>>> * ephemeron through it's key or by traversing an ephemeron through it's value >>>>> * while it's key is at most ephemerally reachable. When the ephemerons that >>>>> * refer to ephemerally reachable key object are cleared, the key object becomes >>>>> * eligible for finalization. >>>> >>>> Looking into this a bit more, I don't think the above is quite right. Specifically, If an ephemeron's key is either strongly of softly reachable, you want the value to remain appropriately strongly/softly reachable. Without this quality, Ephemeron value referents can (and will) be prematurely collected and finalized while the keys are not. This (IMO) needed quality not provided by the behavior you specify? >>> >>> This is not quite true. While ephemeron's value is weakly or even ephemerally-reachable, it is not finalizable, because ephemeraly-reachable is stronger than finaly-reachable. After ephemeron's key becomes ephemeraly-reachable, the ephemeron is cleared by GC which sets it's key *and* value to null atomically. The life of key and value at that moment becomes untangled. Either of them can have a finalizer or not and both of them will eventually be collected if not revived by their finalize() methods. But it can never happen that ephemeron's value is finalized or collected while it's key is still reachable through the ephemeron (while the ephemeron is not cleared yet). >>> >>> But I agree that it would be desirable for ephemeron's value to follow the reachability of it's key. In above specification, if the key is strongly reachable, the value is weakly reachable, so any WeakReferences or SoftReferences pointing at the Ephemeron's value can already be cleared while the key is still strongly reachable. This is arguably no different than current specification of Soft vs. Weak references. A SoftReference can already be cleared while its referent is still reachable through a WeakReference, >> >> We seem to agree about the cleaner behavior specification (in both of our texts below), so the these next paragraphs are really about arguing for why this is an important design choice if/when adding Ephemerons to Java: >> >> It is true the [current] spec allows for soft references to an object to be cleared while weak references to the same object are not: the "determines" in "Suppose that the garbage collector determines at a certain point in time hat an object is RRRR reachable..." part [for RRRR = {soft, weak}] does not have to happen at the same "certain point in time". >> >> However, to my knowledge all current implementations present as if this determination is happening at the same "point in time" for all weakly and softly reachable objects combined. Specifically [in implementations]: if soft reachability is determined for an object at some point in time, then weak reachability for that object is determined at the same point in time. And the weak reachability determination for an object depends on whether the collector chose to clear existing soft references to that object at that same point in time, with the appearance of the choice to clear (or not to clear) soft references to a given object atomically affecting the determination of it's weak reachability. Since the collector is *required* to act on a weak determination when it is made, while it *may* act on a soft determination when it is made, making the combined determination at the same "point in time" eliminates an obviously confusing situation that is not prohibited by the spec: if the determination for weak and soft reachability was not done at the same point in time, then an object that was softly reachable and had it's soft references cleared and queued could later become strongly reachable, and even softly reachable again. When reference processing is done as a STW thing, this "combined determination" effect is a trivial side-effect of STW. When it is done concurrently (or incrementally?), implementations still work to maintain the appearance of combined atomic determination of soft and weak reachability. I know ours does. In our case, we do it because we had no desire to be the ones to argue "I know that all implementations did this atomically because they were STW, but the spec allows us to add this bug to your program?". >> >> So in actual implementations (to my knowledge), finalization is currently the only mechanism that can create this "strange situation" where an object was no longer strongly reachable, had actions triggered as a result from loss of strong reachability (i.e. actually observed by the program as "known to not be strongly reachable"), and later became strongly reachable again. E.g. a finalizer can propagate a strong reference to a previously non-strongly reachable object ('this' in the finalizer, or anything that 'this' transitively refers and was not otherwise reachable when the finalizer was called).. This is one of those "undesired" things that the introduction of Reference types was meant to deal with (Reference types were introduced in 1.2, after finalization was unfortunately already included and spec'ed. And phantom refs were meant to allow for a cleaner form that could replace finalization). And while the specifications of SoftReference and WeakReference do not prohibit it, implementations are not required to allow it, and in practice non of them do (I think), as doing so would most likely expose some "interesting" spec-allowed-but-extremely-surprising things/bugs that none of us want to have to defend... >> >> In this context, it would be a "highly undesirable" design choice to introduce Ephemerons in a way that would them to return a strong reference to an object that has previously been determined to no longer be strongly reachable. Structuring the spec to prohibit this is a better design choice. >> >> To highlight the design choice here, let me describe a specific problem scenario for which the previous (above) spec would cause "re-strengthening" behavior that would break assumptions that are allowed under the current spec: in the above/previously specified behavior an object V that is known to have no finalizers, but has e.g. 3 WeakReference objects that refer to it, can become weakly reachable while both a key referent object K in some ephemeron E with a value referent of V remain strongly reachable. At such a point (V is weakly reachable, K and E are strongly reachable), the collector may determine weak reachability for V, [atomically] clear all weak references to V, and enqueue those weak reference objects on their respective queues. While V is still ephemerally reachable under your previous definition, there are no references to it anywhere other than in ephemeron value referent fields, and weak references that did refer to it have been cleared and queued. Since the ephemeron is still there, and the key is still there, and the ephemeron has not been cleared, an Ephemeron.getValue() call would create a strong reference to an object that was previously determined to not be weakly reachable. Re-creating a strong reference to V after the point where weak references to V were cleared and the weak refs to it were enqueued would be "surprising" to current weak reference based code (the only thing that could cause this under the current spec would be a finalizer), so allowing that (jn the spec) is likely to break all kinds of logic that depends on currently spec'ed weak reference behaviors. >> >> The spec'ed behavior we seem to be agreeing on (below) would prohibit this loophole and would [I think] maintain any reachability-based expectations that current weak-ref based logic can make under the current spec. Maintaining this continuity is an important design choice for adding Ephemerons into the current set of Reference behaviors. >> >> And since I suspect that all implementations will continue to choose to do the "determination" of soft and weak reachability at the same "point in time", this will fit well with how people would build this stuff anyway. >> >> Separate note: It would be separately interesting to consider narrowing the SoftRef spec to require JVM implementations to atomically clear all soft *and* weak references to an object at the same time. I.e. if the garbage collector chooses to clear a soft reference to an object that would become weakly reachable as a result, then all weak references to that object must be [atomically] cleared at the same time. Since I suspect that all current JVM implementations actually adhere to this stronger requirement already, this would not "hurt" anything or require extra work to comply with. [Anyone from Metronome or some other non-STW reference processing implementations want to chime in?]. >> >>> but for Ephemeron's value this might be confusing. The easier to understand conceptual model for Ephemerons might be a pair of (WeakReference, WeakReference) where the key has a virtual strong reference to the value. And this is what we get if we say that reachability of the value follows reachability of the key. >>> >>>> >>>> >>>> For a correctly specified behavior, I think all strengths (from strong down) need to be affected by key/value Ephemeron relationships, but without adding an "ephemerally reachable" strength. E.g. I think you fundamentally need something like this: >>>> >>>> - "An object is strongly reachable if it can be reached by (a) some thread without traversing any reference objects, or by (b) traversing the value of an Ephemeron whose key is strongly reachable. A newly-created object is strongly reachable by the thread that created it" >>>> >>>> - "An object is softly reachable if it is not strongly reachable but can be reached by (a) traversing a soft reference or by (b) traversing the value of an Ephemeron whose key is softly reachable. >>>> >>>> - "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by (a) traversing a weak reference or by (b) traversing the value of an ephemeron whose key is weakly reachable. >>> >>> ...and that's where we stop, because when we make Ephemeron just a special kind of WeakReference, the next thing that happens is: >>> >>> *

    Suppose that the garbage collector determines at a certain point in time >>> * that an object is weakly >>> * reachable. At that time it will atomically clear all weak references to >>> * that object and all weak references to any other weakly-reachable objects >>> * from which that object is reachable through a chain of strong and soft >>> * references. At the same time it will declare all of the formerly >>> * weakly-reachable objects to be finalizable. At the same time or at some >>> * later time it will enqueue those newly-cleared weak references that are >>> * registered with reference queues. >>> >>> ...where "clearing of the WeakReference" means reseting the key *and* value to null in case it is an Ephemeron; and >>> "all weak references to some object" means Ephemerons that have that object as a key (but not those that only have it as a value!) in case of ephemerons >>> >>> ... >>>> I still think that Ephemeron should extend WeakReference, since that places already established rules and expectation on (a) when it will be enqueued, (b) when the collector will clear it (when the the collector encounters the key being weakly reachable), and (c) that clearing of all Ephemeron *and* WeakReference instances who share an identical key value is done atomically, along with (d) all weak references to to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. These last (c, d) parts are critically captured since an Ephemeron *is a* WeakReference, and the statement in WeakReference that says that "? it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references." has a clear application. >>>> >>>> Here are some suggested edits to the JavaDoc to go with this suggested spec'ed behavior: >>>> /** >>>> * Ephemeron objects are a special kind of WeakReference objects, which >>>> * hold two referents (a key referent and a value referent) and do not prevent their >>>> * referents from being made finalizable, finalized, and then reclaimed. >>>> * In addition to the key referent, which adheres to the referent behavior of a >>>> * WeakReference, an ephemeron also holds a value referent whose reachabiliy >>>> * strength is affected by the reachability strength of the key referent: >>>> * The value referent of an Ephemeron instance is considered: >>>> * (a) strongly reachable if the key referent of the same Ephemeron >>>> * object is strongly reachable, or if the value referent is otherwise strongly reachable. >>>> * (b) softly reachable if it is not strongly reachable, and (i) the key referent of >>>> * the same Ephemeron object is softly reachable, or (ii) if the value referent is otherwise >>>> * softly reachable. >>>> * (c) weakly reachable if it is not strongly or softly reachable, and (i) the key referent of >>>> * the same Ephemeron object is weakly reachable, or (ii) if the value referent is otherwise >>>> * weakly reachable. >>>> *

    When the collector clears an Ephemeron object instance (according to the rules >>>> * expressed for clearing WeakReference object instances), the Ephemeron instance's >>>> * key referent value referent are simultaneously and atomically cleared. >>>> *

    By convenience, the Ephemeron's referent is also called the key, and can be >>>> * obtained either by invoking {@link #get} or {@link #getKey} while the value >>>> * can be obtained by invoking {@link #getValue} method. >>>> *... >>> >>> >>> Thanks, this is very nice. I do like this behavior more. >>> >>> Let me see what it takes to implement this strategy... >>> >>> Regards, Peter >>> >> > From peter.levart at gmail.com Sun Jan 24 20:14:18 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 24 Jan 2016 21:14:18 +0100 Subject: Ephemerons In-Reply-To: <5AFD362E-43AD-4234-8D6E-158AB1BA5340@azul.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> <56A37CB4.3000407@gmail.com> <8154B857-931F-4BED-872A-36892E050CCF@azul.com> <56A4AD09.1010901@gmail.com> <5AFD362E-43AD-4234-8D6E-158AB1BA5340@azul.com> Message-ID: <56A5309A.20505@gmail.com> Hi Gil, The algorithm complexity has been on my mind too and I've been thinking of how to maintain a key (or it's address) -> ephemeron mapping most efficiently. As I have written in a reply to Mandy, if we say that individual phase2 of processing of the normal References has complexity O(N) where N is the number of discovered references, then the equivalent phase2 processing of Ephemerons has comparable complexity O(N*d) where N is the number of discovered Ephemerons and d is the maximum number of "hops" in object graph where a "hop" is defined as a point in chain where the value of some ephemeron refers directly or indirectly to the key of some other ephemeron and the value of that other ephemeron continues the chain. Let's hope that in practical data structures, d is not large. The maximum value of d is N, as in one of the tests. The algorithm in the prototype tries to reduce the number of outer iterations to much less than 'd' in many situations by alternating the direction of scanning in each pass. The value(ephemeronX) -> key(ephemeronY) links could be arranged in a zig-zag pattern relative to the order of discovered references in the list though which would be the worst case for the algorithm, but I think the chances of that happening in a real-world data structure are low. But we have to think of possible denial-of-service attacks too, so I agree that worst-case scenario has to be improved. Presently, Reference objects are discovered and hooked on single-linked _discoveredXXX lists (using Reference.discovered field to point to the next in list). Each reference type has it's own set of lists. Multiple lists in a set enable parallel (concurrent?) discovery without contention (a list per thread). So I was thinking that one possibility would be for ephemerons to have a bigger table of discovered lists - not just one per discovery thread, but quite bigger and that the index of a list to which discovered ephemeron is added would be computed from the ephemeron's key. A classical hash-table with buckets. Big enough table would minimize contention when discovery is parallelized and enable the following algorithm (which could also be parallelized): Let each bucket (each head of the list) have 2 boolean flags: include & pending. At the start, set include flags of all buckets to true and then enter the loop: do { reset pending flags of all buckets to false; for each bucket with (include == true), do { scan the ephemerons and for those with live keys, mark value and transitive closure as live. while marking the value and transitive closure as live, for each object that was newly marked alive, compute the index into the ephemeron buckets as though such object was ephemeron's key and set the pending flag of that bucket. } set the include flags of all buckets from the pending flags of the buckets and count # of pending buckets } while (# of pending buckets > 0); Hm, .... Regards, Peter On 01/24/2016 06:43 PM, Gil Tene wrote: > A note on implementation possibilities: > > If I read the implementation correctly, a "weakness" of the current > implementation approach for making sure value-referents (and their > transitively reachable) objects are kept alive if key referents are > alive is that it requires multiple passes through the discovered > Ephemeron list, with the passes terminating when the list stabilizes. > While I think that this is sound (in the sense that it will work), it > carries a potentially high cost when large sets of Ephemerons exist in > the heap. E.g. if the Ephemerons are linked in a k-v list (where the > value referent of one ephemeron is the key of another, in a chain), as > in your code example, there is an N^2 scanning thing going on. And > e.g. if a large set of ephemeron keys become weakly reachable in a > single cycle (e.g. a large cache was discarded) while other ephemeron > participate in some linked list relationship, the entire list of > [stably] weak-keyed ephemerons has to be traversed in each pass (in > case one of them has become live in a previous pass). I'd worry that > these computational complexity issues could become prohibitive enough > in GC cost that you there would be significant resistance to their > adoption. > > Note that in comparison (to my understanding), current ref processing > work involved in GC handling soft/weak/final/phantom refs remains > linear to the number of refs, and does not have an O(N^2) component. > > I believe that there is a relatively simple way to bring Ephemeron > processing to O(N) by establishing reverse mapping during the scan > (the below description assumes STW during the scan): > > 1. Start ref processing with no reverse mapping table established. > > 2. During ref processing, establish an EphemeronKeyReverseMapping > table (logically a Map>) which would map > individual heap addresses to ephemerons who's key referent points to > those addresses. > > 3. Note that since each heap address can show up in multiple key > referents, the map needs to return a (potentially empty) list of > Ephemerons who's keys refer to the address. > > 4. Specifically, starting with an empty list, and for each discovered > Ephemeron, add a reverse-mapping entry to the > EphemeronKeyReverseMapping, mapping from the key referent address to > the Ephemeron. > > 5. During Ephemeron processing (under the case where the referent is > found to be alive and the ephemeron then needs to keep the value > referent alive) mark down the value referent path using a > special ephemeron_keep_alive OopClosure (or a mode flag that affects > the normal keep_alive behavior) which, when reaching a > not-yet-marked-live object [in addition to marking it live and > traversing it as keep_alive would normally do] would look up the > object's address in the EphemeronKeyReverseMapping to get a list of > Ephemerons to traverse, and traverse each of the mapped Ephemerons' > value referent with the same ephemeron_keep_alive closure. > Note: doing reverse-mapping lookups on each not-yet-marked object in a > keep_alive closure will add cost, which is why > this ephemeron_keep_alive pass should probably be done after regular > keep_alive passes have had their chance to mark objects live. This way > only paths that become newly-live via ephemeron processing are subject > to the extra reverse-mapping-lookip cost. > > While I haven't been poking at this too long to see if it has holes, I > think it can produce a reliable result, and is O(N) on the count of > Ephemerons. > > ? Gil. > > >> On Jan 24, 2016, at 2:52 AM, Peter Levart > > wrote: >> >> Hi Gil, >> >> I totally agree with your assessment. We should not introduce another >> way of reviving the almost collectable objects and I fully support >> tightening the specification so that soft and weak references to the >> same referent and to other referents from which this referent is >> reachable are required to be cleared together atomically. >> >> I modified the prototype to (hopefully) adhere to this new Ephemeron >> specification that Gil and I agreed upon. Anyone interested in >> experimenting can find it here: >> >> http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.02/ >> http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.hotspot.02/ >> >> It is rebased to current tip of jdk9-dev repositories (after the bulk >> of merges for jdk-9+102), but still contains the change to remove the >> Cleaner reference type as it has not yet managed to get in... >> >> I have also added a test that is a start for verifying the functionality. >> >> Regards, Peter >> >> On 01/23/2016 07:25 PM, Gil Tene wrote: >>> >>>> On Jan 23, 2016, at 5:14 AM, Peter Levart >>>> wrote: >>>> >>>> Hi Gil, it's good to have this discussion. See comments inline... >>>> >>>> On 01/23/2016 05:13 AM, Gil Tene wrote: >>>> .... >>>>>> On Jan 22, 2016, at 2:49 PM, Peter Levart >>>>>> wrote: >>>>>> >>>>>> Ephemeron always touches definitions of at least two consecutive >>>>>> strengths of reachabilities. The prototype says: >>>>>> >>>>>> *

  • An object is weakly reachable if it is neither >>>>>> * strongly nor softly reachable but can be reached by traversing a >>>>>> * weak reference or by traversing an ephemeron through it's >>>>>> value while >>>>>> * the ephemeron's key is at least weakly reachable. >>>>>> >>>>>> *
  • An object is ephemerally reachable if it is neither >>>>>> * strongly, softly nor weakly reachable but can be reached by >>>>>> traversing an >>>>>> * ephemeron through it's key or by traversing an ephemeron >>>>>> through it's value >>>>>> * while it's key is at most ephemerally reachable. When the >>>>>> ephemerons that >>>>>> * refer to ephemerally reachable key object are cleared, the key >>>>>> object becomes >>>>>> * eligible for finalization. >>>>> >>>>> Looking into this a bit more, I don't think the above is quite >>>>> right. Specifically, If an ephemeron's key is either strongly of >>>>> softly reachable, you want the value to remain appropriately >>>>> strongly/softly reachable. Without this quality, Ephemeron value >>>>> referents can (and will) be prematurely collected and finalized >>>>> while the keys are not. This (IMO) needed quality not provided by >>>>> the behavior you specify? >>>> >>>> This is not quite true. While ephemeron's value is weakly or even >>>> ephemerally-reachable, it is not finalizable, because >>>> ephemeraly-reachable is stronger than finaly-reachable. After >>>> ephemeron's key becomes ephemeraly-reachable, the ephemeron is >>>> cleared by GC which sets it's key *and* value to null atomically. >>>> The life of key and value at that moment becomes untangled. Either >>>> of them can have a finalizer or not and both of them will >>>> eventually be collected if not revived by their finalize() methods. >>>> But it can never happen that ephemeron's value is finalized or >>>> collected while it's key is still reachable through the ephemeron >>>> (while the ephemeron is not cleared yet). >>>> >>>> But I agree that it would be desirable for ephemeron's value to >>>> follow the reachability of it's key. In above specification, if the >>>> key is strongly reachable, the value is weakly reachable, so any >>>> WeakReferences or SoftReferences pointing at the Ephemeron's value >>>> can already be cleared while the key is still strongly reachable. >>>> This is arguably no different than current specification of Soft >>>> vs. Weak references. A SoftReference can already be cleared while >>>> its referent is still reachable through a WeakReference, >>> >>> We seem to agree about the cleaner behavior specification (in both >>> of our texts below), so the these next paragraphs are really about >>> arguing for why this is an important design choice if/when adding >>> Ephemerons to Java: >>> >>> It is true the [current] spec allows for soft references to an >>> object to be cleared while weak references to the same object are >>> not: the "determines" in "Suppose that the garbage collector >>> determines at a certain point in time hat an object is RRRR >>> reachable..." part [for RRRR = {soft, weak}] does not have to happen >>> at the same "certain point in time". >>> >>> However, to my knowledge all current implementations present as if >>> this determination is happening at the same "point in time" for all >>> weakly and softly reachable objects combined. Specifically [in >>> implementations]: if soft reachability is determined for an object >>> at some point in time, then weak reachability for that object is >>> determined at the same point in time. And the weak reachability >>> determination for an object depends on whether the collector chose >>> to clear existing soft references to that object at that same point >>> in time, with the appearance of the choice to clear (or not to >>> clear) soft references to a given object atomically affecting the >>> determination of it's weak reachability. Since the collector is >>> *required* to act on a weak determination when it is made, while it >>> *may* act on a soft determination when it is made, making the >>> combined determination at the same "point in time" eliminates an >>> obviously confusing situation that is not prohibited by the spec: if >>> the determination for weak and soft reachability was not done at the >>> same point in time, then an object that was softly reachable and had >>> it's soft references cleared and queued could later become strongly >>> reachable, and even softly reachable again. When reference >>> processing is done as a STW thing, this "combined determination" >>> effect is a trivial side-effect of STW. When it is done concurrently >>> (or incrementally?), implementations still work to maintain the >>> appearance of combined atomic determination of soft and weak >>> reachability. I know ours does. In our case, we do it because we had >>> no desire to be the ones to argue "I know that all implementations >>> did this atomically because they were STW, but the spec allows us to >>> add this bug to your program?". >>> >>> So in actual implementations (to my knowledge), finalization is >>> currently the only mechanism that can create this "strange >>> situation" where an object was no longer strongly reachable, had >>> actions triggered as a result from loss of strong reachability (i.e. >>> actually observed by the program as "known to not be strongly >>> reachable"), and later became strongly reachable again. E.g. a >>> finalizer can propagate a strong reference to a previously >>> non-strongly reachable object ('this' in the finalizer, or anything >>> that 'this' transitively refers and was not otherwise reachable when >>> the finalizer was called).. This is one of those "undesired" things >>> that the introduction of Reference types was meant to deal with >>> (Reference types were introduced in 1.2, after finalization was >>> unfortunately already included and spec'ed. And phantom refs were >>> meant to allow for a cleaner form that could replace finalization). >>> And while the specifications of SoftReference and WeakReference do >>> not prohibit it, implementations are not required to allow it, and >>> in practice non of them do (I think), as doing so would most likely >>> expose some "interesting" spec-allowed-but-extremely-surprising >>> things/bugs that none of us want to have to defend... >>> >>> In this context, it would be a "highly undesirable" design choice to >>> introduce Ephemerons in a way that would them to return a strong >>> reference to an object that has previously been determined to no >>> longer be strongly reachable. Structuring the spec to prohibit this >>> is a better design choice. >>> >>> To highlight the design choice here, let me describe a specific >>> problem scenario for which the previous (above) spec would cause >>> "re-strengthening" behavior that would break assumptions that are >>> allowed under the current spec: in the above/previously specified >>> behavior an object V that is known to have no finalizers, but has >>> e.g. 3 WeakReference objects that refer to it, can become weakly >>> reachable while both a key referent object K in some ephemeron E >>> with a value referent of V remain strongly reachable. At such a >>> point (V is weakly reachable, K and E are strongly reachable), the >>> collector may determine weak reachability for V, [atomically] clear >>> all weak references to V, and enqueue those weak reference objects >>> on their respective queues. While V is still ephemerally reachable >>> under your previous definition, there are no references to it >>> anywhere other than in ephemeron value referent fields, and weak >>> references that did refer to it have been cleared and queued. Since >>> the ephemeron is still there, and the key is still there, and the >>> ephemeron has not been cleared, an Ephemeron.getValue() call would >>> create a strong reference to an object that was previously >>> determined to not be weakly reachable. Re-creating a strong >>> reference to V after the point where weak references to V were >>> cleared and the weak refs to it were enqueued would be "surprising" >>> to current weak reference based code (the only thing that could >>> cause this under the current spec would be a finalizer), so allowing >>> that (jn the spec) is likely to break all kinds of logic that >>> depends on currently spec'ed weak reference behaviors. >>> >>> The spec'ed behavior we seem to be agreeing on (below) would >>> prohibit this loophole and would [I think] maintain any >>> reachability-based expectations that current weak-ref based logic >>> can make under the current spec. Maintaining this continuity is an >>> important design choice for adding Ephemerons into the current set >>> of Reference behaviors. >>> >>> And since I suspect that all implementations will continue to choose >>> to do the "determination" of soft and weak reachability at the same >>> "point in time", this will fit well with how people would build this >>> stuff anyway. >>> >>> Separate note: It would be separately interesting to consider >>> narrowing the SoftRef spec to require JVM implementations to >>> atomically clear all soft *and* weak references to an object at the >>> same time. I.e. if the garbage collector chooses to clear a soft >>> reference to an object that would become weakly reachable as a >>> result, then all weak references to that object must be [atomically] >>> cleared at the same time. Since I suspect that all current JVM >>> implementations actually adhere to this stronger requirement >>> already, this would not "hurt" anything or require extra work to >>> comply with. [Anyone from Metronome or some other non-STW reference >>> processing implementations want to chime in?]. >>> >>>> but for Ephemeron's value this might be confusing. The easier to >>>> understand conceptual model for Ephemerons might be a pair of >>>> (WeakReference, WeakReference) where the key has a virtual >>>> strong reference to the value. And this is what we get if we say >>>> that reachability of the value follows reachability of the key. >>>> >>>>> >>>>> For a correctly specified behavior, I think all strengths (from >>>>> strong down) need to be affected by key/value Ephemeron >>>>> relationships, but without adding an "ephemerally reachable" >>>>> strength. E.g. I think you fundamentally need something like this: >>>>> >>>>> - "An object is strongly reachable if it can be reached >>>>> by (a) some thread without traversing any reference objects, or by >>>>> (b) traversing the value of an Ephemeron whose key is strongly >>>>> reachable. A newly-created object is strongly reachable by the >>>>> thread that created it" >>>>> >>>>> - "An object is softly reachable if it is not >>>>> strongly reachable but can be reached by (a) traversing a soft >>>>> reference or by (b) traversing the value of an Ephemeron whose key >>>>> is softly reachable. >>>>> >>>>> - "An object is weakly reachable if it is neither >>>>> strongly nor softly reachable but can be reached by (a) traversing >>>>> a weak reference or by (b) traversing the value of an ephemeron >>>>> whose key is weakly reachable. >>>> >>>> ...and that's where we stop, because when we make Ephemeron just a >>>> special kind of WeakReference, the next thing that happens is: >>>> >>>> *

    Suppose that the garbage collector determines at a certain >>>> point in time >>>> * that an object is weakly >>>> * reachable. At that time it will atomically clear all weak >>>> references to >>>> * that object and all weak references to any other >>>> weakly-reachable objects >>>> * from which that object is reachable through a chain of strong >>>> and soft >>>> * references. At the same time it will declare all of the formerly >>>> * weakly-reachable objects to be finalizable. At the same time or >>>> at some >>>> * later time it will enqueue those newly-cleared weak references >>>> that are >>>> * registered with reference queues. >>>> >>>> ...where "clearing of the WeakReference" means reseting the key >>>> *and* value to null in case it is an Ephemeron; and >>>> "all weak references to some object" means Ephemerons that have >>>> that object as a key (but not those that only have it as a value!) >>>> in case of ephemerons >>>> >>>> ... >>>>> I still think that Ephemeron should extend WeakReference, >>>>> since that places already established rules and expectation on (a) >>>>> when it will be enqueued, (b) when the collector will clear it >>>>> (when the the collector encounters the key being weakly >>>>> reachable), and (c) that clearing of all Ephemeron *and* >>>>> WeakReference instances who share an identical key value is done >>>>> atomically, along with (d) all weak references to to any other >>>>> weakly-reachable objects from which that object is reachable >>>>> through a chain of strong and soft references. These last (c, d) >>>>> parts are critically captured since an Ephemeron *is a* >>>>> WeakReference, and the statement in WeakReference that says that >>>>> "? it will atomically clear all weak references to that object and >>>>> all weak references to any other weakly-reachable objects from >>>>> which that object is reachable through a chain of strong and soft >>>>> references." has a clear application. >>>>> >>>>> Here are some suggested edits to the JavaDoc to go with this >>>>> suggested spec'ed behavior: >>>>> /** >>>>> * Ephemeron objects are a special kind of WeakReference >>>>> objects, which >>>>> * hold two referents (a key referent and a value referent) and >>>>> do not prevent their >>>>> * referents from being made finalizable, finalized, and then >>>>> reclaimed. >>>>> * In addition to the key referent, which adheres to the referent >>>>> behavior of a >>>>> * WeakReference, an ephemeron also holds a value referent >>>>> whose reachabiliy >>>>> * strength is affected by the reachability strength of the key >>>>> referent: >>>>> * The value referent of an Ephemeron instance is considered: >>>>> * (a) strongly reachable if the key referent of the same Ephemeron >>>>> * object is strongly reachable, or if the value referent is >>>>> otherwise strongly reachable. >>>>> * (b) softly reachable if it is not strongly reachable, and (i) >>>>> the key referent of >>>>> * the same Ephemeron object is softly reachable, or (ii) if the >>>>> value referent is otherwise >>>>> * softly reachable. >>>>> * (c) weakly reachable if it is not strongly or softly >>>>> reachable, and (i) the key referent of >>>>> * the same Ephemeron object is weakly reachable, or (ii) if the >>>>> value referent is otherwise >>>>> * weakly reachable. >>>>> *

    When the collector clears an Ephemeron object instance >>>>> (according to the rules >>>>> * expressed for clearing WeakReference object instances), the >>>>> Ephemeron instance's >>>>> * key referent value referent are simultaneously and atomically >>>>> cleared. >>>>> *

    By convenience, the Ephemeron's referent is also called >>>>> the key, and can be >>>>> * obtained either by invoking {@link #get} or {@link #getKey} >>>>> while the value >>>>> * can be obtained by invoking {@link #getValue} method. >>>>> *... >>>> >>>> >>>> Thanks, this is very nice. I do like this behavior more. >>>> >>>> Let me see what it takes to implement this strategy... >>>> >>>> Regards, Peter >>>> >>> >> > From yumin.qi at oracle.com Sun Jan 24 22:16:40 2016 From: yumin.qi at oracle.com (Yumin Qi) Date: Sun, 24 Jan 2016 14:16:40 -0800 Subject: RFR: 8147755: ASM should create correct constant tag for invokestatic on handle point to interface static method In-Reply-To: <774410875.2198558.1453649815643.JavaMail.zimbra@u-pem.fr> References: <56A28F6E.30307@oracle.com> <774410875.2198558.1453649815643.JavaMail.zimbra@u-pem.fr> Message-ID: <56A54D48.7020300@oracle.com> Hi, Remi Thanks for your update! Glad to see it fixed in ASM. wish get the new version soon. Thanks Yumin On 1/24/2016 7:36 AM, Remi Forax wrote: > Hi Yumin, > I'm one of the developers of ASM, > I've pushed a new versions of ASM (the future 5.1) in the trunk that should let you create constant method handles on an interface, > when i have updated ASM for Java 8, i've overlooked that case, sorry for that. > > The code is mostly similar to your patch but it does the full roundtrip, being able not only to encode constant method handles on interface but also to decode them. > > Kumar is in CC, i believe, he is the one that sync the private copy of ASM used by the JDK with the SVN trunk. > > regards, > R?mi > > ----- Mail original ----- >> De: "Yumin Qi" >> ?: core-libs-dev at openjdk.java.net, "CHRIS.HEGARTY" , "harold seigel" >> , "Lois Foltan" , "Vladimir Ivanov" >> Envoy?: Vendredi 22 Janvier 2016 21:22:06 >> Objet: RFR: 8147755: ASM should create correct constant tag for invokestatic on handle point to interface static >> method >> >> Hi, All >> >> Can I have a review for >> bug: https://bugs.openjdk.java.net/browse/JDK-8147755 >> Webrev: http://cr.openjdk.java.net/~minqi/8147755/webrev-01/ >> >> Summary: When fix bug 8145148 to follow: >> >> JVMS-5.4.3.3 Method Resolution: >> " If C is an interface, method resolution throws an >> IncompatibleClassChangeError." >> JVMS-5.4.3.4 Interface Method Resolution: >> "If C is not an interface, interface method resolution throws an >> IncompatibleClassChangeError" >> >> In defmeth(default method) tests, a MethodHandle pointing to >> interface static method will violate the spec. Since such a handle in >> asm will generate a tag of Methodref in constantpool for the call site, >> with the fix of 8145148, it requires a tag of InterfaceMethodref. >> >> Fix by create a new constructor of Handle with extra boolean to >> indicate if the handle points to interface static method. >> >> Tests: A test case and resulted constantpool data attached. >> The fix will not affect existing code. >> Also tested with fixed 8145148 and revised defmeth >> with the new version Handle and passed. >> >> Thanks >> Yumin >> >> >> From kumar.x.srinivasan at oracle.com Mon Jan 25 00:35:46 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Sun, 24 Jan 2016 16:35:46 -0800 Subject: RFR: 8147755: ASM should create correct constant tag for invokestatic on handle point to interface static method In-Reply-To: <774410875.2198558.1453649815643.JavaMail.zimbra@u-pem.fr> References: <56A28F6E.30307@oracle.com> <774410875.2198558.1453649815643.JavaMail.zimbra@u-pem.fr> Message-ID: <56A56DE2.404@oracle.com> Hi Remi, > Hi Yumin, > I'm one of the developers of ASM, > I've pushed a new versions of ASM (the future 5.1) in the trunk that should let you create constant method handles on an interface, > when i have updated ASM for Java 8, i've overlooked that case, sorry for that. > > The code is mostly similar to your patch but it does the full roundtrip, being able not only to encode constant method handles on interface but also to decode them. > > Kumar is in CC, i believe, he is the one that sync the private copy of ASM used by the JDK with the SVN trunk. Yes, there are some nits on the JDK side which need to be resolved first, but this is on my TODO list. Thanks Kumar > > regards, > R?mi > > ----- Mail original ----- >> De: "Yumin Qi" >> ?: core-libs-dev at openjdk.java.net, "CHRIS.HEGARTY" , "harold seigel" >> , "Lois Foltan" , "Vladimir Ivanov" >> Envoy?: Vendredi 22 Janvier 2016 21:22:06 >> Objet: RFR: 8147755: ASM should create correct constant tag for invokestatic on handle point to interface static >> method >> >> Hi, All >> >> Can I have a review for >> bug: https://bugs.openjdk.java.net/browse/JDK-8147755 >> Webrev: http://cr.openjdk.java.net/~minqi/8147755/webrev-01/ >> >> Summary: When fix bug 8145148 to follow: >> >> JVMS-5.4.3.3 Method Resolution: >> " If C is an interface, method resolution throws an >> IncompatibleClassChangeError." >> JVMS-5.4.3.4 Interface Method Resolution: >> "If C is not an interface, interface method resolution throws an >> IncompatibleClassChangeError" >> >> In defmeth(default method) tests, a MethodHandle pointing to >> interface static method will violate the spec. Since such a handle in >> asm will generate a tag of Methodref in constantpool for the call site, >> with the fix of 8145148, it requires a tag of InterfaceMethodref. >> >> Fix by create a new constructor of Handle with extra boolean to >> indicate if the handle points to interface static method. >> >> Tests: A test case and resulted constantpool data attached. >> The fix will not affect existing code. >> Also tested with fixed 8145148 and revised defmeth >> with the new version Handle and passed. >> >> Thanks >> Yumin >> >> >> From gil at azul.com Mon Jan 25 03:13:17 2016 From: gil at azul.com (Gil Tene) Date: Mon, 25 Jan 2016 03:13:17 +0000 Subject: Ephemerons In-Reply-To: <56A5309A.20505@gmail.com> References: <569BDAF2.2080404@gmail.com> <7E94B627-1D98-42A6-A5C9-788E6092A5D0@azul.com> <56A2B1EC.6050907@gmail.com> <56A37CB4.3000407@gmail.com> <8154B857-931F-4BED-872A-36892E050CCF@azul.com> <56A4AD09.1010901@gmail.com> <5AFD362E-43AD-4234-8D6E-158AB1BA5340@azul.com> <56A5309A.20505@gmail.com> Message-ID: <82D91E96-7044-4E12-A303-06D8218E135D@azul.com> > On Jan 24, 2016, at 12:14 PM, Peter Levart wrote: > > Hi Gil, > > The algorithm complexity has been on my mind too and I've been thinking of how to maintain a key (or it's address) -> ephemeron mapping most efficiently. This can vary. It's possible to do true O(1) reverse reference lookup in a very memory-wasteful way, obviously, but it's unlikely to be the path any implementation chooses to take. Simple tree based schemes will easily cap reverse reference lookup at O(log N) (and thereby overall Ephemeron processing at O(N log N)). But my guess is that most implementations will opt for a hash based scheme, which will provide "usually O(1)" but cap be capped at O(log N) even with worst-case hashing behavior (e,g. a map scheme with buckets expand to trees instead of linked lists, much like the under-the-hood behavior of Java 8 HashMaps). Since both trees and hash schemes will only need storage space linear to the number of Ephemerons, I think reverse mapping solution will be straight forward to support. BTW, we use a hash based scheme to maintain our forwarding pointer storage in C4 today, and it works very well. Hence my educated guess that hash based schemes will work well for this too. > As I have written in a reply to Mandy, if we say that individual phase2 of processing of the normal References has complexity O(N) where N is the number of discovered references, then the equivalent phase2 processing of Ephemerons has comparable complexity O(N*d) where N is the number of discovered Ephemerons and d is the maximum number of "hops" in object graph where a "hop" is defined as a point in chain where the value of some ephemeron refers directly or indirectly to the key of some other ephemeron and the value of that other ephemeron continues the chain. Let's hope that in practical data structures, d is not large. The maximum value of d is N, as in one of the tests. The algorithm in the prototype tries to reduce the number of outer iterations to much less than 'd' in many situations by alternating the direction of scanning in each pass. The value(ephemeronX) -> key(ephemeronY) links could be arranged in a zig-zag pattern relative to the order of discovered references in the list though which would be the worst case for the algorithm, but I think the chances of that happening in a real-world data structure are low. I've learned the hard way to not assume things about "real world" shapes and topologies for newly proposed constructs. Once they become available, people will use them in new and ingenious ways. Many of which will be very valid and very unanticipated. Since there are [by definition] no common uses of Ephemerons in Java yet, and Ephemerons are not commonly exposed (yet) in other languages and environments that commonly support concurrent thread activity, it's hard to anticipate how they may be put to use beyond the example cases of caches/maps/dictionaries. My guess is that there will be some interesting ways to apply them to concurrent algorithms that would be "hard" to build or solve otherwise. So I would assume that there might be very legitimate use cases that would be natural to generate a very long linked list weaving through ephemeron keys and values. And there is nothing the multi-pass thing can do (by alternating or zig zagging) to reduce the number of passes needed to propogate across those linked ephemerons one-per-pass? A reverse lookup scheme will eliminate the need for iterative passes and eliminate this issue... > But we have to think of possible denial-of-service attacks too, so I agree that worst-case scenario has to be improved. > > Presently, Reference objects are discovered and hooked on single-linked _discoveredXXX lists (using Reference.discovered field to point to the next in list). Each reference type has it's own set of lists. Multiple lists in a set enable parallel (concurrent?) discovery without contention (a list per thread). > > So I was thinking that one possibility would be for ephemerons to have a bigger table of discovered lists - not just one per discovery thread, but quite bigger and that the index of a list to which discovered ephemeron is added would be computed from the ephemeron's key. A classical hash-table with buckets. Big enough table would minimize contention when discovery is parallelized and enable the following algorithm (which could also be parallelized): > > Let each bucket (each head of the list) have 2 boolean flags: include & pending. > At the start, set include flags of all buckets to true and then enter the loop: > do { > reset pending flags of all buckets to false; > for each bucket with (include == true), do { > scan the ephemerons and for those with live keys, mark value and transitive closure as live. > while marking the value and transitive closure as live, for each object that was newly marked alive, > compute the index into the ephemeron buckets as though such object was ephemeron's key > and set the pending flag of that bucket. > } > set the include flags of all buckets from the pending flags of the buckets and count # of pending buckets > } while (# of pending buckets > 0); Interesting. I think that this can be described as a "pessimistically estimated reverse reference scheme", where you don't actually track reverse references, but pessimistically (with some hash) act as if you hit them when you newly mark an object live, causing a force-re-evaluate ephemerons that have been "hit" by the hash. But since I think that an actual reverse reference mapping scheme is fairly simple to implement, and is both "simpler" in many ways and would result in less GC work (no false positive checks forced on ephemerons when marking happens to hit them with a hash), I'd stick with that... > > Hm, .... > > > Regards, Peter > > On 01/24/2016 06:43 PM, Gil Tene wrote: >> A note on implementation possibilities: >> >> If I read the implementation correctly, a "weakness" of the current implementation approach for making sure value-referents (and their transitively reachable) objects are kept alive if key referents are alive is that it requires multiple passes through the discovered Ephemeron list, with the passes terminating when the list stabilizes. While I think that this is sound (in the sense that it will work), it carries a potentially high cost when large sets of Ephemerons exist in the heap. E.g. if the Ephemerons are linked in a k-v list (where the value referent of one ephemeron is the key of another, in a chain), as in your code example, there is an N^2 scanning thing going on. And e.g. if a large set of ephemeron keys become weakly reachable in a single cycle (e.g. a large cache was discarded) while other ephemeron participate in some linked list relationship, the entire list of [stably] weak-keyed ephemerons has to be traversed in each pass (in case one of them has become live in a previous pass). I'd worry that these computational complexity issues could become prohibitive enough in GC cost that you there would be significant resistance to their adoption. >> >> Note that in comparison (to my understanding), current ref processing work involved in GC handling soft/weak/final/phantom refs remains linear to the number of refs, and does not have an O(N^2) component. >> >> I believe that there is a relatively simple way to bring Ephemeron processing to O(N) by establishing reverse mapping during the scan (the below description assumes STW during the scan): >> >> 1. Start ref processing with no reverse mapping table established. >> >> 2. During ref processing, establish an EphemeronKeyReverseMapping table (logically a Map>) which would map individual heap addresses to ephemerons who's key referent points to those addresses. >> >> 3. Note that since each heap address can show up in multiple key referents, the map needs to return a (potentially empty) list of Ephemerons who's keys refer to the address. >> >> 4. Specifically, starting with an empty list, and for each discovered Ephemeron, add a reverse-mapping entry to the EphemeronKeyReverseMapping, mapping from the key referent address to the Ephemeron. >> >> 5. During Ephemeron processing (under the case where the referent is found to be alive and the ephemeron then needs to keep the value referent alive) mark down the value referent path using a special ephemeron_keep_alive OopClosure (or a mode flag that affects the normal keep_alive behavior) which, when reaching a not-yet-marked-live object [in addition to marking it live and traversing it as keep_alive would normally do] would look up the object's address in the EphemeronKeyReverseMapping to get a list of Ephemerons to traverse, and traverse each of the mapped Ephemerons' value referent with the same ephemeron_keep_alive closure. >> Note: doing reverse-mapping lookups on each not-yet-marked object in a keep_alive closure will add cost, which is why this ephemeron_keep_alive pass should probably be done after regular keep_alive passes have had their chance to mark objects live. This way only paths that become newly-live via ephemeron processing are subject to the extra reverse-mapping-lookip cost. >> >> While I haven't been poking at this too long to see if it has holes, I think it can produce a reliable result, and is O(N) on the count of Ephemerons. >> >> ? Gil. >> >> >>> On Jan 24, 2016, at 2:52 AM, Peter Levart < peter.levart at gmail.com > wrote: >>> >>> Hi Gil, >>> >>> I totally agree with your assessment. We should not introduce another way of reviving the almost collectable objects and I fully support tightening the specification so that soft and weak references to the same referent and to other referents from which this referent is reachable are required to be cleared together atomically. >>> >>> I modified the prototype to (hopefully) adhere to this new Ephemeron specification that Gil and I agreed upon. Anyone interested in experimenting can find it here: >>> >>> http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.jdk.02/ >>> http://cr.openjdk.java.net/~plevart/misc/Ephemeron/webrev.hotspot.02/ >>> >>> It is rebased to current tip of jdk9-dev repositories (after the bulk of merges for jdk-9+102), but still contains the change to remove the Cleaner reference type as it has not yet managed to get in... >>> >>> I have also added a test that is a start for verifying the functionality. >>> >>> Regards, Peter >>> >>> On 01/23/2016 07:25 PM, Gil Tene wrote: >>>> >>>>> On Jan 23, 2016, at 5:14 AM, Peter Levart > wrote: >>>>> >>>>> Hi Gil, it's good to have this discussion. See comments inline... >>>>> >>>>> On 01/23/2016 05:13 AM, Gil Tene wrote: >>>>> .... >>>>>>> On Jan 22, 2016, at 2:49 PM, Peter Levart < peter.levart at gmail.com > wrote: >>>>>>> >>>>>>> Ephemeron always touches definitions of at least two consecutive strengths of reachabilities. The prototype says: >>>>>>> >>>>>>> *

  • An object is weakly reachable if it is neither >>>>>>> * strongly nor softly reachable but can be reached by traversing a >>>>>>> * weak reference or by traversing an ephemeron through it's value while >>>>>>> * the ephemeron's key is at least weakly reachable. >>>>>>> >>>>>>> *
  • An object is ephemerally reachable if it is neither >>>>>>> * strongly, softly nor weakly reachable but can be reached by traversing an >>>>>>> * ephemeron through it's key or by traversing an ephemeron through it's value >>>>>>> * while it's key is at most ephemerally reachable. When the ephemerons that >>>>>>> * refer to ephemerally reachable key object are cleared, the key object becomes >>>>>>> * eligible for finalization. >>>>>> >>>>>> Looking into this a bit more, I don't think the above is quite right. Specifically, If an ephemeron's key is either strongly of softly reachable, you want the value to remain appropriately strongly/softly reachable. Without this quality, Ephemeron value referents can (and will) be prematurely collected and finalized while the keys are not. This (IMO) needed quality not provided by the behavior you specify? >>>>> >>>>> This is not quite true. While ephemeron's value is weakly or even ephemerally-reachable, it is not finalizable, because ephemeraly-reachable is stronger than finaly-reachable. After ephemeron's key becomes ephemeraly-reachable, the ephemeron is cleared by GC which sets it's key *and* value to null atomically. The life of key and value at that moment becomes untangled. Either of them can have a finalizer or not and both of them will eventually be collected if not revived by their finalize() methods. But it can never happen that ephemeron's value is finalized or collected while it's key is still reachable through the ephemeron (while the ephemeron is not cleared yet). >>>>> >>>>> But I agree that it would be desirable for ephemeron's value to follow the reachability of it's key. In above specification, if the key is strongly reachable, the value is weakly reachable, so any WeakReferences or SoftReferences pointing at the Ephemeron's value can already be cleared while the key is still strongly reachable. This is arguably no different than current specification of Soft vs. Weak references. A SoftReference can already be cleared while its referent is still reachable through a WeakReference, >>>> >>>> We seem to agree about the cleaner behavior specification (in both of our texts below), so the these next paragraphs are really about arguing for why this is an important design choice if/when adding Ephemerons to Java: >>>> >>>> It is true the [current] spec allows for soft references to an object to be cleared while weak references to the same object are not: the "determines" in "Suppose that the garbage collector determines at a certain point in time hat an object is RRRR reachable..." part [for RRRR = {soft, weak}] does not have to happen at the same "certain point in time". >>>> >>>> However, to my knowledge all current implementations present as if this determination is happening at the same "point in time" for all weakly and softly reachable objects combined. Specifically [in implementations]: if soft reachability is determined for an object at some point in time, then weak reachability for that object is determined at the same point in time. And the weak reachability determination for an object depends on whether the collector chose to clear existing soft references to that object at that same point in time, with the appearance of the choice to clear (or not to clear) soft references to a given object atomically affecting the determination of it's weak reachability. Since the collector is *required* to act on a weak determination when it is made, while it *may* act on a soft determination when it is made, making the combined determination at the same "point in time" eliminates an obviously confusing situation that is not prohibited by the spec: if the determination for weak and soft reachability was not done at the same point in time, then an object that was softly reachable and had it's soft references cleared and queued could later become strongly reachable, and even softly reachable again. When reference processing is done as a STW thing, this "combined determination" effect is a trivial side-effect of STW. When it is done concurrently (or incrementally?), implementations still work to maintain the appearance of combined atomic determination of soft and weak reachability. I know ours does. In our case, we do it because we had no desire to be the ones to argue "I know that all implementations did this atomically because they were STW, but the spec allows us to add this bug to your program?". >>>> >>>> So in actual implementations (to my knowledge), finalization is currently the only mechanism that can create this "strange situation" where an object was no longer strongly reachable, had actions triggered as a result from loss of strong reachability (i.e. actually observed by the program as "known to not be strongly reachable"), and later became strongly reachable again. E.g. a finalizer can propagate a strong reference to a previously non-strongly reachable object ('this' in the finalizer, or anything that 'this' transitively refers and was not otherwise reachable when the finalizer was called).. This is one of those "undesired" things that the introduction of Reference types was meant to deal with (Reference types were introduced in 1.2, after finalization was unfortunately already included and spec'ed. And phantom refs were meant to allow for a cleaner form that could replace finalization). And while the specifications of SoftReference and WeakReference do not prohibit it, implementations are not required to allow it, and in practice non of them do (I think), as doing so would most likely expose some "interesting" spec-allowed-but-extremely-surprising things/bugs that none of us want to have to defend... >>>> >>>> In this context, it would be a "highly undesirable" design choice to introduce Ephemerons in a way that would them to return a strong reference to an object that has previously been determined to no longer be strongly reachable. Structuring the spec to prohibit this is a better design choice. >>>> >>>> To highlight the design choice here, let me describe a specific problem scenario for which the previous (above) spec would cause "re-strengthening" behavior that would break assumptions that are allowed under the current spec: in the above/previously specified behavior an object V that is known to have no finalizers, but has e.g. 3 WeakReference objects that refer to it, can become weakly reachable while both a key referent object K in some ephemeron E with a value referent of V remain strongly reachable. At such a point (V is weakly reachable, K and E are strongly reachable), the collector may determine weak reachability for V, [atomically] clear all weak references to V, and enqueue those weak reference objects on their respective queues. While V is still ephemerally reachable under your previous definition, there are no references to it anywhere other than in ephemeron value referent fields, and weak references that did refer to it have been cleared and queued. Since the ephemeron is still there, and the key is still there, and the ephemeron has not been cleared, an Ephemeron.getValue() call would create a strong reference to an object that was previously determined to not be weakly reachable. Re-creating a strong reference to V after the point where weak references to V were cleared and the weak refs to it were enqueued would be "surprising" to current weak reference based code (the only thing that could cause this under the current spec would be a finalizer), so allowing that (jn the spec) is likely to break all kinds of logic that depends on currently spec'ed weak reference behaviors. >>>> >>>> The spec'ed behavior we seem to be agreeing on (below) would prohibit this loophole and would [I think] maintain any reachability-based expectations that current weak-ref based logic can make under the current spec. Maintaining this continuity is an important design choice for adding Ephemerons into the current set of Reference behaviors. >>>> >>>> And since I suspect that all implementations will continue to choose to do the "determination" of soft and weak reachability at the same "point in time", this will fit well with how people would build this stuff anyway. >>>> >>>> Separate note: It would be separately interesting to consider narrowing the SoftRef spec to require JVM implementations to atomically clear all soft *and* weak references to an object at the same time. I.e. if the garbage collector chooses to clear a soft reference to an object that would become weakly reachable as a result, then all weak references to that object must be [atomically] cleared at the same time. Since I suspect that all current JVM implementations actually adhere to this stronger requirement already, this would not "hurt" anything or require extra work to comply with. [Anyone from Metronome or some other non-STW reference processing implementations want to chime in?]. >>>> >>>>> but for Ephemeron's value this might be confusing. The easier to understand conceptual model for Ephemerons might be a pair of (WeakReference, WeakReference) where the key has a virtual strong reference to the value. And this is what we get if we say that reachability of the value follows reachability of the key. >>>>> >>>>>> >>>>>> >>>>>> For a correctly specified behavior, I think all strengths (from strong down) need to be affected by key/value Ephemeron relationships, but without adding an "ephemerally reachable" strength. E.g. I think you fundamentally need something like this: >>>>>> >>>>>> - "An object is strongly reachable if it can be reached by (a) some thread without traversing any reference objects, or by (b) traversing the value of an Ephemeron whose key is strongly reachable. A newly-created object is strongly reachable by the thread that created it" >>>>>> >>>>>> - "An object is softly reachable if it is not strongly reachable but can be reached by (a) traversing a soft reference or by (b) traversing the value of an Ephemeron whose key is softly reachable. >>>>>> >>>>>> - "An object is weakly reachable if it is neither strongly nor softly reachable but can be reached by (a) traversing a weak reference or by (b) traversing the value of an ephemeron whose key is weakly reachable. >>>>> >>>>> ...and that's where we stop, because when we make Ephemeron just a special kind of WeakReference, the next thing that happens is: >>>>> >>>>> *

    Suppose that the garbage collector determines at a certain point in time >>>>> * that an object is weakly >>>>> * reachable. At that time it will atomically clear all weak references to >>>>> * that object and all weak references to any other weakly-reachable objects >>>>> * from which that object is reachable through a chain of strong and soft >>>>> * references. At the same time it will declare all of the formerly >>>>> * weakly-reachable objects to be finalizable. At the same time or at some >>>>> * later time it will enqueue those newly-cleared weak references that are >>>>> * registered with reference queues. >>>>> >>>>> ...where "clearing of the WeakReference" means reseting the key *and* value to null in case it is an Ephemeron; and >>>>> "all weak references to some object" means Ephemerons that have that object as a key (but not those that only have it as a value!) in case of ephemerons >>>>> >>>>> ... >>>>>> I still think that Ephemeron should extend WeakReference, since that places already established rules and expectation on (a) when it will be enqueued, (b) when the collector will clear it (when the the collector encounters the key being weakly reachable), and (c) that clearing of all Ephemeron *and* WeakReference instances who share an identical key value is done atomically, along with (d) all weak references to to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references. These last (c, d) parts are critically captured since an Ephemeron *is a* WeakReference, and the statement in WeakReference that says that "? it will atomically clear all weak references to that object and all weak references to any other weakly-reachable objects from which that object is reachable through a chain of strong and soft references." has a clear application. >>>>>> >>>>>> Here are some suggested edits to the JavaDoc to go with this suggested spec'ed behavior: >>>>>> /** >>>>>> * Ephemeron objects are a special kind of WeakReference objects, which >>>>>> * hold two referents (a key referent and a value referent) and do not prevent their >>>>>> * referents from being made finalizable, finalized, and then reclaimed. >>>>>> * In addition to the key referent, which adheres to the referent behavior of a >>>>>> * WeakReference, an ephemeron also holds a value referent whose reachabiliy >>>>>> * strength is affected by the reachability strength of the key referent: >>>>>> * The value referent of an Ephemeron instance is considered: >>>>>> * (a) strongly reachable if the key referent of the same Ephemeron >>>>>> * object is strongly reachable, or if the value referent is otherwise strongly reachable. >>>>>> * (b) softly reachable if it is not strongly reachable, and (i) the key referent of >>>>>> * the same Ephemeron object is softly reachable, or (ii) if the value referent is otherwise >>>>>> * softly reachable. >>>>>> * (c) weakly reachable if it is not strongly or softly reachable, and (i) the key referent of >>>>>> * the same Ephemeron object is weakly reachable, or (ii) if the value referent is otherwise >>>>>> * weakly reachable. >>>>>> *

    When the collector clears an Ephemeron object instance (according to the rules >>>>>> * expressed for clearing WeakReference object instances), the Ephemeron instance's >>>>>> * key referent value referent are simultaneously and atomically cleared. >>>>>> *

    By convenience, the Ephemeron's referent is also called the key, and can be >>>>>> * obtained either by invoking {@link #get} or {@link #getKey} while the value >>>>>> * can be obtained by invoking {@link #getValue} method. >>>>>> *... >>>>> >>>>> >>>>> Thanks, this is very nice. I do like this behavior more. >>>>> >>>>> Let me see what it takes to implement this strategy... >>>>> >>>>> Regards, Peter >>>>> >>>> >>> >> > From ramanand.patil at oracle.com Mon Jan 25 07:35:13 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Sun, 24 Jan 2016 23:35:13 -0800 (PST) Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale In-Reply-To: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> Message-ID: <575e40e1-9041-424a-8057-f8c289e032ae@default> Hi all, Please review the trivial test bug fix for: https://bugs.openjdk.java.net/browse/JDK-8147912 Bug Description: Since the test case "parseWithZoneWithoutOffset" is using hard code as input data it will fail on some non-English locales (de_DE locale). Webrev: http://cr.openjdk.java.net/~rpatil/8147912/webrev.00 Fix: Even though hardcoded data is not preferred in compatibility test cases, this case was exception. English is provided as the default locale data for DateTimeFormatter in this testcase. Regards, Ramanand. From paul.sandoz at oracle.com Mon Jan 25 08:43:13 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 25 Jan 2016 09:43:13 +0100 Subject: OpenJDK8: java.util.stream.Stream.onClose In-Reply-To: <56a34cdf.460f1c0a.03c7.3dbd@mx.google.com> References: <56a34cdf.460f1c0a.03c7.3dbd@mx.google.com> Message-ID: > On 23 Jan 2016, at 10:50, Timo Kinnunen wrote: > > Hi, > > The pipeline stages don?t really need to mutate themselves (as they can mutate the Stream if needed). So they should be shareable between Streams, alleviating some of cost of creating new copies of a Stream, right? Yes, we want to avoid creating new stages (Stream instances) in the pipeline for stuff that can be associated more efficiently as state of the pipeline (attached to the head) Paul. From Alan.Bateman at oracle.com Mon Jan 25 13:31:37 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jan 2016 13:31:37 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A50588.20000@gmail.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> <56A50588.20000@gmail.com> Message-ID: <56A623B9.9090305@oracle.com> On 24/01/2016 17:10, Peter Levart wrote: > Hi, > > I had an idea recently on how to expedite the collection of an object. > It is simple - just don't let it live long. > > Here's a concept prototype: > > http://cr.openjdk.java.net/~plevart/misc/CloseableMemory/CloseableMemory.java > > > The overhead of the check in access methods (getByte()/setByte()) > amounts to one volatile read of an oop variable that changes once per > say 5 to 10 seconds. That's the period a special guard object is > alive. It's reachability is tracked by the GC and extends to the end > of each access method (using Reference.reachabilityFence). Every few > seconds, the guard object is changed with new fresh one so that the > chance of the guard and its tracking Cleaner being promoted to old > generation is very low. > > Could something like that enable a low-overhead > CloseableMappedByteBuffer? > If I read this correctly then this still depends on timely reference processing. The switching guard trick to keep the Cleaner from being promoted is interesting of course. So if we were to do something like this with MBB then it would mean the file mapping would still exist and so we'd have the usual issue of trying to delete the underlying file. Then we have the issue of file or socket I/O where threads could be blocked doing I/O with regions of the buffer and so delay indefinitely any release. That said, maybe it is an option for Uwe. -Alan. From miroslav.kos at oracle.com Mon Jan 25 13:56:39 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Mon, 25 Jan 2016 14:56:39 +0100 Subject: RFR [9]: 8145104: NPE is thrown when JAXBContextFactory implementation is specified in system property, 8145112: newInstance(String, ClassLoader): java.lang.JAXBException should not be wrapped as expected according to spec In-Reply-To: <56938E7F.1010800@oracle.com> References: <567953BF.5010206@oracle.com> <56938E7F.1010800@oracle.com> Message-ID: <56A62997.8020208@oracle.com> Ping again - would somebody find some time to review those, please? Thanks M. On 11/01/16 12:14, Miroslav Kos wrote: > Ping ... anybody? > > On 22/12/15 14:44, Miroslav Kos wrote: >> Hi everybody, >> I'd like to ask for reviewing following patch. It fixes two issues: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8145104 >> NPE is thrown when JAXBContextFactory implementation is specified in >> system property >> >> In short, it's a regression of recent changes in JAXB-AP. The old >> version of API is using static methods to create JAXBContext >> instance, where as the new configuration facility uses defined >> interface. The methods are called via core reflection, so when >> invoking non-static method of spi implemetation, instance of the spi >> must be created first. This fixes NPE which occured in certain >> scenarios. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8145112 >> newInstance(String, ClassLoader): java.lang.JAXBException should not >> be wrapped as expected according to spec >> >> There are still some changes in javadoc being discussed, but anyway >> this double-wrapping exception into JAXBException doesn't make any >> sense. JAXBException thrown from JAXB spi provider is re-thrown as is. >> >> Fix: http://cr.openjdk.java.net/~mkos/8145104/jaxws.01/ >> Tests: http://cr.openjdk.java.net/~mkos/8145104/jdk.02/ >> >> Added 2 new jtreg tests, the tests added also to standalone project. >> >> Thanks >> Miran >> > From sean.coffey at oracle.com Mon Jan 25 14:39:28 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 25 Jan 2016 14:39:28 +0000 Subject: RFR 8065076/9, test/java/net/SocketPermission/SocketPermissionTest.java failed intermittently In-Reply-To: <56A2418A.1060903@oracle.com> References: <568F74F7.70100@oracle.com> <56973B1A.30000@oracle.com> <67D3FEF2-7B00-4C90-8D87-6258B7AE547A@oracle.com> <56A2418A.1060903@oracle.com> Message-ID: <56A633A0.3000309@oracle.com> The changes look fine to me also Chris. Regards, Sean. On 22/01/16 14:49, Chris Hegarty wrote: > On 21/01/16 22:55, Felix Yang wrote: >> Hi Chris, >> your fix is cool. I will assign the bug to you:) >> a comment on this fix. The test changed system SecurityManager and >> it is not executed with othervm mode. I think you need to rollback the >> change after test. > > I will revert the change and have the test run in othervm mode. > > I did do a complete test run with this change and it did not cause > any problems, but then again the policy is all permissions! > > Thanks Felix. > > -Chris. > > > Otherwise it may affect other tests. >> >> Thanks, >> Felix >>> On Jan 20, 2016, at 12:45 PM, Chris Hegarty >> > wrote: >>> >>> On 20 Jan 2016, at 06:36, Chris Hegarty >> > wrote: >>> >>>> Felix, >>>> >>>> On 14 Jan 2016, at 06:07, Felix Yang >>> > wrote: >>>> >>>>> Hi all, >>>>> please review the fix for >>>>> test/java/net/SocketPermission/SocketPermissionTest.java, which >>>>> fails frequently with "java.net.BindException: Address already in >>>>> use". >>>>> >>>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8065076 >>>>> Webrev: http://cr.openjdk.java.net/~xiaofeya/8065076/webrev.00 >>>> >>>> My preference is to avoid getFreePort. It is problematic and I >>>> believe just obfuscates >>>> the intermittent failures further. >>>> >>>> In many of the test scenarios the ?listening? socket can be created >>>> before the specific >>>> access control context and associated permission are created. >>>> >>>> I?ll see if I can get some time to try this out. >>> >>> I spent a little time on this today. I basically rewrote the test, but >>> kept the >>> same test scenarios. The use of data providers was cute, but not >>> workable >>> since there is no common supertype for the socket classes. I decided to >>> just expand out the test cases manually. This will give the same test >>> coverage, but should be stable since it creates the sockets first, >>> on an >>> ephemeral port, and then constructs the permissions appropriately given >>> that port. >>> >>> http://cr.openjdk.java.net/~chegar/8065076/ >>> >>> The webrev diffs are almost useless, just review the new file, and >>> compare >>> test scenarios against the what is in the old file. >>> >>> -Chris. >> From nadeesh.tv at oracle.com Mon Jan 25 15:25:04 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Mon, 25 Jan 2016 20:55:04 +0530 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit Message-ID: <56A63E50.5030007@oracle.com> Hi all, Please review a fix for conversion between Chronounit and Timeunit Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ -- Thanks and Regards, Nadeesh TV From peter.levart at gmail.com Mon Jan 25 15:27:29 2016 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 25 Jan 2016 16:27:29 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A623B9.9090305@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> <56A50588.20000@gmail.com> <56A623B9.9090305@oracle.com> Message-ID: <56A63EE1.4080100@gmail.com> On 01/25/2016 02:31 PM, Alan Bateman wrote: > On 24/01/2016 17:10, Peter Levart wrote: >> Hi, >> >> I had an idea recently on how to expedite the collection of an >> object. It is simple - just don't let it live long. >> >> Here's a concept prototype: >> >> http://cr.openjdk.java.net/~plevart/misc/CloseableMemory/CloseableMemory.java >> >> >> The overhead of the check in access methods (getByte()/setByte()) >> amounts to one volatile read of an oop variable that changes once per >> say 5 to 10 seconds. That's the period a special guard object is >> alive. It's reachability is tracked by the GC and extends to the end >> of each access method (using Reference.reachabilityFence). Every few >> seconds, the guard object is changed with new fresh one so that the >> chance of the guard and its tracking Cleaner being promoted to old >> generation is very low. >> >> Could something like that enable a low-overhead >> CloseableMappedByteBuffer? >> > If I read this correctly then this still depends on timely reference > processing. Right, the hope is that Reference(s) and referents that only live for a few seconds get this timely processing. > The switching guard trick to keep the Cleaner from being promoted is > interesting of course. So if we were to do something like this with > MBB then it would mean the file mapping would still exist and so we'd > have the usual issue of trying to delete the underlying file. There could be a method on MBB to register a callback to be called after the buffer is unmapped. > Then we have the issue of file or socket I/O where threads could be > blocked doing I/O with regions of the buffer and so delay indefinitely > any release. Yes, that could be a problem. If the access to mapped memory blocked for a very long time, then the guard and its Cleaner could be promoted to old gen in the meanwhile. But let me ask something. Doesn't GC processing require (at least in some phases) that user threads be stopped in a safepoint? What happens when a user thread is blocked by kernel on memory access? Such thread can't be stopped in a safepoint. Safepoint can't begin, so GC can't proceed and therefore can't promote objects to old generation at that time. Am I right? If yes, then time does not pass for objects while a user thread is blocked on memory access, so they won't get promoted to old gen any time sooner after the user thread is unblocked. Regards, Peter > > That said, maybe it is an option for Uwe. > > -Alan. From scolebourne at joda.org Mon Jan 25 15:31:14 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 25 Jan 2016 15:31:14 +0000 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: <56A63E50.5030007@oracle.com> References: <56A63E50.5030007@oracle.com> Message-ID: Typo "TimeUnitequivalent" Otherwise looks good. thanks Stephen On 25 January 2016 at 15:25, nadeesh tv wrote: > Hi all, > > Please review a fix for conversion between Chronounit and Timeunit > > Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 > > webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ > > -- > Thanks and Regards, > Nadeesh TV > > From chris.hegarty at oracle.com Mon Jan 25 15:38:17 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 25 Jan 2016 15:38:17 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options Message-ID: <56A64169.80908@oracle.com> JOpt Simple [1] is a Java library for parsing command line options. The JDK has several different home-grown versions of command line option-parsing code. Where possible, new and existing tools in the JDK should consider using JOpt Simple. JOpt Simple is being used in a number of tools in project jigsaw [2]. This issue proposes to integrate JOpt Simple into JDK 9 as a new module that can be used internally by tools in the JDK. The webrev proposes to add the new module, it is yet to be used. Qualified exports to specific JDK tool modules can be added later. http://cr.openjdk.java.net/~chegar/8148154/ -Chris. [1] https://pholser.github.io/jopt-simple/ [2] http://hg.openjdk.java.net/jigsaw/jake/jdk/file/tip/src/jdk.internal.opt/share/classes From claes.redestad at oracle.com Mon Jan 25 15:39:07 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 25 Jan 2016 16:39:07 +0100 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A64169.80908@oracle.com> References: <56A64169.80908@oracle.com> Message-ID: <56A6419B.7000901@oracle.com> +1 module-info.java missing? "jdk.internal.opt" might be ever so slightly misleading, wouldn't "jdk.internal.joptsimple" be a better name for the module? /Claes On 2016-01-25 16:38, Chris Hegarty wrote: > JOpt Simple [1] is a Java library for parsing command line options. > The JDK has several different home-grown versions of command line > option-parsing code. Where possible, new and existing tools in the > JDK should consider using JOpt Simple. JOpt Simple is being used in > a number of tools in project jigsaw [2]. > > This issue proposes to integrate JOpt Simple into JDK 9 as a new > module that can be used internally by tools in the JDK. The webrev > proposes to add the new module, it is yet to be used. Qualified > exports to specific JDK tool modules can be added later. > > http://cr.openjdk.java.net/~chegar/8148154/ > > -Chris. > > [1] https://pholser.github.io/jopt-simple/ > [2] > http://hg.openjdk.java.net/jigsaw/jake/jdk/file/tip/src/jdk.internal.opt/share/classes > From chris.hegarty at oracle.com Mon Jan 25 15:45:29 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 25 Jan 2016 15:45:29 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A6419B.7000901@oracle.com> References: <56A64169.80908@oracle.com> <56A6419B.7000901@oracle.com> Message-ID: <56A64319.6080609@oracle.com> On 25/01/16 15:39, Claes Redestad wrote: > +1 Thanks for looking at this Claes. > module-info.java missing? No. There is an update to modules.xml. This is all that is required in jdk9/dev, for now. > "jdk.internal.opt" might be ever so slightly misleading, wouldn't > "jdk.internal.joptsimple" be a better name for the module? For consistency, I took the same terse approach applied to the jdk.internal.le module. -Chris. > /Claes > > On 2016-01-25 16:38, Chris Hegarty wrote: >> JOpt Simple [1] is a Java library for parsing command line options. >> The JDK has several different home-grown versions of command line >> option-parsing code. Where possible, new and existing tools in the >> JDK should consider using JOpt Simple. JOpt Simple is being used in >> a number of tools in project jigsaw [2]. >> >> This issue proposes to integrate JOpt Simple into JDK 9 as a new >> module that can be used internally by tools in the JDK. The webrev >> proposes to add the new module, it is yet to be used. Qualified >> exports to specific JDK tool modules can be added later. >> >> http://cr.openjdk.java.net/~chegar/8148154/ >> >> -Chris. >> >> [1] https://pholser.github.io/jopt-simple/ >> [2] >> http://hg.openjdk.java.net/jigsaw/jake/jdk/file/tip/src/jdk.internal.opt/share/classes >> > From Roger.Riggs at Oracle.com Mon Jan 25 15:45:25 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jan 2016 10:45:25 -0500 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A6419B.7000901@oracle.com> References: <56A64169.80908@oracle.com> <56A6419B.7000901@oracle.com> Message-ID: <56A64315.2050104@Oracle.com> Hi Chris, Are there any tests that can be included? Roger On 1/25/2016 10:39 AM, Claes Redestad wrote: > +1 > > module-info.java missing? > > "jdk.internal.opt" might be ever so slightly misleading, wouldn't > "jdk.internal.joptsimple" be a better name for the module? > > /Claes > > On 2016-01-25 16:38, Chris Hegarty wrote: >> JOpt Simple [1] is a Java library for parsing command line options. >> The JDK has several different home-grown versions of command line >> option-parsing code. Where possible, new and existing tools in the >> JDK should consider using JOpt Simple. JOpt Simple is being used in >> a number of tools in project jigsaw [2]. >> >> This issue proposes to integrate JOpt Simple into JDK 9 as a new >> module that can be used internally by tools in the JDK. The webrev >> proposes to add the new module, it is yet to be used. Qualified >> exports to specific JDK tool modules can be added later. >> >> http://cr.openjdk.java.net/~chegar/8148154/ >> >> -Chris. >> >> [1] https://pholser.github.io/jopt-simple/ >> [2] >> http://hg.openjdk.java.net/jigsaw/jake/jdk/file/tip/src/jdk.internal.opt/share/classes >> > From Alan.Bateman at oracle.com Mon Jan 25 15:47:33 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jan 2016 15:47:33 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A64169.80908@oracle.com> References: <56A64169.80908@oracle.com> Message-ID: <56A64395.3040402@oracle.com> On 25/01/2016 15:38, Chris Hegarty wrote: > JOpt Simple [1] is a Java library for parsing command line options. > The JDK has several different home-grown versions of command line > option-parsing code. Where possible, new and existing tools in the > JDK should consider using JOpt Simple. JOpt Simple is being used in > a number of tools in project jigsaw [2]. > > This issue proposes to integrate JOpt Simple into JDK 9 as a new > module that can be used internally by tools in the JDK. The webrev > proposes to add the new module, it is yet to be used. Qualified > exports to specific JDK tool modules can be added later. > > http://cr.openjdk.java.net/~chegar/8148154/ Bringing this into jdk9/dev in the advance looks good. One thing, should we add a README so that we know exactly which version of JOpt Simple this is? -Alan From Roger.Riggs at Oracle.com Mon Jan 25 15:53:23 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jan 2016 10:53:23 -0500 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A64319.6080609@oracle.com> References: <56A64169.80908@oracle.com> <56A6419B.7000901@oracle.com> <56A64319.6080609@oracle.com> Message-ID: <56A644F3.30004@Oracle.com> Hi Chris, On 1/25/2016 10:45 AM, Chris Hegarty wrote: > On 25/01/16 15:39, Claes Redestad wrote: > >> "jdk.internal.opt" might be ever so slightly misleading, wouldn't >> "jdk.internal.joptsimple" be a better name for the module? > > For consistency, I took the same terse approach applied to the > jdk.internal.le module. More explicit is better; I think 'le' was/is too terse to be informative. and a simple 'opt' is too vague. $.02, Roger > > -Chris. > > >> /Claes >> >> On 2016-01-25 16:38, Chris Hegarty wrote: >>> JOpt Simple [1] is a Java library for parsing command line options. >>> The JDK has several different home-grown versions of command line >>> option-parsing code. Where possible, new and existing tools in the >>> JDK should consider using JOpt Simple. JOpt Simple is being used in >>> a number of tools in project jigsaw [2]. >>> >>> This issue proposes to integrate JOpt Simple into JDK 9 as a new >>> module that can be used internally by tools in the JDK. The webrev >>> proposes to add the new module, it is yet to be used. Qualified >>> exports to specific JDK tool modules can be added later. >>> >>> http://cr.openjdk.java.net/~chegar/8148154/ >>> >>> -Chris. >>> >>> [1] https://pholser.github.io/jopt-simple/ >>> [2] >>> http://hg.openjdk.java.net/jigsaw/jake/jdk/file/tip/src/jdk.internal.opt/share/classes >>> >>> >> From Alan.Bateman at oracle.com Mon Jan 25 16:03:13 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jan 2016 16:03:13 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A64319.6080609@oracle.com> References: <56A64169.80908@oracle.com> <56A6419B.7000901@oracle.com> <56A64319.6080609@oracle.com> Message-ID: <56A64741.3040109@oracle.com> On 25/01/2016 15:45, Chris Hegarty wrote: > : > >> "jdk.internal.opt" might be ever so slightly misleading, wouldn't >> "jdk.internal.joptsimple" be a better name for the module? > > For consistency, I took the same terse approach applied to the > jdk.internal.le module. Right, and the module name is deliberately named because it is JDK-internal, it will not have any unconditional exports. -Alan. From nadeesh.tv at oracle.com Mon Jan 25 16:06:59 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Mon, 25 Jan 2016 21:36:59 +0530 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: References: <56A63E50.5030007@oracle.com> Message-ID: <56A64823.20200@oracle.com> Hi all, Please see the updated webrev http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ -- Thanks and Regards, Nadeesh TV On 1/25/2016 9:01 PM, Stephen Colebourne wrote: > Typo "TimeUnitequivalent" > Otherwise looks good. > thanks > Stephen > > > > On 25 January 2016 at 15:25, nadeesh tv wrote: > >> Hi all, >> >> Please review a fix for conversion between Chronounit and Timeunit >> >> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >> >> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >> >> -- >> Thanks and Regards, >> Nadeesh TV >> >> -- Thanks and Regards, Nadeesh TV From chris.hegarty at oracle.com Mon Jan 25 16:09:52 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 25 Jan 2016 16:09:52 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A64395.3040402@oracle.com> References: <56A64169.80908@oracle.com> <56A64395.3040402@oracle.com> Message-ID: <56A648D0.10804@oracle.com> On 25/01/16 15:47, Alan Bateman wrote: > > > On 25/01/2016 15:38, Chris Hegarty wrote: >> JOpt Simple [1] is a Java library for parsing command line options. >> The JDK has several different home-grown versions of command line >> option-parsing code. Where possible, new and existing tools in the >> JDK should consider using JOpt Simple. JOpt Simple is being used in >> a number of tools in project jigsaw [2]. >> >> This issue proposes to integrate JOpt Simple into JDK 9 as a new >> module that can be used internally by tools in the JDK. The webrev >> proposes to add the new module, it is yet to be used. Qualified >> exports to specific JDK tool modules can be added later. >> >> http://cr.openjdk.java.net/~chegar/8148154/ > Bringing this into jdk9/dev in the advance looks good. One thing, should > we add a README so that we know exactly which version of JOpt Simple > this is? Good idea Alan. Added and webrev updated in-place. -Chris. From chris.hegarty at oracle.com Mon Jan 25 16:11:11 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 25 Jan 2016 16:11:11 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A64315.2050104@Oracle.com> References: <56A64169.80908@oracle.com> <56A6419B.7000901@oracle.com> <56A64315.2050104@Oracle.com> Message-ID: <56A6491F.3070404@oracle.com> On 25/01/16 15:45, Roger Riggs wrote: > Hi Chris, > > Are there any tests that can be included? No, I am not planning to add specific tests for JOpt Simple. The tools that eventually use it, should provide sufficient testing. -Chris. > Roger > > > On 1/25/2016 10:39 AM, Claes Redestad wrote: >> +1 >> >> module-info.java missing? >> >> "jdk.internal.opt" might be ever so slightly misleading, wouldn't >> "jdk.internal.joptsimple" be a better name for the module? >> >> /Claes >> >> On 2016-01-25 16:38, Chris Hegarty wrote: >>> JOpt Simple [1] is a Java library for parsing command line options. >>> The JDK has several different home-grown versions of command line >>> option-parsing code. Where possible, new and existing tools in the >>> JDK should consider using JOpt Simple. JOpt Simple is being used in >>> a number of tools in project jigsaw [2]. >>> >>> This issue proposes to integrate JOpt Simple into JDK 9 as a new >>> module that can be used internally by tools in the JDK. The webrev >>> proposes to add the new module, it is yet to be used. Qualified >>> exports to specific JDK tool modules can be added later. >>> >>> http://cr.openjdk.java.net/~chegar/8148154/ >>> >>> -Chris. >>> >>> [1] https://pholser.github.io/jopt-simple/ >>> [2] >>> http://hg.openjdk.java.net/jigsaw/jake/jdk/file/tip/src/jdk.internal.opt/share/classes >>> >> > From Alan.Bateman at oracle.com Mon Jan 25 16:14:26 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jan 2016 16:14:26 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A63EE1.4080100@gmail.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> <56A50588.20000@gmail.com> <56A623B9.9090305@oracle.com> <56A63EE1.4080100@gmail.com> Message-ID: <56A649E2.6020002@oracle.com> On 25/01/2016 15:27, Peter Levart wrote: > > But let me ask something. Doesn't GC processing require (at least in > some phases) that user threads be stopped in a safepoint? What happens > when a user thread is blocked by kernel on memory access? Such thread > can't be stopped in a safepoint. Safepoint can't begin, so GC can't > proceed and therefore can't promote objects to old generation at that > time. Am I right? If yes, then time does not pass for objects while a > user thread is blocked on memory access, so they won't get promoted to > old gen any time sooner after the user thread is unblocked. With memory mapping then it's very possible that a memory access to stall waiting for pages to be faulted in. So yes, I assume any STW/safepoint is going to be stall. On the other hand then threads doing file or socket I/O on these buffers will be in native (maybe blocked, maybe not) and so aren't an issue. They will safepoint on return or if they attempt to re-enter. -Alan From Alan.Bateman at oracle.com Mon Jan 25 16:16:26 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jan 2016 16:16:26 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A648D0.10804@oracle.com> References: <56A64169.80908@oracle.com> <56A64395.3040402@oracle.com> <56A648D0.10804@oracle.com> Message-ID: <56A64A5A.8030405@oracle.com> On 25/01/2016 16:09, Chris Hegarty wrote: > Bringing this into jdk9/dev in the advance looks good. One thing, should >> we add a README so that we know exactly which version of JOpt Simple >> this is? > > Good idea Alan. Added and webrev updated in-place. Thanks, that should make it easy to identify. We need to make sure there is a bug tracking an update to the THIRD_PARTY_README too. -Alan. From chris.hegarty at oracle.com Mon Jan 25 16:32:46 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 25 Jan 2016 16:32:46 +0000 Subject: RFR [9] 8148154: JOpt Simple - a Java library for parsing command line options In-Reply-To: <56A64A5A.8030405@oracle.com> References: <56A64169.80908@oracle.com> <56A64395.3040402@oracle.com> <56A648D0.10804@oracle.com> <56A64A5A.8030405@oracle.com> Message-ID: <56A64E2E.6060402@oracle.com> On 25/01/16 16:16, Alan Bateman wrote: > > > On 25/01/2016 16:09, Chris Hegarty wrote: >> Bringing this into jdk9/dev in the advance looks good. One thing, should >>> we add a README so that we know exactly which version of JOpt Simple >>> this is? >> >> Good idea Alan. Added and webrev updated in-place. > Thanks, that should make it easy to identify. Thanks. > We need to make sure there > is a bug tracking an update to the THIRD_PARTY_README too. Filed: https://bugs.openjdk.java.net/browse/JDK-8148164 -Chris. From Alan.Bateman at oracle.com Mon Jan 25 16:54:05 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 25 Jan 2016 16:54:05 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> Message-ID: <56A6532D.6050805@oracle.com> On 22/01/2016 23:10, Steve Drach wrote: > Hi Alan, et. al., > > I?ve released a new webrev that addresses all the issues you raised. > > http://cr.openjdk.java.net/~sdrach/8132734/webrev.03/index.html > > > Specifically: > >> For Release then I have to admit that I dislike _9 and wonder if >> other options were considered? javax.lang.model.SourceVersion uses >> the RELEASE_xx convention for example. > > Changed to VERSION_9, i.e. Release.VERSION_9 > >> >> Also I wonder about Release.ROOT and whether Release.UNVERSIONED was >> considered? In general the phrase "root entry" in the javadoc makes >> me think the root or top-most directory. An alternative that might be >> clearer is to say "unversioned entry" and define that term clearly in >> the class description. > > Changed to BASE, i.e. Release.BASE > This looks better. Release.BASE is probably okay although it still feels like Release.UNVERSIONED, esp. when it is defined as "Represents unversioned entries". I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? -Alan. From Roger.Riggs at Oracle.com Mon Jan 25 16:54:00 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 25 Jan 2016 11:54:00 -0500 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: <56A64823.20200@oracle.com> References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> Message-ID: <56A65328.1080207@Oracle.com> Hi Stephen, Nadeesh, TimeUnit.toChronoUnit is a static method. It seems redundant to have to pass an instance to a static method of its type. cu = TimeUnit.toChronoUnit(TimeUnit.SECONDS); Instead of: TimeUnit tu = TimeUnit.SECONDS; ChronoUnit cu = tu.toChronoUnit(); Minor edits please: in @param and @return use the type name when referring to the type. For example, TimeUnit vs timeUnit (the parameter). in @throws, use the parameter name instead of "the unit"; For example, + * @throws IllegalArgumentException if timeUnit cannot be converted Thanks, Roger On 1/25/2016 11:06 AM, nadeesh tv wrote: > Hi all, > > Please see the updated webrev > http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ > > -- > Thanks and Regards, > Nadeesh TV > > > On 1/25/2016 9:01 PM, Stephen Colebourne wrote: >> Typo "TimeUnitequivalent" >> Otherwise looks good. >> thanks >> Stephen >> >> >> >> On 25 January 2016 at 15:25, nadeesh tv wrote: >> >>> Hi all, >>> >>> Please review a fix for conversion between Chronounit and Timeunit >>> >>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >>> >>> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> >>> > From xueming.shen at oracle.com Mon Jan 25 17:47:03 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 25 Jan 2016 09:47:03 -0800 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <56A3F349.5070500@oracle.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> Message-ID: <56A65F97.4080500@oracle.com> The proposed change looks fine with me. sherman On 1/23/16 1:40 PM, Roger Riggs wrote: > Hi, > > Looks good. > > I would want to verify that it is defined such that if it later needs > to be > abstracted up to ChronoLocalDate and apply to compatible Chronologies > and respective local dates such as JapaneseDate there is no conflict. > It should not be an issue since Period implements ChronoPeriod. > In the respective implementations, the estimation/computation of the > number of steps > would need to depend on the Chronology's definition of months. > > I can sponsor this enhancement. > > Thanks, Roger > > > > On 1/20/16 10:15 AM, Stephen Colebourne wrote: >> I'm happy with the logic and specification of this proposal. I think it >> will be a useful addition. >> >> I'll let the Oracle team chime in to do a further review. >> >> thanks >> Stephen >> >> >> >> On 16 January 2016 at 13:31, Tagir F. Valeev wrote: >> >>> Hello! >>> >>> Thanks for review! Here's the updated patch: >>> http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ >>> >>> SC> The docs say that if the end date is before the start date, the >>> SC> stream is empty. While I can see just about why LongStream.range() >>> SC> works that way, I don't think this API should. The minimum is an >>> SC> exception, but it would be easy to support negative in the >>> SC> days-only case or the months-only case. The problem is where there >>> SC> are both months/years and days and those should be exceptions. >>> >>> Now datesUntil(endExclusive) throws an IllegalArgumentException if end >>> date is before start date. >>> >>> datesUntil(endExclusive, step) supports negative periods. It throws >>> IllegalArgumentException if: >>> - step is zero >>> - step.toTotalMonths() and step.getDays() both non-zero and have >>> opposite sign >>> - step is negative and end date is after start date >>> - step is positive and end date is before start date >>> >>> Otherwise it works correctly: you can use >>> LocalDate.of(2016, 1, 1) >>> .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); >>> >>> Steps like Period.of(-1, -1, -1) are also supported. >>> >>> SC> The single-arg implementation uses plusDays() with an >>> SC> incrementing number. When the performance patch goes in, the >>> SC> proposed streaming implementation will be optimal for small >>> SC> streams but less optimal for large ones. The fastest way to loop >>> SC> over a list of dates would be to manually generate them by >>> SC> incrementing the day until it exceeds the length of month, and so >>> SC> on. However, this would be serial. >>> >>> As I understand, plusDays performance patch is already pushed. >>> >>> It's possible to write custom Spliterator which would use >>> previous.plusDays(1) in tryAdvance() and jump (via >>> LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is >>> requested. However this adds at least one additional class and more >>> complexity. I guess, such optimization can be considered later as >>> separate issue when API is stabilized. >>> >>> SC> As such, I think the best way to write this, taking account of >>> SC> how plusDays() is implemented, is as follows: >>> >>> SC> LongStream.range(start.toEpochDay(), >>> SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); >>> >>> Ok, now it's done this way. >>> >>> SC> In the period-based method, it would be best to capture the case >>> SC> where totalMonths == 0 and days > 0 and forward to another method >>> SC> that ignores months. That private method can share implementation >>> SC> with the public single-arg method (passing in 1). >>> >>> This case still more complex than one-day case as it requires division >>> and multiplication. Thus I'd keep these case separately. However I >>> simplified "months = 0" path using ofEpochDay, now it should be >>> faster. >>> >>> SC> The docs for the period-based method should specify the strategy >>> SC> that produces the results (in abstract terms). This needs to cover >>> SC> that the result is equivalent to always adding to the start period >>> SC> a multiple of the period. >>> >>> I added some clarifications, please check. >>> >>> SC> Some nits: >>> SC> I prefer to avoid @link in the first sentence. Just using 'stream' >>> would be sufficient. >>> >>> Done. >>> >>> SC> The first sentence should be a summary. In this case it probably >>> has a >>> bit too much detail. >>> >>> Done. >>> >>> SC> The @return has 'values' on a new line when it could be on the same >>> line. >>> >>> I set now line length = 100 characters in my IDE. Is it ok? >>> >>> SC> If statements need braces. >>> >>> Done. >>> >>> With best regards, >>> Tagir Valeev. >>> >>> > From spliterator at gmail.com Mon Jan 25 18:14:50 2016 From: spliterator at gmail.com (Stefan Zobel) Date: Mon, 25 Jan 2016 19:14:50 +0100 Subject: Stream.limit parallel ordered performance In-Reply-To: <919184407.20160124173651@gmail.com> References: <919184407.20160124173651@gmail.com> Message-ID: 2016-01-24 12:36 GMT+01:00 Tagir F. Valeev : > Hello! > > I'm investigating Stream.limit() performance for parallel ordered > streams when no SUBSIZED optimization is performed. > > Consider the following code, for example: > > AtomicInteger counter = new AtomicInteger(); > int[] result = IntStream.range(0, 1_000_000).parallel().filter(x -> true) > .peek(x -> counter.incrementAndGet()).limit(10).toArray(); > System.out.println(Arrays.toString(result)); > System.out.println(counter.get()); > > How much the counter.get() would print? It changes from launch to > launch, but usually within 375000..625000. This is just insane. On my > 4-core system parallel stream creates 16 individual tasks. I expect > that every individual task should consume no more than 10 elements, so > in total no more than 160 elements should be consumed in this case. > > Here's a patch which addresses this issue: > http://cr.openjdk.java.net/~tvaleev/patches/limit/limit-patch.txt > > In the limit case non-root leaf tasks may switch to copyIntoWithCancel > to control the count of consumed elements and do not consume more than > necessary. > > This change seems to fix the issue addressed in comment (at least > partially): > > // @@@ OOMEs will occur for LongStream.range(0, Long.MAX_VALUE).filter(i -> true).limit(n) > // regardless of the value of n > // Need to adjust the target size of splitting for the > // SliceTask from say (size / k) to say min(size / k, 1 << 14) > // This will limit the size of the buffers created at the leaf nodes > // cancellation will be more aggressive cancelling later tasks > // if the target slice size has been reached from a given task, > // cancellation should also clear local results if any > > I checked with the following code: > > for(int n : new int[] {10, 100, 1000, 5000, 10000, 50000, 100000, 500000, 1000000}) { > System.out.println(n); > long[] arr = LongStream.range(0, Long.MAX_VALUE).filter(i -> true).parallel().limit(n).toArray(); > long[] ref = LongStream.range(0, n).toArray(); > System.out.println(Arrays.equals(arr, ref)); > } > > It works correctly after applying my patch (while dies with OOME > without patch, as comment suggests). > > Currently existing unit tests also pass with my patch. However I'm > very new in the internals of parallel stream processing, so it's > possible that I'm missing something. Please review! If this looks > reasonable I will log an issue and write new test cases. > > Thank you in advance, > With best regards, > Tagir Valeev. > Hi Tagir, I'm not an expert. To me, your suggestion appears to be very sensible. Regards, Stefan From rafael.wth at gmail.com Mon Jan 25 19:24:01 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Mon, 25 Jan 2016 20:24:01 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> References: <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Vladmir, hello Remi, what bothers me about instrumenting a lambda expression's target method is the difficulty of locating the method that contains the code and setting it into context. This is a lot of work to do since one needs to first locate any invokedynamic call sites and interpret the connection via the LambdaMetafactory. This is difficult to instrument without greater efforts and does not feel like a clean solution. Many real life instrumentations are based on the assumption of being able to instrument any class of a given subtype or other any other class property. For example, one would want to instrument any class that implements InputStream to discover a resource leak. If the interface is however functional and implemented as a lambda expression, a tool that manipulating all class files of this type stops working. At the same time, end users do not really understand the difference of their former anonymous class that is now expressed a lambda expression and perceive the beaviour as a regression. I have myself implemented a custom solution for my code generation library that instruments the LambdaMetafactory to apply class file transformers to these classes manually. This does however show two disadvantages: 1. javac creates the target method of a lambda expression as a private method. The only way to implement a lambda expression legally in byte code is by loading the generated implementation anonymously what requires non-public API. While it is possible to call Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, it is again required to use non-standardized APIs which might fail on differing implementations of the JVM. 2. At the same time, there is no standardized way to receive all ClassFileTransformers that are currently registered on the VM. this also requires calls into internal APIs that are not necessarily supported on other platforms. I fully understand the hesitation to support this from a technical point of view but in reality, people are already dependant on this feature and disallowing the instrumentation of lambda classes will only inspire work-arrounds that reduce the stability of software using this APIs that does currently work without problems for non-lambda classes. Best regards, Rafael PS: While implementing my solution, I found that the LambdaMetafactory of the Open JDK creates private, final methods for the serialization bits. The methods should however not be final as they are already private. 2016-01-23 13:55 GMT+01:00 Remi Forax : > I agree with Vladimir, > > You should not be able to transform/redefine a VM anonymous class because the transformer will certainly mess up with the order of the constant pool entries. > > Slightly off-topic, about ASM, when you create a ClassWriter [1], you can pass a ClassReader of an existing class, in that case ASM copy the constant pool from the class reader to the class writer so the constant pool is preserved. > > R?mi > > [1] http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 > > ----- Mail original ----- >> De: "Vladimir Ivanov" >> ?: "Rafael Winterhalter" >> Cc: "Coleen Phillimore" , core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com >> Spitsyn" , "Daniel Daugherty" >> Envoy?: Vendredi 22 Janvier 2016 18:47:31 >> Objet: Re: ClassFileTransformer does not apply to anonymous classes >> >> Rafael, >> >> First of all, I'd like to agree on the terminology. There's some >> confusion there. Anonymous term is ambiguous in Java. There are >> anonymous classes on language level and there's >> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense >> classes. I prefer to call them VM anonymous classes to avoid confusion. >> >> I assume that whenever you use anonymous you assume "VM anonymous". >> >> > thank you for your response. While I completely understand your view >> > from a VM implementor's point of view, as a practicioner I would >> > recommend against it. Not being able to instrument lambda expressions >> > puts a severe limitation onto using the instrumentation API >> > alltogether. For example, a monitoring application that promises to >> > record all invocations of a method of a certain interface that only >> > works 95% of the time is not 5% less usefull but might no longer be >> > useful at all. People have build large applications based on the >> > assumption that all user application code can be instrumented and such >> > a regression would hurt them. For example, until today, over 30 people >> > that use my code generation framework reached out to me and reported >> > the reported behavior as a bug in my library and people are only >> > starting to migrate their applications to Java 8. The outcome is >> > simply not intuitive as using a lambda expression over an anonyous >> > inner class should not change an application's behavior. >> Can you elaborate on that point, please? >> >> I don't see any problems with instrumenting user code. Javac represents >> lambda expression body as a private static method of the enclosing >> class, which can be instrumented. VM anonymous classes are used only as >> a mechanism to glue functional interfaces and lambda expressions >> together at runtime. >> >> For example: >> static void f(Runnable r) { r.run(); } >> f(() -> {}); >> >> Test::f (7 bytes) >> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) >> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) >> >> Why do you need to instrument Test$$Lambda$1/... and not >> Test::lambda$main$0? >> >> > The currently used workaround that people use is to instrument the >> > LambdaMetaFactory class itself to apply the transformer manually when >> > it is being created. This solution is spreading as a standard approach >> > and I am sure that forbidding a redefinition alltogether would only >> > inspire to other workarrounds for being able to migrate running code >> > to the next Java version. >> It doesn't sound like a workaround, but as a solution for a different >> problem. >> >> If there's a need to gather profile for every lambda expression >> instantiation site (indy call), then redefining bootstrap method is the >> right way to go IMO. It allows to intercept and customize indy call site >> binding. >> >> What does sound as a workaround is an attempt to instrument classes >> produced by LambdaMetaFactory. Right now, it generates a fresh VM >> anonymous class on every request, but it is an implementation detail and >> not a requirement. >> >> For example, LambdaMetaFactory can reuse wrappers on per-functional >> interface basis. It would provide significant footprint savings for >> lambda expression-rich code bases (usually, there are much more >> instantiations than functional interface flavors). >> >> > Furthermore, I do not think that not being able to patch constant pool >> > indices in the generated code introduces any problems in practice. >> > Most real-world instrumentation only appends new constant pool entries >> > without interfering with the existant pool. Rather, I would like to >> > see an exception being raised if one attempts to change the original >> > constant pool without me being able to consider this from a technical >> > perspective. I think this would serve to be sufficient for most >> > people. >> If you care only about lambda expressions, then constant pool patching >> shouldn't be a problem (AFAIR it isn't used for lambda expressions). But >> in a more generic case (even for java.lang.invoke purposes), the >> coupling between class file and CP patches is very tight and can be >> easily broken with benign class file transformations. Considering ASM >> library, I'm not sure that even simple instrumentations preserve >> original constant pool structure. >> >> Probably, VM can do some structural checks on CP to forbid any >> modifications except appends, but there are still aliasing problems - >> sharing doesn't work (even for string constants), so new code shouldn't >> use original CP entries. >> >> I'd prefer to avoid such increase in complexity in VM implementation. >> >> > I really hope that there is a way to allow for patching anonymously >> > loaded classes even without exposing the constant pool patches. >> From VM implementation perspective I don't see any problems with >> allowing class file redefinition for VM anonymous classes. What I want >> to do is to discourage from doing so. Again - it is very fragile and >> there's little we can do to fix that. >> >> > So far, thank you for looking into this. I am sure this a more complex >> > matter than what I am able to comprehend. I appreciate that you are >> > taking the time to consider my opinion! >> >> Best regards, >> Vladimir Ivanov >> From gil at azul.com Mon Jan 25 19:32:56 2016 From: gil at azul.com (Gil Tene) Date: Mon, 25 Jan 2016 19:32:56 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A50588.20000@gmail.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> <56A50588.20000@gmail.com> Message-ID: I assume your goal here is to get the resources released with the next newgen collections (following a close()), rather than wait for an oldgen (if the resource was held by an old object). That's a cool thing. With that in mind, you can replace the repeated periodic polling/flipping/allocation and external calling changeGuard() with a simple internal GC-detecor that would call changeGuard() allocate a new guard only once per newgen GC cycle. This can take the form of adding a simple GCDetector inside your implementation: private class GCDetector { @Override protected void finalize() throws Throwable { GCDetector detector = new GCDetector(); changeGuard(); Reference.reachabilityFence(detector); } } // The reason to use finalize here instead of a phantom ref // based cleaner is that it would trigger immediately after the cycle, // rather than potentially take an extra cycle to trigger. // This can be done with a weakRef based cleaner instead // (but probably when one is added to the JDK, otherwise you'd // need your own polling thread and logic here...). You'll need to allocate a single instance of GCDetector during construction of a CloseableMemory, without retaining a reference to it after construction. This will start a finalize-triggering chain per instance, with the chain "ticking" once per newgen cycle. If you want to avoid having one of these (coming and going on each GC cycle) per CloseableMemory instance, you can use a common static detector and a registration mechanism (where each registered instance would have it's changeGuard() method call? ? Gil. > On Jan 24, 2016, at 9:10 AM, Peter Levart wrote: > > Hi, > > I had an idea recently on how to expedite the collection of an object. It is simple - just don't let it live long. > > Here's a concept prototype: > > http://cr.openjdk.java.net/~plevart/misc/CloseableMemory/CloseableMemory.java > > The overhead of the check in access methods (getByte()/setByte()) amounts to one volatile read of an oop variable that changes once per say 5 to 10 seconds. That's the period a special guard object is alive. It's reachability is tracked by the GC and extends to the end of each access method (using Reference.reachabilityFence). Every few seconds, the guard object is changed with new fresh one so that the chance of the guard and its tracking Cleaner being promoted to old generation is very low. > > Could something like that enable a low-overhead CloseableMappedByteBuffer? > > Regards, Peter > > On 01/23/2016 09:31 PM, Andrew Haley wrote: >> On 23/01/16 20:01, Uwe Schindler wrote: >> >>> It depends how small! If the speed is still somewhere between Java 8 >>> ByteBuffer performance and the recent Hotspot improvements in Java >>> 9, I agree with trying it out. But some volatile memory access on >>> every access is a no-go. The code around ByteBufferIndexInput in >>> Lucene is the most performance-critical critical code, because on >>> every search query or sorting there is all the work happening in >>> there (millions of iterations with positional ByteBuffer.get* >>> calls). As ByteBuffers are limited to 2 GiB, we also need lots of >>> hairy code to work around that limitation! >> Yes, I see that code. It would be helpful if there were a >> self-contained but realistic benchmark using that code. That way, >> some simple experiments would allow changes to be measured. >> >>> If you look at ByteBufferIndexInput's code you will see that we >>> simply do stuff like trying to read from one bytebuffer and only if >>> we catch an BufferUnderflowException we fall back to handling buffer >>> switches: Instead of checking bounds on every access, we have >>> fallback code only happening on exceptions. E.g. if you are 3 bytes >>> before end of one buffer slice and read a long, it will throw >>> BufferUnderflow. When this happens the code will fall back to read >>> byte by byte from 2 different buffers and reassemble the long): >> I'm surprised you don't see painful deoptimization traps when that >> happens. I suppose it's rare enough that you don't care. There's a >> new group of methods in JDK9 called Objects.checkIndex() which are >> intended to provide a very efficient way to do bounds checks. It might >> be interesting to see if they work well with ByteBufferIndexInput: >> that's an important use case. >> >> BTW, does anyone here know why we don't have humongous ByteBuffers >> with a long index? >> >> Andrew. > From martinrb at google.com Mon Jan 25 21:42:35 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jan 2016 13:42:35 -0800 Subject: API review of VarHandles In-Reply-To: <56A22D69.7040404@cs.oswego.edu> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> <56A22D69.7040404@cs.oswego.edu> Message-ID: On Fri, Jan 22, 2016 at 5:23 AM, Doug Lea

    wrote: > On 01/22/2016 04:51 AM, Andrew Haley wrote: >> >> On 22/01/16 00:01, Vitaly Davidovich wrote: >>> >>> I think the get/setOpaque methods need a bit more explanation ("opaque" >>> is >>> an odd naming choice, IMO). Specifically, it says the operations are >>> done >>> in program order but have no effect on inter-thread ordering/visibility. >>> Is this spec verbiage for a full compiler-only fence? > > > > Basically yes. It would be the same as C "volatile" (not C++/C11 > "atomic"), if C volatile had a memory model spec. But in any case > we can't use "volatile" for this in Java because it already means > something else. It would be nice if we could actually say this in the (non-normative part of the) javadoc. >> >> It's like C++ memory_order_relaxed, I guessed. I understand that but >> not "opaque". >> > > The new name is required because opaque is only "like" C++ > memory_order_relaxed. > See the jmm-dev list discussions last year for details, but annoyingly, > Java normal variables and C++ memory_order_relaxed are not quite the > same, and neither are the same as C-volatile/opaque. Could we make atomicity guarantees clear for every method? When can we be sure not to get OOTA values? From vitalyd at gmail.com Mon Jan 25 21:52:25 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 25 Jan 2016 16:52:25 -0500 Subject: API review of VarHandles In-Reply-To: <56A22D69.7040404@cs.oswego.edu> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <56A1FBA1.9010108@redhat.com> <56A22D69.7040404@cs.oswego.edu> Message-ID: Hi Doug, I somehow missed your reply and just now noticed it after Martin's last email quoted it -- thanks for elaborating. The new name is required because opaque is only "like" C++ > memory_order_relaxed. > See the jmm-dev list discussions last year for details, but annoyingly, > Java normal variables and C++ memory_order_relaxed are not quite the > same, and neither are the same as C-volatile/opaque. AFAICT, difference between java normal load/store and memory_order_relaxed is the latter guaranteeing atomicity. If I'm understanding you correctly, "opaque" is *just* compiler barrier and does not guarantee atomicity? If that's so, what is/are the use case(s) for opaque? Andrew mentioned preventing roach motel movement of loads/stores into the motel. Is there something more than that? Assuming my above statement is correct, it would seem like having memory_order_relaxed would be more useful than simply "no code motion around this load/store", or if not more useful, just useful on its own; there's plenty of code that doesn't care about ordering but would not tolerate torn reads/stores. On Fri, Jan 22, 2016 at 8:23 AM, Doug Lea
    wrote: > On 01/22/2016 04:51 AM, Andrew Haley wrote: > >> On 22/01/16 00:01, Vitaly Davidovich wrote: >> >>> I think the get/setOpaque methods need a bit more explanation ("opaque" >>> is >>> an odd naming choice, IMO). Specifically, it says the operations are >>> done >>> in program order but have no effect on inter-thread ordering/visibility. >>> Is this spec verbiage for a full compiler-only fence? >>> >> > > Basically yes. It would be the same as C "volatile" (not C++/C11 > "atomic"), if C volatile had a memory model spec. But in any case > we can't use "volatile" for this in Java because it already means > something else. > > >> It's like C++ memory_order_relaxed, I guessed. I understand that but >> not "opaque". >> >> > The new name is required because opaque is only "like" C++ > memory_order_relaxed. > See the jmm-dev list discussions last year for details, but annoyingly, > Java normal variables and C++ memory_order_relaxed are not quite the > same, and neither are the same as C-volatile/opaque. > > -Doug > > > > From martinrb at google.com Mon Jan 25 22:08:41 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jan 2016 14:08:41 -0800 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: This is my first attempt to understand java.lang.invoke. It seems like a new alien world, non-java-like. I'll need to rebuild my internal performance model of java code, perhaps by staring at jit compiled code. I expect new APIs for atomic variable access to appear in j.u.c.atomic, not j.l.invoke. Aleksey has made atomic updaters fast. Can we continue on that road, providing more updaters or more methods on current updaters? Sequential consistency is much easier than release/acquire, which is in turn much easier than load/load store/store. (using the latter is almost always a mistake?) Memory ordering combined with variables is easier than independent fences. I continue to be uneasy about handing out the sharpest knives in public APIs. We're not providing much in the way of scare text to get users to think twice. From christoph.langer at sap.com Mon Jan 25 23:49:27 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Mon, 25 Jan 2016 23:49:27 +0000 Subject: Regarding the removal of com/sun/org/apache/xalan/internal/xslt/EnvironmentCheck.java Message-ID: <0DFD2E72402C9243A8630A7759B27E4338FC742E@DEWDFEMB12B.global.corp.sap> Hi, as I was using the xslt EnvironmentCheck class in my private testcase and recognizing it didn't exist anymore in JDK 9, I stumbled over this thread from last July below. I understand that you want to clean up and remove unnecessary code but I found the class quite useful as it would some up several (to me seemingly) interesting jaxp version information. The other Version.java class was removed, too. Was the information collected by these classes really that crappy that it could just be removed? Where would I get my XML version information from now? Thanks and best regards Christoph On 7/28/2015 10:32 AM, Daniel Fuchs wrote: > On 28/07/15 19:20, huizhe wang wrote: >> Hi Daniel, >> >> >> On 7/28/2015 8:22 AM, Daniel Fuchs wrote: >>> Hi, >>> >>> Please find below a fix for yet another cleanup for jaxp: >> >> Thanks for yet another cleanup! And, there is a lot more to come :-) >>> >>> 8130059: jaxp: Investigate removal of >>> com/sun/org/apache/xalan/internal/xslt/EnvironmentCheck.java >>> https://bugs.openjdk.java.net/browse/JDK-8130059 >>> >>> http://cr.openjdk.java.net/~dfuchs/webrev_8130059/webrev.00/ >>> >>> EnvironmentCheck doesn't seem to serve any purpose in JDK 9. >> >> Agree. It'd be a confusion more than anything else if used since it >> produces many irrelevant information. >> >>> It is not called anywhere. The proposal is to remove it. >>> By doing a full grep on the JDK I also identified another >>> unused class (Hashtree2Node.java) which referred to EnvironmentCheck >>> inside a comment. >>> I took the liberty to remove that class as well. >> >> Ok. The webrev looks good to me. If you'd want to remove the two >> Version classes as shown in the test, that would be fine with me too. >> Then you could remove the whole test. > > Thanks Joe! > > One of the two version classes (xalan) is used by ...xslt.Process.java > I already logged another bug to investigate removing that as > well (JDK-8130058). Great! > > Maybe we should remove the two versions classes as part of that > other bug? Or I could update my webrev to just remove the xerces > Version.java now, which as far as I can see is not called anywhere. > > As you prefer :-) I agree as you planned, considering removing the two version classes in JDK-8130058. Cheers :-) Joe > > cheers, > > -- daniel > >> >> Best regards, >> Joe >> >>> >>> As for the latter cleanup, what triggered this is that EnvironmentCheck >>> is using sun.boot.class.path... >>> >>> best regards, >>> >>> -- daniel >> > From huizhe.wang at oracle.com Tue Jan 26 00:15:42 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 25 Jan 2016 16:15:42 -0800 Subject: Regarding the removal of com/sun/org/apache/xalan/internal/xslt/EnvironmentCheck.java In-Reply-To: <0DFD2E72402C9243A8630A7759B27E4338FC742E@DEWDFEMB12B.global.corp.sap> References: <0DFD2E72402C9243A8630A7759B27E4338FC742E@DEWDFEMB12B.global.corp.sap> Message-ID: <56A6BAAE.3080400@oracle.com> Hi Christoph, What did you use from the class? As in the discussion, many of the information are either inaccurate or irrelevant in the JDK (vs as a standalone impl). Thanks, Joe On 1/25/2016 3:49 PM, Langer, Christoph wrote: > Hi, > > as I was using the xslt EnvironmentCheck class in my private testcase and recognizing it didn't exist anymore in JDK 9, I stumbled over this thread from last July below. > > I understand that you want to clean up and remove unnecessary code but I found the class quite useful as it would some up several (to me seemingly) interesting jaxp version information. The other Version.java class was removed, too. Was the information collected by these classes really that crappy that it could just be removed? Where would I get my XML version information from now? > > Thanks and best regards > Christoph > > > On 7/28/2015 10:32 AM, Daniel Fuchs wrote: > >/ On 28/07/15 19:20, huizhe wang wrote:/ > >>/ Hi Daniel,/ > >>/ / > >>/ / > >>/ On 7/28/2015 8:22 AM, Daniel Fuchs wrote:/ > >>>/ Hi,/ > >>>/ / > >>>/ Please find below a fix for yet another cleanup for jaxp:/ > >>/ / > >>/ Thanks for yet another cleanup! And, there is a lot more to come :-)/ > >>>/ / > >>>/ 8130059: jaxp: Investigate removal of/ > >>>/ com/sun/org/apache/xalan/internal/xslt/EnvironmentCheck.java/ > >>>/ //https://bugs.openjdk.java.net/browse/JDK-8130059/// > >>>/ / > >>>/ //http://cr.openjdk.java.net/~dfuchs/webrev_8130059/webrev.00/ /// > >>>/ / > >>>/ EnvironmentCheck doesn't seem to serve any purpose in JDK 9./ > >>/ / > >>/ Agree. It'd be a confusion more than anything else if used since it/ > >>/ produces many irrelevant information./ > >>/ / > >>>/ It is not called anywhere. The proposal is to remove it./ > >>>/ By doing a full grep on the JDK I also identified another/ > >>>/ unused class (Hashtree2Node.java) which referred to EnvironmentCheck/ > >>>/ inside a comment./ > >>>/ I took the liberty to remove that class as well./ > >>/ / > >>/ Ok. The webrev looks good to me. If you'd want to remove the two/ > >>/ Version classes as shown in the test, that would be fine with me too./ > >>/ Then you could remove the whole test./ > >/ / > >/ Thanks Joe!/ > >/ / > >/ One of the two version classes (xalan) is used by ...xslt.Process.java/ > >/ I already logged another bug to investigate removing that as/ > >/ well (JDK-8130058)./ > > Great! > >/ / > >/ Maybe we should remove the two versions classes as part of that/ > >/ other bug? Or I could update my webrev to just remove the xerces/ > >/ Version.java now, which as far as I can see is not called anywhere./ > >/ / > >/ As you prefer :-)/ > > I agree as you planned, considering removing the two version classes in > JDK-8130058. > > Cheers :-) > > Joe > > >/ / > >/ cheers,/ > >/ / > >/ -- daniel/ > >/ / > >>/ / > >>/ Best regards,/ > >>/ Joe/ > >>/ / > >>>/ / > >>>/ As for the latter cleanup, what triggered this is that EnvironmentCheck/ > >>>/ is using sun.boot.class.path.../ > >>>/ / > >>>/ best regards,/ > >>>/ / > >>>/ -- daniel/ > >>/ / > > > From martinrb at google.com Tue Jan 26 00:49:41 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 25 Jan 2016 16:49:41 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: <56A2AD9D.70802@oracle.com> References: <56A133FC.4040806@oracle.com> <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> <56A2AD9D.70802@oracle.com> Message-ID: I already regret touching crufty old Vector, but it had the same bug to fix, and more surprisingly the same sort of performance improvement from doing the obvious port. So ... Stuart, please review also http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Vector-grow/ https://bugs.openjdk.java.net/browse/JDK-8148174 On Fri, Jan 22, 2016 at 2:30 PM, Stuart Marks wrote: > > > On 1/22/16 12:02 PM, Martin Buchholz wrote: >> >> I went "by the book" as you suggested and now throw >> InvalidObjectException when size < 0. >> (But I've been saying for a decade: if we're serious about >> Serialization, it needs to be someone's full time job) > > > "Admiral, if we go by the book, years could turn into decades." > > Thanks for the update; this looks great to me. > > s'marks From mikael.vidstedt at oracle.com Tue Jan 26 04:57:09 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Mon, 25 Jan 2016 20:57:09 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <565628F5.2080801@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> Message-ID: <56A6FCA5.7040808@oracle.com> I've finally found some time to return to this and have a new version of the patch which looks more promising: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.02/webrev/ This uses memcpy to read/write the native data and the preliminary benchmark numbers on linux/x64 shows the expected performance. I'll work on verifying that it doesn't impact other platforms negatively over the next days/weeks. Btw, I also played around with implementing it in pure Java, but there are some issues getting C2 to correctly vectorize the loop given the native data accesses, so the native code seems to be needed for now. Cheers, Mikael On 2015-11-25 13:32, Mikael Vidstedt wrote: > > Have you looked anything at the performance of the generated code? As > you may have seen I was playing around with an alternative > implementation[1] which has the benefit of being pure C++ without > compiler specific hints. That said, when I did some initial > benchmarking of that it did seem like the performance impact was > significant. I didn't have time to look at more in detail why, and > will not have time to return to that until late next week earliest. It > would be interesting to understand what type of performance you see > with your patch. > > Cheers, > Mikael > > > [1] > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.01/webrev/jdk.patch > > On 2015-11-25 11:42, Coleen Phillimore wrote: >> Sending to core-libs mailing list. >> >> On 11/25/15 2:19 PM, Alexander Smundak wrote: >>> Please take a look at >>> http://cr.openjdk.java.net/~asmundak/8141491/jdk/webrev.00 >>> that fixes the problem. >>> >>> It utilizes the ability of some (GCC and Clang) to declare data >>> alignment explicitly. >>> I have verified it works on x86_64 Linux by running >>> jdk/test/java/nio/Buffer/Basic.java test >>> >>> I need a sponsor. >>> >>> Sasha >> > From huaming.li at oracle.com Tue Jan 26 07:08:26 2016 From: huaming.li at oracle.com (Hamlin Li) Date: Tue, 26 Jan 2016 15:08:26 +0800 Subject: RFR: 8076458: java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout Message-ID: <56A71B6A.9090605@oracle.com> Hi everyone, Would you please help to review the fix for bug https://bugs.openjdk.java.net/browse/JDK-8076458, java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout. webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.00/ Thank you -Hamlin From rafael.wth at gmail.com Tue Jan 26 08:39:33 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Tue, 26 Jan 2016 09:39:33 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> Message-ID: Another note on this subject: I found that applying a reretransformation on a lambda expression's class does not provide the original class file to the retransformer but the class file that resulted from a previous retransformation. (If a lambda class is retransformed several times.) This is different to "normal" classes where the original class file is provided. 2016-01-25 20:24 GMT+01:00 Rafael Winterhalter : > Hi Vladmir, hello Remi, > > what bothers me about instrumenting a lambda expression's target > method is the difficulty of locating the method that contains the code > and setting it into context. This is a lot of work to do since one > needs to first locate any invokedynamic call sites and interpret the > connection via the LambdaMetafactory. This is difficult to instrument > without greater efforts and does not feel like a clean solution. > > Many real life instrumentations are based on the assumption of being > able to instrument any class of a given subtype or other any other > class property. For example, one would want to instrument any class > that implements InputStream to discover a resource leak. If the > interface is however functional and implemented as a lambda > expression, a tool that manipulating all class files of this type > stops working. At the same time, end users do not really understand > the difference of their former anonymous class that is now expressed a > lambda expression and perceive the beaviour as a regression. > > I have myself implemented a custom solution for my code generation > library that instruments the LambdaMetafactory to apply class file > transformers to these classes manually. This does however show two > disadvantages: > > 1. javac creates the target method of a lambda expression as a private > method. The only way to implement a lambda expression legally in byte > code is by loading the generated implementation anonymously what > requires non-public API. While it is possible to call > Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, > it is again required to use non-standardized APIs which might fail on > differing implementations of the JVM. > > 2. At the same time, there is no standardized way to receive all > ClassFileTransformers that are currently registered on the VM. this > also requires calls into internal APIs that are not necessarily > supported on other platforms. > > I fully understand the hesitation to support this from a technical > point of view but in reality, people are already dependant on this > feature and disallowing the instrumentation of lambda classes will > only inspire work-arrounds that reduce the stability of software using > this APIs that does currently work without problems for non-lambda > classes. > > Best regards, Rafael > > PS: While implementing my solution, I found that the LambdaMetafactory > of the Open JDK creates private, final methods for the serialization > bits. The methods should however not be final as they are already > private. > > > 2016-01-23 13:55 GMT+01:00 Remi Forax : >> I agree with Vladimir, >> >> You should not be able to transform/redefine a VM anonymous class because the transformer will certainly mess up with the order of the constant pool entries. >> >> Slightly off-topic, about ASM, when you create a ClassWriter [1], you can pass a ClassReader of an existing class, in that case ASM copy the constant pool from the class reader to the class writer so the constant pool is preserved. >> >> R?mi >> >> [1] http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 >> >> ----- Mail original ----- >>> De: "Vladimir Ivanov" >>> ?: "Rafael Winterhalter" >>> Cc: "Coleen Phillimore" , core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com >>> Spitsyn" , "Daniel Daugherty" >>> Envoy?: Vendredi 22 Janvier 2016 18:47:31 >>> Objet: Re: ClassFileTransformer does not apply to anonymous classes >>> >>> Rafael, >>> >>> First of all, I'd like to agree on the terminology. There's some >>> confusion there. Anonymous term is ambiguous in Java. There are >>> anonymous classes on language level and there's >>> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense >>> classes. I prefer to call them VM anonymous classes to avoid confusion. >>> >>> I assume that whenever you use anonymous you assume "VM anonymous". >>> >>> > thank you for your response. While I completely understand your view >>> > from a VM implementor's point of view, as a practicioner I would >>> > recommend against it. Not being able to instrument lambda expressions >>> > puts a severe limitation onto using the instrumentation API >>> > alltogether. For example, a monitoring application that promises to >>> > record all invocations of a method of a certain interface that only >>> > works 95% of the time is not 5% less usefull but might no longer be >>> > useful at all. People have build large applications based on the >>> > assumption that all user application code can be instrumented and such >>> > a regression would hurt them. For example, until today, over 30 people >>> > that use my code generation framework reached out to me and reported >>> > the reported behavior as a bug in my library and people are only >>> > starting to migrate their applications to Java 8. The outcome is >>> > simply not intuitive as using a lambda expression over an anonyous >>> > inner class should not change an application's behavior. >>> Can you elaborate on that point, please? >>> >>> I don't see any problems with instrumenting user code. Javac represents >>> lambda expression body as a private static method of the enclosing >>> class, which can be instrumented. VM anonymous classes are used only as >>> a mechanism to glue functional interfaces and lambda expressions >>> together at runtime. >>> >>> For example: >>> static void f(Runnable r) { r.run(); } >>> f(() -> {}); >>> >>> Test::f (7 bytes) >>> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) >>> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) >>> >>> Why do you need to instrument Test$$Lambda$1/... and not >>> Test::lambda$main$0? >>> >>> > The currently used workaround that people use is to instrument the >>> > LambdaMetaFactory class itself to apply the transformer manually when >>> > it is being created. This solution is spreading as a standard approach >>> > and I am sure that forbidding a redefinition alltogether would only >>> > inspire to other workarrounds for being able to migrate running code >>> > to the next Java version. >>> It doesn't sound like a workaround, but as a solution for a different >>> problem. >>> >>> If there's a need to gather profile for every lambda expression >>> instantiation site (indy call), then redefining bootstrap method is the >>> right way to go IMO. It allows to intercept and customize indy call site >>> binding. >>> >>> What does sound as a workaround is an attempt to instrument classes >>> produced by LambdaMetaFactory. Right now, it generates a fresh VM >>> anonymous class on every request, but it is an implementation detail and >>> not a requirement. >>> >>> For example, LambdaMetaFactory can reuse wrappers on per-functional >>> interface basis. It would provide significant footprint savings for >>> lambda expression-rich code bases (usually, there are much more >>> instantiations than functional interface flavors). >>> >>> > Furthermore, I do not think that not being able to patch constant pool >>> > indices in the generated code introduces any problems in practice. >>> > Most real-world instrumentation only appends new constant pool entries >>> > without interfering with the existant pool. Rather, I would like to >>> > see an exception being raised if one attempts to change the original >>> > constant pool without me being able to consider this from a technical >>> > perspective. I think this would serve to be sufficient for most >>> > people. >>> If you care only about lambda expressions, then constant pool patching >>> shouldn't be a problem (AFAIR it isn't used for lambda expressions). But >>> in a more generic case (even for java.lang.invoke purposes), the >>> coupling between class file and CP patches is very tight and can be >>> easily broken with benign class file transformations. Considering ASM >>> library, I'm not sure that even simple instrumentations preserve >>> original constant pool structure. >>> >>> Probably, VM can do some structural checks on CP to forbid any >>> modifications except appends, but there are still aliasing problems - >>> sharing doesn't work (even for string constants), so new code shouldn't >>> use original CP entries. >>> >>> I'd prefer to avoid such increase in complexity in VM implementation. >>> >>> > I really hope that there is a way to allow for patching anonymously >>> > loaded classes even without exposing the constant pool patches. >>> From VM implementation perspective I don't see any problems with >>> allowing class file redefinition for VM anonymous classes. What I want >>> to do is to discourage from doing so. Again - it is very fragile and >>> there's little we can do to fix that. >>> >>> > So far, thank you for looking into this. I am sure this a more complex >>> > matter than what I am able to comprehend. I appreciate that you are >>> > taking the time to consider my opinion! >>> >>> Best regards, >>> Vladimir Ivanov >>> From adinn at redhat.com Tue Jan 26 08:58:01 2016 From: adinn at redhat.com (Andrew Dinn) Date: Tue, 26 Jan 2016 08:58:01 +0000 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> Message-ID: <56A73519.5040500@redhat.com> On 26/01/16 08:39, Rafael Winterhalter wrote: > Another note on this subject: I found that applying a > reretransformation on a lambda expression's class does not provide the > original class file to the retransformer but the class file that > resulted from a previous retransformation. (If a lambda class is > retransformed several times.) This is different to "normal" classes > where the original class file is provided. I think it is also /clearly wrong/ since it makes it at very least difficult and perhaps, depending on circumstances, impossible for a transformer to restore the original bytecode. regards, Andrew Dinn ----------- From forax at univ-mlv.fr Tue Jan 26 09:02:57 2016 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 26 Jan 2016 10:02:57 +0100 (CET) Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> Message-ID: <147644520.594985.1453798977640.JavaMail.zimbra@u-pem.fr> Hello, ----- Mail original ----- > De: "Rafael Winterhalter" > ?: "Remi Forax" > Cc: "Vladimir Ivanov" , "Coleen Phillimore" , > core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com Spitsyn" , "Daniel > Daugherty" > Envoy?: Lundi 25 Janvier 2016 20:24:01 > Objet: Re: ClassFileTransformer does not apply to anonymous classes > > Hi Vladmir, hello Remi, > > what bothers me about instrumenting a lambda expression's target > method is the difficulty of locating the method that contains the code > and setting it into context. This is a lot of work to do since one > needs to first locate any invokedynamic call sites and interpret the > connection via the LambdaMetafactory. This is difficult to instrument > without greater efforts and does not feel like a clean solution. > > Many real life instrumentations are based on the assumption of being > able to instrument any class of a given subtype or other any other > class property. For example, one would want to instrument any class > that implements InputStream to discover a resource leak. If the > interface is however functional and implemented as a lambda > expression, a tool that manipulating all class files of this type > stops working. At the same time, end users do not really understand > the difference of their former anonymous class that is now expressed a > lambda expression and perceive the beaviour as a regression. The lambda body is analyzed because the body is translated to a static method (which can be non static BTW) which is part of the class that uses the lambda. The lambda proxy is a runtime artifact and the way it is currently implemented will likely to change in the future, using a vm anonymous class is a workaround of the fact that there is currently no way to load a function/method without a class. You should not try to analyze the lambda proxy, it doesn't contain codes but it's a straw glue between the interface and the body of the lambda. > > I have myself implemented a custom solution for my code generation > library that instruments the LambdaMetafactory to apply class file > transformers to these classes manually. This does however show two > disadvantages: > > 1. javac creates the target method of a lambda expression as a private > method. The only way to implement a lambda expression legally in byte > code is by loading the generated implementation anonymously what > requires non-public API. While it is possible to call > Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, > it is again required to use non-standardized APIs which might fail on > differing implementations of the JVM. The invokedynamic corresponding to a lambda creation contains a link to the private method, it's i believe all you need, but i'm not sure because what you're doing exactly is fuzy for me. > > 2. At the same time, there is no standardized way to receive all > ClassFileTransformers that are currently registered on the VM. this > also requires calls into internal APIs that are not necessarily > supported on other platforms. > > I fully understand the hesitation to support this from a technical > point of view but in reality, people are already dependant on this > feature and disallowing the instrumentation of lambda classes will > only inspire work-arrounds that reduce the stability of software using > this APIs that does currently work without problems for non-lambda > classes. again, there is no such thing as a lambda class, trying to see a lambda as an anonymous class will never fully work. a lambda is a lambda, you have the link between the functional interface and the lambda body encoded into the invokedynamic which is used to create the lambda, you should not try to get more information, because as i said, it may change. > > Best regards, Rafael > > PS: While implementing my solution, I found that the LambdaMetafactory > of the Open JDK creates private, final methods for the serialization > bits. The methods should however not be final as they are already > private. yes, private is enough but it's not a big deal. > regards, R?mi > > 2016-01-23 13:55 GMT+01:00 Remi Forax : > > I agree with Vladimir, > > > > You should not be able to transform/redefine a VM anonymous class because > > the transformer will certainly mess up with the order of the constant pool > > entries. > > > > Slightly off-topic, about ASM, when you create a ClassWriter [1], you can > > pass a ClassReader of an existing class, in that case ASM copy the > > constant pool from the class reader to the class writer so the constant > > pool is preserved. > > > > R?mi > > > > [1] > > http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 > > > > ----- Mail original ----- > >> De: "Vladimir Ivanov" > >> ?: "Rafael Winterhalter" > >> Cc: "Coleen Phillimore" , > >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com > >> Spitsyn" , "Daniel Daugherty" > >> > >> Envoy?: Vendredi 22 Janvier 2016 18:47:31 > >> Objet: Re: ClassFileTransformer does not apply to anonymous classes > >> > >> Rafael, > >> > >> First of all, I'd like to agree on the terminology. There's some > >> confusion there. Anonymous term is ambiguous in Java. There are > >> anonymous classes on language level and there's > >> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense > >> classes. I prefer to call them VM anonymous classes to avoid confusion. > >> > >> I assume that whenever you use anonymous you assume "VM anonymous". > >> > >> > thank you for your response. While I completely understand your view > >> > from a VM implementor's point of view, as a practicioner I would > >> > recommend against it. Not being able to instrument lambda expressions > >> > puts a severe limitation onto using the instrumentation API > >> > alltogether. For example, a monitoring application that promises to > >> > record all invocations of a method of a certain interface that only > >> > works 95% of the time is not 5% less usefull but might no longer be > >> > useful at all. People have build large applications based on the > >> > assumption that all user application code can be instrumented and such > >> > a regression would hurt them. For example, until today, over 30 people > >> > that use my code generation framework reached out to me and reported > >> > the reported behavior as a bug in my library and people are only > >> > starting to migrate their applications to Java 8. The outcome is > >> > simply not intuitive as using a lambda expression over an anonyous > >> > inner class should not change an application's behavior. > >> Can you elaborate on that point, please? > >> > >> I don't see any problems with instrumenting user code. Javac represents > >> lambda expression body as a private static method of the enclosing > >> class, which can be instrumented. VM anonymous classes are used only as > >> a mechanism to glue functional interfaces and lambda expressions > >> together at runtime. > >> > >> For example: > >> static void f(Runnable r) { r.run(); } > >> f(() -> {}); > >> > >> Test::f (7 bytes) > >> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) > >> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) > >> > >> Why do you need to instrument Test$$Lambda$1/... and not > >> Test::lambda$main$0? > >> > >> > The currently used workaround that people use is to instrument the > >> > LambdaMetaFactory class itself to apply the transformer manually when > >> > it is being created. This solution is spreading as a standard approach > >> > and I am sure that forbidding a redefinition alltogether would only > >> > inspire to other workarrounds for being able to migrate running code > >> > to the next Java version. > >> It doesn't sound like a workaround, but as a solution for a different > >> problem. > >> > >> If there's a need to gather profile for every lambda expression > >> instantiation site (indy call), then redefining bootstrap method is the > >> right way to go IMO. It allows to intercept and customize indy call site > >> binding. > >> > >> What does sound as a workaround is an attempt to instrument classes > >> produced by LambdaMetaFactory. Right now, it generates a fresh VM > >> anonymous class on every request, but it is an implementation detail and > >> not a requirement. > >> > >> For example, LambdaMetaFactory can reuse wrappers on per-functional > >> interface basis. It would provide significant footprint savings for > >> lambda expression-rich code bases (usually, there are much more > >> instantiations than functional interface flavors). > >> > >> > Furthermore, I do not think that not being able to patch constant pool > >> > indices in the generated code introduces any problems in practice. > >> > Most real-world instrumentation only appends new constant pool entries > >> > without interfering with the existant pool. Rather, I would like to > >> > see an exception being raised if one attempts to change the original > >> > constant pool without me being able to consider this from a technical > >> > perspective. I think this would serve to be sufficient for most > >> > people. > >> If you care only about lambda expressions, then constant pool patching > >> shouldn't be a problem (AFAIR it isn't used for lambda expressions). But > >> in a more generic case (even for java.lang.invoke purposes), the > >> coupling between class file and CP patches is very tight and can be > >> easily broken with benign class file transformations. Considering ASM > >> library, I'm not sure that even simple instrumentations preserve > >> original constant pool structure. > >> > >> Probably, VM can do some structural checks on CP to forbid any > >> modifications except appends, but there are still aliasing problems - > >> sharing doesn't work (even for string constants), so new code shouldn't > >> use original CP entries. > >> > >> I'd prefer to avoid such increase in complexity in VM implementation. > >> > >> > I really hope that there is a way to allow for patching anonymously > >> > loaded classes even without exposing the constant pool patches. > >> From VM implementation perspective I don't see any problems with > >> allowing class file redefinition for VM anonymous classes. What I want > >> to do is to discourage from doing so. Again - it is very fragile and > >> there's little we can do to fix that. > >> > >> > So far, thank you for looking into this. I am sure this a more complex > >> > matter than what I am able to comprehend. I appreciate that you are > >> > taking the time to consider my opinion! > >> > >> Best regards, > >> Vladimir Ivanov > >> > From aph at redhat.com Tue Jan 26 09:04:55 2016 From: aph at redhat.com (Andrew Haley) Date: Tue, 26 Jan 2016 09:04:55 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56A6FCA5.7040808@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> Message-ID: <56A736B7.4020601@redhat.com> On 26/01/16 04:57, Mikael Vidstedt wrote: > > I've finally found some time to return to this and have a new version of > the patch which looks more promising: > > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.02/webrev/ > > This uses memcpy to read/write the native data and the preliminary > benchmark numbers on linux/x64 shows the expected performance. I'll work > on verifying that it doesn't impact other platforms negatively over the > next days/weeks. I agree that memcpy is the right thing to use. It's portable and is inlined well on production-quality C compilers. Andrew. From rafael.wth at gmail.com Tue Jan 26 09:30:54 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Tue, 26 Jan 2016 10:30:54 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <147644520.594985.1453798977640.JavaMail.zimbra@u-pem.fr> References: <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> <147644520.594985.1453798977640.JavaMail.zimbra@u-pem.fr> Message-ID: Maybe this is a more graphic example of a problem that end-users currently face: http://stackoverflow.com/questions/33912026/intercepting-calls-to-java-8-lambda-expressions-using-byte-buddy 2016-01-26 10:02 GMT+01:00 : > Hello, > > ----- Mail original ----- >> De: "Rafael Winterhalter" >> ?: "Remi Forax" >> Cc: "Vladimir Ivanov" , "Coleen Phillimore" , >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com Spitsyn" , "Daniel >> Daugherty" >> Envoy?: Lundi 25 Janvier 2016 20:24:01 >> Objet: Re: ClassFileTransformer does not apply to anonymous classes >> >> Hi Vladmir, hello Remi, >> >> what bothers me about instrumenting a lambda expression's target >> method is the difficulty of locating the method that contains the code >> and setting it into context. This is a lot of work to do since one >> needs to first locate any invokedynamic call sites and interpret the >> connection via the LambdaMetafactory. This is difficult to instrument >> without greater efforts and does not feel like a clean solution. >> >> Many real life instrumentations are based on the assumption of being >> able to instrument any class of a given subtype or other any other >> class property. For example, one would want to instrument any class >> that implements InputStream to discover a resource leak. If the >> interface is however functional and implemented as a lambda >> expression, a tool that manipulating all class files of this type >> stops working. At the same time, end users do not really understand >> the difference of their former anonymous class that is now expressed a >> lambda expression and perceive the beaviour as a regression. > > The lambda body is analyzed because the body is translated to a static method (which can be non static BTW) which is part of the class that uses the lambda. > The lambda proxy is a runtime artifact and the way it is currently implemented will likely to change in the future, using a vm anonymous class is a workaround of the fact that there is currently no way to load a function/method without a class. You should not try to analyze the lambda proxy, it doesn't contain codes but it's a straw glue between the interface and the body of the lambda. > >> >> I have myself implemented a custom solution for my code generation >> library that instruments the LambdaMetafactory to apply class file >> transformers to these classes manually. This does however show two >> disadvantages: >> >> 1. javac creates the target method of a lambda expression as a private >> method. The only way to implement a lambda expression legally in byte >> code is by loading the generated implementation anonymously what >> requires non-public API. While it is possible to call >> Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, >> it is again required to use non-standardized APIs which might fail on >> differing implementations of the JVM. > > The invokedynamic corresponding to a lambda creation contains a link to the private method, it's i believe all you need, but i'm not sure because what you're doing exactly is fuzy for me. > >> >> 2. At the same time, there is no standardized way to receive all >> ClassFileTransformers that are currently registered on the VM. this >> also requires calls into internal APIs that are not necessarily >> supported on other platforms. >> >> I fully understand the hesitation to support this from a technical >> point of view but in reality, people are already dependant on this >> feature and disallowing the instrumentation of lambda classes will >> only inspire work-arrounds that reduce the stability of software using >> this APIs that does currently work without problems for non-lambda >> classes. > > again, there is no such thing as a lambda class, trying to see a lambda as an anonymous class will never fully work. > a lambda is a lambda, you have the link between the functional interface and the lambda body encoded into the invokedynamic which is used to create the lambda, you should not try to get more information, because as i said, it may change. > >> >> Best regards, Rafael >> >> PS: While implementing my solution, I found that the LambdaMetafactory >> of the Open JDK creates private, final methods for the serialization >> bits. The methods should however not be final as they are already >> private. > > yes, private is enough but it's not a big deal. > >> > > regards, > R?mi > >> >> 2016-01-23 13:55 GMT+01:00 Remi Forax : >> > I agree with Vladimir, >> > >> > You should not be able to transform/redefine a VM anonymous class because >> > the transformer will certainly mess up with the order of the constant pool >> > entries. >> > >> > Slightly off-topic, about ASM, when you create a ClassWriter [1], you can >> > pass a ClassReader of an existing class, in that case ASM copy the >> > constant pool from the class reader to the class writer so the constant >> > pool is preserved. >> > >> > R?mi >> > >> > [1] >> > http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 >> > >> > ----- Mail original ----- >> >> De: "Vladimir Ivanov" >> >> ?: "Rafael Winterhalter" >> >> Cc: "Coleen Phillimore" , >> >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com >> >> Spitsyn" , "Daniel Daugherty" >> >> >> >> Envoy?: Vendredi 22 Janvier 2016 18:47:31 >> >> Objet: Re: ClassFileTransformer does not apply to anonymous classes >> >> >> >> Rafael, >> >> >> >> First of all, I'd like to agree on the terminology. There's some >> >> confusion there. Anonymous term is ambiguous in Java. There are >> >> anonymous classes on language level and there's >> >> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense >> >> classes. I prefer to call them VM anonymous classes to avoid confusion. >> >> >> >> I assume that whenever you use anonymous you assume "VM anonymous". >> >> >> >> > thank you for your response. While I completely understand your view >> >> > from a VM implementor's point of view, as a practicioner I would >> >> > recommend against it. Not being able to instrument lambda expressions >> >> > puts a severe limitation onto using the instrumentation API >> >> > alltogether. For example, a monitoring application that promises to >> >> > record all invocations of a method of a certain interface that only >> >> > works 95% of the time is not 5% less usefull but might no longer be >> >> > useful at all. People have build large applications based on the >> >> > assumption that all user application code can be instrumented and such >> >> > a regression would hurt them. For example, until today, over 30 people >> >> > that use my code generation framework reached out to me and reported >> >> > the reported behavior as a bug in my library and people are only >> >> > starting to migrate their applications to Java 8. The outcome is >> >> > simply not intuitive as using a lambda expression over an anonyous >> >> > inner class should not change an application's behavior. >> >> Can you elaborate on that point, please? >> >> >> >> I don't see any problems with instrumenting user code. Javac represents >> >> lambda expression body as a private static method of the enclosing >> >> class, which can be instrumented. VM anonymous classes are used only as >> >> a mechanism to glue functional interfaces and lambda expressions >> >> together at runtime. >> >> >> >> For example: >> >> static void f(Runnable r) { r.run(); } >> >> f(() -> {}); >> >> >> >> Test::f (7 bytes) >> >> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) >> >> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) >> >> >> >> Why do you need to instrument Test$$Lambda$1/... and not >> >> Test::lambda$main$0? >> >> >> >> > The currently used workaround that people use is to instrument the >> >> > LambdaMetaFactory class itself to apply the transformer manually when >> >> > it is being created. This solution is spreading as a standard approach >> >> > and I am sure that forbidding a redefinition alltogether would only >> >> > inspire to other workarrounds for being able to migrate running code >> >> > to the next Java version. >> >> It doesn't sound like a workaround, but as a solution for a different >> >> problem. >> >> >> >> If there's a need to gather profile for every lambda expression >> >> instantiation site (indy call), then redefining bootstrap method is the >> >> right way to go IMO. It allows to intercept and customize indy call site >> >> binding. >> >> >> >> What does sound as a workaround is an attempt to instrument classes >> >> produced by LambdaMetaFactory. Right now, it generates a fresh VM >> >> anonymous class on every request, but it is an implementation detail and >> >> not a requirement. >> >> >> >> For example, LambdaMetaFactory can reuse wrappers on per-functional >> >> interface basis. It would provide significant footprint savings for >> >> lambda expression-rich code bases (usually, there are much more >> >> instantiations than functional interface flavors). >> >> >> >> > Furthermore, I do not think that not being able to patch constant pool >> >> > indices in the generated code introduces any problems in practice. >> >> > Most real-world instrumentation only appends new constant pool entries >> >> > without interfering with the existant pool. Rather, I would like to >> >> > see an exception being raised if one attempts to change the original >> >> > constant pool without me being able to consider this from a technical >> >> > perspective. I think this would serve to be sufficient for most >> >> > people. >> >> If you care only about lambda expressions, then constant pool patching >> >> shouldn't be a problem (AFAIR it isn't used for lambda expressions). But >> >> in a more generic case (even for java.lang.invoke purposes), the >> >> coupling between class file and CP patches is very tight and can be >> >> easily broken with benign class file transformations. Considering ASM >> >> library, I'm not sure that even simple instrumentations preserve >> >> original constant pool structure. >> >> >> >> Probably, VM can do some structural checks on CP to forbid any >> >> modifications except appends, but there are still aliasing problems - >> >> sharing doesn't work (even for string constants), so new code shouldn't >> >> use original CP entries. >> >> >> >> I'd prefer to avoid such increase in complexity in VM implementation. >> >> >> >> > I really hope that there is a way to allow for patching anonymously >> >> > loaded classes even without exposing the constant pool patches. >> >> From VM implementation perspective I don't see any problems with >> >> allowing class file redefinition for VM anonymous classes. What I want >> >> to do is to discourage from doing so. Again - it is very fragile and >> >> there's little we can do to fix that. >> >> >> >> > So far, thank you for looking into this. I am sure this a more complex >> >> > matter than what I am able to comprehend. I appreciate that you are >> >> > taking the time to consider my opinion! >> >> >> >> Best regards, >> >> Vladimir Ivanov >> >> >> From daniel.fuchs at oracle.com Tue Jan 26 09:30:43 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 26 Jan 2016 10:30:43 +0100 Subject: Regarding the removal of com/sun/org/apache/xalan/internal/xslt/EnvironmentCheck.java In-Reply-To: <0DFD2E72402C9243A8630A7759B27E4338FC742E@DEWDFEMB12B.global.corp.sap> References: <0DFD2E72402C9243A8630A7759B27E4338FC742E@DEWDFEMB12B.global.corp.sap> Message-ID: <56A73CC3.1020204@oracle.com> Hi Christoph, Instead of trying to access com/sun/org/apache/xalan/internal/xslt/EnvironmentCheck.java in your test - would it be possible for your test to embed that functionality instead? I am not sure how much sense that makes - the workings of that class seemed a bit convoluted - and not quite adapted to the JAXP implementation embedded in the JDK itself, but I was wondering whether you could apply the same technique than what we did when removing the internal xalan/xslt Process class which was not used either: http://cr.openjdk.java.net/~dfuchs/webrev_8130058/webrev.01/ If you really need EnvironmentCheck.java, then embedding it in your test is probably the better option, as it will allow you to 'fix it' to work with jdk9/jigsaw as well (e.g. using the jrt: File System instead of sun.boot.class.path etc...) best regard, -- daniel On 1/26/16 12:49 AM, Langer, Christoph wrote: > Hi, > > as I was using the xslt EnvironmentCheck class in my private testcase > and recognizing it didn?t exist anymore in JDK 9, I stumbled over this > thread from last July below. > > I understand that you want to clean up and remove unnecessary code but I > found the class quite useful as it would some up several (to me > seemingly) interesting jaxp version information. The other Version.java > class was removed, too. Was the information collected by these classes > really that crappy that it could just be removed? Where would I get my > XML version information from now? > > Thanks and best regards > > Christoph > > On 7/28/2015 10:32 AM, Daniel Fuchs wrote: > >>/On 28/07/15 19:20, huizhe wang wrote:/ > >>>/Hi Daniel,/ > >>>// > >>>// > >>>/On 7/28/2015 8:22 AM, Daniel Fuchs wrote:/ > >>>>/Hi,/ > >>>>// > >>>>/Please find below a fix for yet another cleanup for jaxp:/ > >>>// > >>>/Thanks for yet another cleanup! And, there is a lot more to come :-)/ > >>>>// > >>>>/8130059: jaxp: Investigate removal of/ > >>>>/com/sun/org/apache/xalan/internal/xslt/EnvironmentCheck.java/ > >>>>///https://bugs.openjdk.java.net/browse/JDK-8130059/// > >>>>// > >>>>///http://cr.openjdk.java.net/~dfuchs/webrev_8130059/webrev.00/ > /// > >>>>// > >>>>/EnvironmentCheck doesn't seem to serve any purpose in JDK 9./ > >>>// > >>>/Agree. It'd be a confusion more than anything else if used since it/ > >>>/produces many irrelevant information./ > >>>// > >>>>/It is not called anywhere. The proposal is to remove it./ > >>>>/By doing a full grep on the JDK I also identified another/ > >>>>/unused class (Hashtree2Node.java) which referred to EnvironmentCheck/ > >>>>/inside a comment./ > >>>>/I took the liberty to remove that class as well./ > >>>// > >>>/Ok. The webrev looks good to me. If you'd want to remove the two/ > >>>/Version classes as shown in the test, that would be fine with me too./ > >>>/Then you could remove the whole test./ > >>// > >>/Thanks Joe!/ > >>// > >>/One of the two version classes (xalan) is used by ...xslt.Process.java/ > >>/I already logged another bug to investigate removing that as/ > >>/well (JDK-8130058)./ > > Great! > >>// > >>/Maybe we should remove the two versions classes as part of that/ > >>/other bug? Or I could update my webrev to just remove the xerces/ > >>/Version.java now, which as far as I can see is not called anywhere./ > >>// > >>/As you prefer :-)/ > > I agree as you planned, considering removing the two version classes in > > JDK-8130058. > > Cheers :-) > > Joe > >>// > >>/cheers,/ > >>// > >>/-- daniel/ > >>// > >>>// > >>>/Best regards,/ > >>>/Joe/ > >>>// > >>>>// > >>>>/As for the latter cleanup, what triggered this is that EnvironmentCheck/ > >>>>/is using sun.boot.class.path.../ > >>>>// > >>>>/best regards,/ > >>>>// > >>>>/-- daniel/ > >>>// > >> > From paul.sandoz at oracle.com Tue Jan 26 10:38:47 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 11:38:47 +0100 Subject: RFR 8148115: Stream.findFirst for unordered source optimization In-Reply-To: <916767911.20160123184332@gmail.com> References: <1152739344.20160122121906@gmail.com> <30A68664-D525-41BF-9688-FAB7E1F6944A@oracle.com> <916767911.20160123184332@gmail.com> Message-ID: Hi Tagir, Looks good. Just small things. FindOps.java ? 293 if (!this.mustFindFirst) { 310 if (this.mustFindFirst) { Small thing, no need for ?this?. LambdaTestHelpers ? 404 public static void assertContains(Optional actual, Iterator it) { 405 if (actual.isPresent()) { 406 T r = actual.get(); 407 boolean contained = false; 408 while (!contained && it.hasNext()) { 409 contained = Objects.equals(r, it.next()); 410 } 411 assertTrue(contained); 412 } 413 else { 414 assertFalse(it.hasNext()); 415 } 416 } You can now use Optional.ifPresentOrElse, which was added in 9: ifPresentOrElse( t -> { ?}, () -> assertFalse(it.hasNext()): Up to you. Personally i prefer that to the isPresent then get. It potentially works better if there was also a non-optional receiving method to reduce the lambda size. ? I was wondering if the findFirst tests would require updating. For primitive streams, you are correct there are no unordered providers of data. in certain cases unordered is induced e.g. forEach testing. A focused fix would be to exercise with say s.filter().unordered(), a more general fix would be as you say to add another provider for int, long and double (backed by say HashSet, or more simply/efficiently using unordered()). We have to carefully review that because that may increase test times. We don?t have to apply to all data sets, plus we could also refine the ref data sets too in that regard, thus keep the overall actions similar or even smaller. Shall we separate that in a new issue? ? I will take this and other patches (close/limit etc) you propose in bulk. Thanks, Paul. > On 23 Jan 2016, at 13:43, Tagir F. Valeev wrote: > > Hello, Paul! > > Thank you for review. I implemented it according to your suggestions, > here's issue: > https://bugs.openjdk.java.net/browse/JDK-8148115 > And webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8148115/r1/ > > This patch required some changes in existing tests as FindFirstOpTest > asserted that findFirst() result must be equal to .iterator().next() > even if the source is HashSet. I extracted the relevant part of > FindAnyOpTest into assertContains(Optional, Iterator) method and > placed it into LambdaTestHelpers. > > The corresponding changes in primitive tests are unnecessary, because > it seems that primitive data providers never produce unordered > streams. I can try to add unordered primitive sources and correct > these tests as well if it's necessary. > > With best regards, > Tagir Valeev. > > > PS> Hi Tagir, > > PS> Thanks for looking at this. > > PS> I would prefer if a boolean was passed into the task constructor e.g.: > > PS> @Override > PS> public O evaluateParallel(PipelineHelper helper, > PS> Spliterator spliterator) { > PS> // This takes into account the upstream ops flags and the terminal > PS> // op flags and therefore takes into account findFirst or findAny > PS> boolean mustFindFirst = > PS> StreamOpFlag.ORDERED.isKnown(helper.getStreamAndOpFlags()); > PS> return new FindTask<>(this, mustFindFirst, helper, spliterator).invoke(); > PS> } > > PS> so we keep the flag evaluation logic within the evaluate method. > > PS> Then change FindOp.mustFindFirst to be an int of the opFlags calculated in the constructor: > > PS> this.opFlags = StreamOpFlag.IS_SHORT_CIRCUIT | (mustFindFirst ? 0 : StreamOpFlag.NOT_ORDERED); > > PS> Paul. > >>> On 22 Jan 2016, at 07:19, Tagir F. Valeev wrote: >>> >>> Hello! >>> >>> Seems that currently Stream.findFirst is not optimized for unordered >>> source. I think it should work as findAny in this case. Here's a small >>> patch which fixes this: >>> >>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/find_patch.txt >>> >>> Simple JMH test: >>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/FindTest.java >>> >>> Original: >>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_orig.txt >>> # JMH 1.11.2 (released 85 days ago) >>> # VM version: JDK 9-ea, VM 9-ea+99-2015-12-23-183325.javare.4146.nc >>> Benchmark Mode Cnt Score Error Units >>> FindTest.findAny avgt 30 12439,631 ? 1787,866 us/op >>> FindTest.findAnyUnordered avgt 30 12923,080 ? 1072,537 us/op >>> FindTest.findFirst avgt 30 48047,467 ? 2713,489 us/op >>> FindTest.findFirstUnordered avgt 30 52648,893 ? 3934,682 us/op >>> >>> Patched: >>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_patched.txt >>> Benchmark Mode Cnt Score Error Units >>> FindTest.findAny avgt 30 11312,238 ? 386,627 us/op >>> FindTest.findAnyUnordered avgt 30 12136,953 ? 1536,817 us/op >>> FindTest.findFirst avgt 30 47517,776 ? 2844,607 us/op >>> FindTest.findFirstUnordered avgt 30 13147,492 ? 1140,592 us/op >>> >>> If you think it's a reasonable thing to patch, I can log an issue, >>> generate webrev and check whether jtreg tests pass. >>> >>> With best regards, >>> Tagir Valeev. >>> > From paul.sandoz at oracle.com Tue Jan 26 10:52:30 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 11:52:30 +0100 Subject: RFR 8147505: Clarification of BaseStream.onClose() behavior In-Reply-To: <0886057.20160123163820@gmail.com> References: <1666489469.20160115103543@gmail.com> <68E50067-D3FB-414F-80AD-40F188DAE5FC@oracle.com> <1616962894.20160116091332@gmail.com> <0886057.20160123163820@gmail.com> Message-ID: <757F647A-DE7A-49A3-BC0F-938B4D0E8CD2@oracle.com> Hi Tagir, StreamCloseTest.java ? 181 try(Stream s = countTo(100).stream()) { 182 s.map(x -> x).forEach(i -> {}); 183 checkISE(() -> s.onClose(() -> fail("2"))); 184 } We don?t need this one, it?s redundant. The other performing the s.close() is i think useful because i don?t recall it being tested in other places like this, plus it also tests the idempotent behaviour, perhaps worth a comment on this in the test. Otherwise, +1 Again, i will handle in bulk with your other fixes. Many thanks, Paul. > On 23 Jan 2016, at 11:38, Tagir F. Valeev wrote: > > Hello! > > Here's a webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8147505/r1/ > > I just added a check in AbstractPipeline.onClose and the corresponding > unit-test. To me it seems that no documentation corrections are > necessary, because "onClose" is already documented that it's > an intermediate operation: > > http://download.java.net/jdk9/docs/api/java/util/stream/BaseStream.html#onClose-java.lang.Runnable- > > Also it's already documented: > >> A stream should be operated on (invoking an intermediate or terminal >> stream operation) only once. This rules out, for example, "forked" >> streams, where the same source feeds two or more pipelines, or >> multiple traversals of the same stream. A stream implementation may >> throw IllegalStateException if it detects that the stream is being >> reused. > > http://download.java.net/jdk9/docs/api/java/util/stream/Stream.html > > So this already implies that invoking intermediate operation "onClose" > after stream terminal operation is not allowed and may throw > IllegalStateException. > > Please review! > > With best regards, > Tagir Valeev. > > TFV> Seems that my original letter sounds confusing due to my bad English. > TFV> Every occurrence of "idempotent" should be read as "not idempotent". > TFV> Sorry for this! > > PS>> We did not bother to throw ISEs from parallel/sequence/onClose > PS>> primarily because they return ?this? and they are kind of > PS>> harmless, but the undefined behaviour you point out could be shut > PS>> down without much concern for backwards compatibility. We can > PS>> modify them to check whether they have been linked or consumed > PS>> (while not updating the previous stage). > > PS>> Wanna log an issue? > > TFV> Sure: > TFV> https://bugs.openjdk.java.net/browse/JDK-8147505 > > TFV> Thanks, > TFV> Tagir Valeev. > > PS>> Thanks, > PS>> Paul. > > >>>> On 15 Jan 2016, at 05:35, Tagir F. Valeev wrote: >>>> >>>> Hello! >>>> >>>> Current documentation for BaseStream.onClose() does not specify >>>> explicitly how the stream would behave if additional close handlers >>>> are registered after the stream is consumed or even closed. The >>>> implementation actually allows registering the additional close >>>> handlers after consumption or closure and close() method is >>>> idempotent: newly registered handlers are perfectly called: >>>> >>>> Stream s = Stream.of("content"); >>>> s = s.onClose(() -> System.out.println("A")); >>>> s.forEach(System.out::println); >>>> s = s.onClose(() -> System.out.println("B")); >>>> s.close(); // prints A and B >>>> s = s.onClose(() -> System.out.println("C")); >>>> s.close(); // prints C >>>> >>>> (removing "s =" produces the same result). >>>> >>>> I think if such behavior is intended, it should be explicitly noted in >>>> the specification, as third-party implementations of BaseStream >>>> interface should provide consistent behavior. Or at least it should be >>>> noted that some BaseStream implementations may have idempotent close() >>>> method, so the derived AutoCloseable objects (which own the BaseStream >>>> object) should be aware about this behavior and provide idempotent >>>> close() method as well. Without knowing this one may write: >>>> >>>> class MyObject implements AutoCloseable { >>>> private BaseStream s; >>>> >>>> public MyObject(BaseStream s) { >>>> this.s = s; >>>> } >>>> >>>> @Override >>>> public void close() throws Exception { >>>> if(s != null) { >>>> s.close(); >>>> s = null; >>>> } >>>> } >>>> } >>>> >>>> Such code closes the stream only once, so newly registered handlers >>>> will not be called if MyObject::close is called the second time. >>>> >>>> However, to my opinion the current behavior is weird and should be >>>> changed in order not to allow registering new close handles (throwing >>>> IllegalStateException) when the stream is already closed, or even >>>> better when the stream is linked/consumed. As far as I know currently >>>> in JDK close handlers are registered only for non-consumed stream, so >>>> such change would not break existing JDK code (though may break some >>>> strange third party code). It should be noted that AutoCloseable >>>> interface discourages idempotent close() methods (though not forbids >>>> them). >>>> >>>> What do you think? >>>> >>>> With best regards, >>>> Tagir Valeev. >>>> > From paul.sandoz at oracle.com Tue Jan 26 11:15:51 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 12:15:51 +0100 Subject: Stream.limit parallel ordered performance In-Reply-To: <919184407.20160124173651@gmail.com> References: <919184407.20160124173651@gmail.com> Message-ID: Hi Tagir, It is insane :-) in hindsight i cannot believe i missed this trick! You patch looks reasonable and i don?t see any issue with it. Each leaf-task will collect at most n elements. The truncation will take care of everything else. Note that it is still the case that in almost all scenarios this is likely to be a bad form of parallel stream. I think the comment you refer to still applies but now for larger n, so we should refine it. Paul. > On 24 Jan 2016, at 12:36, Tagir F. Valeev wrote: > > Hello! > > I'm investigating Stream.limit() performance for parallel ordered > streams when no SUBSIZED optimization is performed. > > Consider the following code, for example: > > AtomicInteger counter = new AtomicInteger(); > int[] result = IntStream.range(0, 1_000_000).parallel().filter(x -> true) > .peek(x -> counter.incrementAndGet()).limit(10).toArray(); > System.out.println(Arrays.toString(result)); > System.out.println(counter.get()); > > How much the counter.get() would print? It changes from launch to > launch, but usually within 375000..625000. This is just insane. On my > 4-core system parallel stream creates 16 individual tasks. I expect > that every individual task should consume no more than 10 elements, so > in total no more than 160 elements should be consumed in this case. > > Here's a patch which addresses this issue: > http://cr.openjdk.java.net/~tvaleev/patches/limit/limit-patch.txt > > In the limit case non-root leaf tasks may switch to copyIntoWithCancel > to control the count of consumed elements and do not consume more than > necessary. > > This change seems to fix the issue addressed in comment (at least > partially): > > // @@@ OOMEs will occur for LongStream.range(0, Long.MAX_VALUE).filter(i -> true).limit(n) > // regardless of the value of n > // Need to adjust the target size of splitting for the > // SliceTask from say (size / k) to say min(size / k, 1 << 14) > // This will limit the size of the buffers created at the leaf nodes > // cancellation will be more aggressive cancelling later tasks > // if the target slice size has been reached from a given task, > // cancellation should also clear local results if any > > I checked with the following code: > > for(int n : new int[] {10, 100, 1000, 5000, 10000, 50000, 100000, 500000, 1000000}) { > System.out.println(n); > long[] arr = LongStream.range(0, Long.MAX_VALUE).filter(i -> true).parallel().limit(n).toArray(); > long[] ref = LongStream.range(0, n).toArray(); > System.out.println(Arrays.equals(arr, ref)); > } > > It works correctly after applying my patch (while dies with OOME > without patch, as comment suggests). > > Currently existing unit tests also pass with my patch. However I'm > very new in the internals of parallel stream processing, so it's > possible that I'm missing something. Please review! If this looks > reasonable I will log an issue and write new test cases. > > Thank you in advance, > With best regards, > Tagir Valeev. > From forax at univ-mlv.fr Tue Jan 26 12:03:10 2016 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Tue, 26 Jan 2016 13:03:10 +0100 (CET) Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> <147644520.594985.1453798977640.JavaMail.zimbra@u-pem.fr> Message-ID: <710812902.735970.1453809790007.JavaMail.zimbra@u-pem.fr> Hi Rafael, why not providing a LambdaInterceptor as asked ? My point is that a lambda should be considered as a feature by itself instead as a way to create a class that implement an interface at runtime. If you do that, your code will still work when future release (maybe a point release), the implementation of the lambda proxy class will change. BTW, i maybe wrong, but as far as remember, the first release of Java 8 was not using vm anonymous classes for lambdas. regards, R?mi ----- Mail original ----- > De: "Rafael Winterhalter" > ?: "R?mi Forax" > Cc: "Vladimir Ivanov" , "Coleen Phillimore" , > core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com Spitsyn" , "Daniel > Daugherty" > Envoy?: Mardi 26 Janvier 2016 10:30:54 > Objet: Re: ClassFileTransformer does not apply to anonymous classes > > Maybe this is a more graphic example of a problem that end-users > currently face: > http://stackoverflow.com/questions/33912026/intercepting-calls-to-java-8-lambda-expressions-using-byte-buddy > > 2016-01-26 10:02 GMT+01:00 : > > Hello, > > > > ----- Mail original ----- > >> De: "Rafael Winterhalter" > >> ?: "Remi Forax" > >> Cc: "Vladimir Ivanov" , "Coleen Phillimore" > >> , > >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com Spitsyn" > >> , "Daniel > >> Daugherty" > >> Envoy?: Lundi 25 Janvier 2016 20:24:01 > >> Objet: Re: ClassFileTransformer does not apply to anonymous classes > >> > >> Hi Vladmir, hello Remi, > >> > >> what bothers me about instrumenting a lambda expression's target > >> method is the difficulty of locating the method that contains the code > >> and setting it into context. This is a lot of work to do since one > >> needs to first locate any invokedynamic call sites and interpret the > >> connection via the LambdaMetafactory. This is difficult to instrument > >> without greater efforts and does not feel like a clean solution. > >> > >> Many real life instrumentations are based on the assumption of being > >> able to instrument any class of a given subtype or other any other > >> class property. For example, one would want to instrument any class > >> that implements InputStream to discover a resource leak. If the > >> interface is however functional and implemented as a lambda > >> expression, a tool that manipulating all class files of this type > >> stops working. At the same time, end users do not really understand > >> the difference of their former anonymous class that is now expressed a > >> lambda expression and perceive the beaviour as a regression. > > > > The lambda body is analyzed because the body is translated to a static > > method (which can be non static BTW) which is part of the class that uses > > the lambda. > > The lambda proxy is a runtime artifact and the way it is currently > > implemented will likely to change in the future, using a vm anonymous > > class is a workaround of the fact that there is currently no way to load a > > function/method without a class. You should not try to analyze the lambda > > proxy, it doesn't contain codes but it's a straw glue between the > > interface and the body of the lambda. > > > >> > >> I have myself implemented a custom solution for my code generation > >> library that instruments the LambdaMetafactory to apply class file > >> transformers to these classes manually. This does however show two > >> disadvantages: > >> > >> 1. javac creates the target method of a lambda expression as a private > >> method. The only way to implement a lambda expression legally in byte > >> code is by loading the generated implementation anonymously what > >> requires non-public API. While it is possible to call > >> Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, > >> it is again required to use non-standardized APIs which might fail on > >> differing implementations of the JVM. > > > > The invokedynamic corresponding to a lambda creation contains a link to the > > private method, it's i believe all you need, but i'm not sure because what > > you're doing exactly is fuzy for me. > > > >> > >> 2. At the same time, there is no standardized way to receive all > >> ClassFileTransformers that are currently registered on the VM. this > >> also requires calls into internal APIs that are not necessarily > >> supported on other platforms. > >> > >> I fully understand the hesitation to support this from a technical > >> point of view but in reality, people are already dependant on this > >> feature and disallowing the instrumentation of lambda classes will > >> only inspire work-arrounds that reduce the stability of software using > >> this APIs that does currently work without problems for non-lambda > >> classes. > > > > again, there is no such thing as a lambda class, trying to see a lambda as > > an anonymous class will never fully work. > > a lambda is a lambda, you have the link between the functional interface > > and the lambda body encoded into the invokedynamic which is used to create > > the lambda, you should not try to get more information, because as i said, > > it may change. > > > >> > >> Best regards, Rafael > >> > >> PS: While implementing my solution, I found that the LambdaMetafactory > >> of the Open JDK creates private, final methods for the serialization > >> bits. The methods should however not be final as they are already > >> private. > > > > yes, private is enough but it's not a big deal. > > > >> > > > > regards, > > R?mi > > > >> > >> 2016-01-23 13:55 GMT+01:00 Remi Forax : > >> > I agree with Vladimir, > >> > > >> > You should not be able to transform/redefine a VM anonymous class > >> > because > >> > the transformer will certainly mess up with the order of the constant > >> > pool > >> > entries. > >> > > >> > Slightly off-topic, about ASM, when you create a ClassWriter [1], you > >> > can > >> > pass a ClassReader of an existing class, in that case ASM copy the > >> > constant pool from the class reader to the class writer so the constant > >> > pool is preserved. > >> > > >> > R?mi > >> > > >> > [1] > >> > http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 > >> > > >> > ----- Mail original ----- > >> >> De: "Vladimir Ivanov" > >> >> ?: "Rafael Winterhalter" > >> >> Cc: "Coleen Phillimore" , > >> >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com > >> >> Spitsyn" , "Daniel Daugherty" > >> >> > >> >> Envoy?: Vendredi 22 Janvier 2016 18:47:31 > >> >> Objet: Re: ClassFileTransformer does not apply to anonymous classes > >> >> > >> >> Rafael, > >> >> > >> >> First of all, I'd like to agree on the terminology. There's some > >> >> confusion there. Anonymous term is ambiguous in Java. There are > >> >> anonymous classes on language level and there's > >> >> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense > >> >> classes. I prefer to call them VM anonymous classes to avoid confusion. > >> >> > >> >> I assume that whenever you use anonymous you assume "VM anonymous". > >> >> > >> >> > thank you for your response. While I completely understand your view > >> >> > from a VM implementor's point of view, as a practicioner I would > >> >> > recommend against it. Not being able to instrument lambda expressions > >> >> > puts a severe limitation onto using the instrumentation API > >> >> > alltogether. For example, a monitoring application that promises to > >> >> > record all invocations of a method of a certain interface that only > >> >> > works 95% of the time is not 5% less usefull but might no longer be > >> >> > useful at all. People have build large applications based on the > >> >> > assumption that all user application code can be instrumented and > >> >> > such > >> >> > a regression would hurt them. For example, until today, over 30 > >> >> > people > >> >> > that use my code generation framework reached out to me and reported > >> >> > the reported behavior as a bug in my library and people are only > >> >> > starting to migrate their applications to Java 8. The outcome is > >> >> > simply not intuitive as using a lambda expression over an anonyous > >> >> > inner class should not change an application's behavior. > >> >> Can you elaborate on that point, please? > >> >> > >> >> I don't see any problems with instrumenting user code. Javac represents > >> >> lambda expression body as a private static method of the enclosing > >> >> class, which can be instrumented. VM anonymous classes are used only as > >> >> a mechanism to glue functional interfaces and lambda expressions > >> >> together at runtime. > >> >> > >> >> For example: > >> >> static void f(Runnable r) { r.run(); } > >> >> f(() -> {}); > >> >> > >> >> Test::f (7 bytes) > >> >> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) > >> >> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) > >> >> > >> >> Why do you need to instrument Test$$Lambda$1/... and not > >> >> Test::lambda$main$0? > >> >> > >> >> > The currently used workaround that people use is to instrument the > >> >> > LambdaMetaFactory class itself to apply the transformer manually when > >> >> > it is being created. This solution is spreading as a standard > >> >> > approach > >> >> > and I am sure that forbidding a redefinition alltogether would only > >> >> > inspire to other workarrounds for being able to migrate running code > >> >> > to the next Java version. > >> >> It doesn't sound like a workaround, but as a solution for a different > >> >> problem. > >> >> > >> >> If there's a need to gather profile for every lambda expression > >> >> instantiation site (indy call), then redefining bootstrap method is the > >> >> right way to go IMO. It allows to intercept and customize indy call > >> >> site > >> >> binding. > >> >> > >> >> What does sound as a workaround is an attempt to instrument classes > >> >> produced by LambdaMetaFactory. Right now, it generates a fresh VM > >> >> anonymous class on every request, but it is an implementation detail > >> >> and > >> >> not a requirement. > >> >> > >> >> For example, LambdaMetaFactory can reuse wrappers on per-functional > >> >> interface basis. It would provide significant footprint savings for > >> >> lambda expression-rich code bases (usually, there are much more > >> >> instantiations than functional interface flavors). > >> >> > >> >> > Furthermore, I do not think that not being able to patch constant > >> >> > pool > >> >> > indices in the generated code introduces any problems in practice. > >> >> > Most real-world instrumentation only appends new constant pool > >> >> > entries > >> >> > without interfering with the existant pool. Rather, I would like to > >> >> > see an exception being raised if one attempts to change the original > >> >> > constant pool without me being able to consider this from a technical > >> >> > perspective. I think this would serve to be sufficient for most > >> >> > people. > >> >> If you care only about lambda expressions, then constant pool patching > >> >> shouldn't be a problem (AFAIR it isn't used for lambda expressions). > >> >> But > >> >> in a more generic case (even for java.lang.invoke purposes), the > >> >> coupling between class file and CP patches is very tight and can be > >> >> easily broken with benign class file transformations. Considering ASM > >> >> library, I'm not sure that even simple instrumentations preserve > >> >> original constant pool structure. > >> >> > >> >> Probably, VM can do some structural checks on CP to forbid any > >> >> modifications except appends, but there are still aliasing problems - > >> >> sharing doesn't work (even for string constants), so new code shouldn't > >> >> use original CP entries. > >> >> > >> >> I'd prefer to avoid such increase in complexity in VM implementation. > >> >> > >> >> > I really hope that there is a way to allow for patching anonymously > >> >> > loaded classes even without exposing the constant pool patches. > >> >> From VM implementation perspective I don't see any problems with > >> >> allowing class file redefinition for VM anonymous classes. What I want > >> >> to do is to discourage from doing so. Again - it is very fragile and > >> >> there's little we can do to fix that. > >> >> > >> >> > So far, thank you for looking into this. I am sure this a more > >> >> > complex > >> >> > matter than what I am able to comprehend. I appreciate that you are > >> >> > taking the time to consider my opinion! > >> >> > >> >> Best regards, > >> >> Vladimir Ivanov > >> >> > >> > From Alan.Bateman at oracle.com Tue Jan 26 12:28:02 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jan 2016 12:28:02 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56A6FCA5.7040808@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> Message-ID: <56A76652.5090500@oracle.com> On 26/01/2016 04:57, Mikael Vidstedt wrote: > > I've finally found some time to return to this and have a new version > of the patch which looks more promising: > > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.02/webrev/ > > This uses memcpy to read/write the native data and the preliminary > benchmark numbers on linux/x64 shows the expected performance. I'll > work on verifying that it doesn't impact other platforms negatively > over the next days/weeks. > > Btw, I also played around with implementing it in pure Java, but there > are some issues getting C2 to correctly vectorize the loop given the > native data accesses, so the native code seems to be needed for now. This looks good and maybe it was a good thing that the compiler upgrade ran into this. We eliminated most of the native methods in this area a few years ago but had to leave the byte swapping methods. It would be good to eliminate those some day. -Alan From chris.hegarty at oracle.com Tue Jan 26 12:27:55 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 26 Jan 2016 12:27:55 +0000 Subject: RFR [9] 8148220: Update TEST.groups to include jdk/internal/math and jdk/internal/misc Message-ID: <9A6E8F7C-0465-41B5-A88D-246BDE11226A@oracle.com> The test groups, that make up jdk_core, should be updated to include jdk/internal/math and jdk/internal/misc. These updates were overlooked when doing re-orgaziation and cleanup in preparation for JEP 260. diff --git a/test/TEST.groups b/test/TEST.groups --- a/test/TEST.groups +++ b/test/TEST.groups @@ -77,6 +77,7 @@ sun/misc \ sun/reflect \ jdk/lambda \ + jdk/internal/misc \ vm @@ -145,7 +146,8 @@ java/util/stream jdk_math = \ - java/math + java/math \ + jdk/internal/math jdk_io = \ java/io -Chris. From forax at univ-mlv.fr Tue Jan 26 12:28:21 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Tue, 26 Jan 2016 13:28:21 +0100 (CET) Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> <56A2AD9D.70802@oracle.com> Message-ID: <1482625420.744530.1453811301415.JavaMail.zimbra@u-pem.fr> Hi Martin, sorry for jumping on this conversation lately, please don't be seduced by the dark side, I understand that the code is 1 byte bigger, but i think that the following code final int s = this.size; Object[] elementData = this.elementData; if (s == elementData.length) { elementData = grow(); } is more readable than final int s; Object[] elementData; if ((s = size) == (elementData = this.elementData).length) elementData = grow(); cheers, R?mi ----- Mail original ----- > De: "Martin Buchholz" > ?: "Stuart Marks" > Cc: "core-libs-dev" > Envoy?: Mardi 26 Janvier 2016 01:49:41 > Objet: Re: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) > > I already regret touching crufty old Vector, but it had the same bug > to fix, and more surprisingly the same sort of performance improvement > from doing the obvious port. So ... Stuart, please review also > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Vector-grow/ > https://bugs.openjdk.java.net/browse/JDK-8148174 > > > On Fri, Jan 22, 2016 at 2:30 PM, Stuart Marks > wrote: > > > > > > On 1/22/16 12:02 PM, Martin Buchholz wrote: > >> > >> I went "by the book" as you suggested and now throw > >> InvalidObjectException when size < 0. > >> (But I've been saying for a decade: if we're serious about > >> Serialization, it needs to be someone's full time job) > > > > > > "Admiral, if we go by the book, years could turn into decades." > > > > Thanks for the update; this looks great to me. > > > > s'marks > From vladimir.x.ivanov at oracle.com Tue Jan 26 12:29:30 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 26 Jan 2016 15:29:30 +0300 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> Message-ID: <56A766AA.8050906@oracle.com> > what bothers me about instrumenting a lambda expression's target > method is the difficulty of locating the method that contains the code > and setting it into context. This is a lot of work to do since one > needs to first locate any invokedynamic call sites and interpret the > connection via the LambdaMetafactory. This is difficult to instrument > without greater efforts and does not feel like a clean solution. > > Many real life instrumentations are based on the assumption of being > able to instrument any class of a given subtype or other any other > class property. For example, one would want to instrument any class > that implements InputStream to discover a resource leak. If the > interface is however functional and implemented as a lambda > expression, a tool that manipulating all class files of this type > stops working. At the same time, end users do not really understand > the difference of their former anonymous class that is now expressed a > lambda expression and perceive the beaviour as a regression. I agree, it becomes more complex without the ability to instrument any class: not only you have to enumerate all loaded lasses, but also inspect all lambda expressions. It means dedicated support for lambda expressions, but IMO the complications are far from insurmountable and makes the design cleaner. Otherwise, you have to support reliable instrumentation of VM anonymous classes, which is harder (again, IMO). Also, you still need dedicated support for lambdas to be able to instrument them individually. Ability to instrument VM anonymous classes doesn't cover that (only the case of instrument all classes fulfilling some requirement). But I'd like to hear your opinion about that. Have you tried to instrument lambda expression bodies instead of tweaking LambdaMetafactory implementation? > I have myself implemented a custom solution for my code generation > library that instruments the LambdaMetafactory to apply class file > transformers to these classes manually. This does however show two > disadvantages: > > 1. javac creates the target method of a lambda expression as a private > method. The only way to implement a lambda expression legally in byte > code is by loading the generated implementation anonymously what > requires non-public API. While it is possible to call > Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, > it is again required to use non-standardized APIs which might fail on > differing implementations of the JVM. No, U::dAC usage is just an implementation detail. It is not required for lambda expression support. What is crucial is the ability to construct appropriate method handle to the method which represents the body. That's where invokedynamic bootstrap methods helps. Lookup object (1st argument [1]) has enough privileges to construct method handles to any methods (even private) in the caller. After that you are free to decide how to wrap the method handle to the lambda body. > 2. At the same time, there is no standardized way to receive all > ClassFileTransformers that are currently registered on the VM. this > also requires calls into internal APIs that are not necessarily > supported on other platforms. > > I fully understand the hesitation to support this from a technical > point of view but in reality, people are already dependant on this > feature and disallowing the instrumentation of lambda classes will > only inspire work-arrounds that reduce the stability of software using > this APIs that does currently work without problems for non-lambda > classes. As your experience demonstrates, instrumentation support of VM anonymous classes is far from production quality. If anybody heavily relies on it, then they should face the same problems. But you are the first who stumbled upon and reported them. So, it seems we are lucky and can decide what is the correct behavior (and not just stick with how it works now). > Best regards, Rafael > > PS: While implementing my solution, I found that the LambdaMetafactory > of the Open JDK creates private, final methods for the serialization > bits. The methods should however not be final as they are already > private. There's no such thing as too much safety, right? ;-) Best regards, Vladimir Ivanov [1] public class LambdaMetafactory { public static CallSite metafactory( MethodHandles.Lookup caller, String invokedName, MethodType invokedType, MethodType samMethodType, MethodHandle implMethod, MethodType instantiatedMethodType) > > > 2016-01-23 13:55 GMT+01:00 Remi Forax : >> I agree with Vladimir, >> >> You should not be able to transform/redefine a VM anonymous class because the transformer will certainly mess up with the order of the constant pool entries. >> >> Slightly off-topic, about ASM, when you create a ClassWriter [1], you can pass a ClassReader of an existing class, in that case ASM copy the constant pool from the class reader to the class writer so the constant pool is preserved. >> >> R?mi >> >> [1] http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 >> >> ----- Mail original ----- >>> De: "Vladimir Ivanov" >>> ?: "Rafael Winterhalter" >>> Cc: "Coleen Phillimore" , core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com >>> Spitsyn" , "Daniel Daugherty" >>> Envoy?: Vendredi 22 Janvier 2016 18:47:31 >>> Objet: Re: ClassFileTransformer does not apply to anonymous classes >>> >>> Rafael, >>> >>> First of all, I'd like to agree on the terminology. There's some >>> confusion there. Anonymous term is ambiguous in Java. There are >>> anonymous classes on language level and there's >>> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense >>> classes. I prefer to call them VM anonymous classes to avoid confusion. >>> >>> I assume that whenever you use anonymous you assume "VM anonymous". >>> >>>> thank you for your response. While I completely understand your view >>>> from a VM implementor's point of view, as a practicioner I would >>>> recommend against it. Not being able to instrument lambda expressions >>>> puts a severe limitation onto using the instrumentation API >>>> alltogether. For example, a monitoring application that promises to >>>> record all invocations of a method of a certain interface that only >>>> works 95% of the time is not 5% less usefull but might no longer be >>>> useful at all. People have build large applications based on the >>>> assumption that all user application code can be instrumented and such >>>> a regression would hurt them. For example, until today, over 30 people >>>> that use my code generation framework reached out to me and reported >>>> the reported behavior as a bug in my library and people are only >>>> starting to migrate their applications to Java 8. The outcome is >>>> simply not intuitive as using a lambda expression over an anonyous >>>> inner class should not change an application's behavior. >>> Can you elaborate on that point, please? >>> >>> I don't see any problems with instrumenting user code. Javac represents >>> lambda expression body as a private static method of the enclosing >>> class, which can be instrumented. VM anonymous classes are used only as >>> a mechanism to glue functional interfaces and lambda expressions >>> together at runtime. >>> >>> For example: >>> static void f(Runnable r) { r.run(); } >>> f(() -> {}); >>> >>> Test::f (7 bytes) >>> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) >>> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) >>> >>> Why do you need to instrument Test$$Lambda$1/... and not >>> Test::lambda$main$0? >>> >>>> The currently used workaround that people use is to instrument the >>>> LambdaMetaFactory class itself to apply the transformer manually when >>>> it is being created. This solution is spreading as a standard approach >>>> and I am sure that forbidding a redefinition alltogether would only >>>> inspire to other workarrounds for being able to migrate running code >>>> to the next Java version. >>> It doesn't sound like a workaround, but as a solution for a different >>> problem. >>> >>> If there's a need to gather profile for every lambda expression >>> instantiation site (indy call), then redefining bootstrap method is the >>> right way to go IMO. It allows to intercept and customize indy call site >>> binding. >>> >>> What does sound as a workaround is an attempt to instrument classes >>> produced by LambdaMetaFactory. Right now, it generates a fresh VM >>> anonymous class on every request, but it is an implementation detail and >>> not a requirement. >>> >>> For example, LambdaMetaFactory can reuse wrappers on per-functional >>> interface basis. It would provide significant footprint savings for >>> lambda expression-rich code bases (usually, there are much more >>> instantiations than functional interface flavors). >>> >>>> Furthermore, I do not think that not being able to patch constant pool >>>> indices in the generated code introduces any problems in practice. >>>> Most real-world instrumentation only appends new constant pool entries >>>> without interfering with the existant pool. Rather, I would like to >>>> see an exception being raised if one attempts to change the original >>>> constant pool without me being able to consider this from a technical >>>> perspective. I think this would serve to be sufficient for most >>>> people. >>> If you care only about lambda expressions, then constant pool patching >>> shouldn't be a problem (AFAIR it isn't used for lambda expressions). But >>> in a more generic case (even for java.lang.invoke purposes), the >>> coupling between class file and CP patches is very tight and can be >>> easily broken with benign class file transformations. Considering ASM >>> library, I'm not sure that even simple instrumentations preserve >>> original constant pool structure. >>> >>> Probably, VM can do some structural checks on CP to forbid any >>> modifications except appends, but there are still aliasing problems - >>> sharing doesn't work (even for string constants), so new code shouldn't >>> use original CP entries. >>> >>> I'd prefer to avoid such increase in complexity in VM implementation. >>> >>>> I really hope that there is a way to allow for patching anonymously >>>> loaded classes even without exposing the constant pool patches. >>> From VM implementation perspective I don't see any problems with >>> allowing class file redefinition for VM anonymous classes. What I want >>> to do is to discourage from doing so. Again - it is very fragile and >>> there's little we can do to fix that. >>> >>>> So far, thank you for looking into this. I am sure this a more complex >>>> matter than what I am able to comprehend. I appreciate that you are >>>> taking the time to consider my opinion! >>> >>> Best regards, >>> Vladimir Ivanov >>> From rafael.wth at gmail.com Tue Jan 26 12:31:13 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Tue, 26 Jan 2016 13:31:13 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <710812902.735970.1453809790007.JavaMail.zimbra@u-pem.fr> References: <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> <147644520.594985.1453798977640.JavaMail.zimbra@u-pem.fr> <710812902.735970.1453809790007.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi, this is what I am doing. At least using Byte Buddy, you turn a switch such that Byte Buddy redefines the LambdaMetafactory to spin custom classes where Byte Buddy also applies a ClassFileTransformer. Currently this results in the (almost) exact class file as if the default LambdaMetafactory had applied this transformation. Therefore, I do not understand why it would be more harmful as a solution if this was standard behavior. As mentioned, one problem that I see with my solution is that I have to code against the Unsafe API what I would like to avoid. The class is surely being removed at some point and then I would already need to implement a version sensitive instrumentation what, in the long run, might result in unstable behavior. At the same time, I could not fall back to public API class loading as it is impossible to regularly load a class that invokes the private method containing the lambda expression's method body. Also, spinning your own LambdaMetafactory is quite an effort if other people would want to do it without the help of Byte Buddy. Especially exception handling is difficult to get right (and to test) and supporting lambda serialization is another bigger task to get done correctly. In my view, it would therefore be desireable if the JVM could natively handle instrumentation (as I would also expect it from the contract of the Instrumentation API that does not even mention this). Best regards, Rafael 2016-01-26 13:03 GMT+01:00 : > Hi Rafael, > why not providing a LambdaInterceptor as asked ? > > My point is that a lambda should be considered as a feature by itself instead as a way to create a class that implement an interface at runtime. > If you do that, your code will still work when future release (maybe a point release), the implementation of the lambda proxy class will change. > > BTW, i maybe wrong, but as far as remember, the first release of Java 8 was not using vm anonymous classes for lambdas. > > regards, > R?mi > > ----- Mail original ----- >> De: "Rafael Winterhalter" >> ?: "R?mi Forax" >> Cc: "Vladimir Ivanov" , "Coleen Phillimore" , >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com Spitsyn" , "Daniel >> Daugherty" >> Envoy?: Mardi 26 Janvier 2016 10:30:54 >> Objet: Re: ClassFileTransformer does not apply to anonymous classes >> >> Maybe this is a more graphic example of a problem that end-users >> currently face: >> http://stackoverflow.com/questions/33912026/intercepting-calls-to-java-8-lambda-expressions-using-byte-buddy >> >> 2016-01-26 10:02 GMT+01:00 : >> > Hello, >> > >> > ----- Mail original ----- >> >> De: "Rafael Winterhalter" >> >> ?: "Remi Forax" >> >> Cc: "Vladimir Ivanov" , "Coleen Phillimore" >> >> , >> >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com Spitsyn" >> >> , "Daniel >> >> Daugherty" >> >> Envoy?: Lundi 25 Janvier 2016 20:24:01 >> >> Objet: Re: ClassFileTransformer does not apply to anonymous classes >> >> >> >> Hi Vladmir, hello Remi, >> >> >> >> what bothers me about instrumenting a lambda expression's target >> >> method is the difficulty of locating the method that contains the code >> >> and setting it into context. This is a lot of work to do since one >> >> needs to first locate any invokedynamic call sites and interpret the >> >> connection via the LambdaMetafactory. This is difficult to instrument >> >> without greater efforts and does not feel like a clean solution. >> >> >> >> Many real life instrumentations are based on the assumption of being >> >> able to instrument any class of a given subtype or other any other >> >> class property. For example, one would want to instrument any class >> >> that implements InputStream to discover a resource leak. If the >> >> interface is however functional and implemented as a lambda >> >> expression, a tool that manipulating all class files of this type >> >> stops working. At the same time, end users do not really understand >> >> the difference of their former anonymous class that is now expressed a >> >> lambda expression and perceive the beaviour as a regression. >> > >> > The lambda body is analyzed because the body is translated to a static >> > method (which can be non static BTW) which is part of the class that uses >> > the lambda. >> > The lambda proxy is a runtime artifact and the way it is currently >> > implemented will likely to change in the future, using a vm anonymous >> > class is a workaround of the fact that there is currently no way to load a >> > function/method without a class. You should not try to analyze the lambda >> > proxy, it doesn't contain codes but it's a straw glue between the >> > interface and the body of the lambda. >> > >> >> >> >> I have myself implemented a custom solution for my code generation >> >> library that instruments the LambdaMetafactory to apply class file >> >> transformers to these classes manually. This does however show two >> >> disadvantages: >> >> >> >> 1. javac creates the target method of a lambda expression as a private >> >> method. The only way to implement a lambda expression legally in byte >> >> code is by loading the generated implementation anonymously what >> >> requires non-public API. While it is possible to call >> >> Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, >> >> it is again required to use non-standardized APIs which might fail on >> >> differing implementations of the JVM. >> > >> > The invokedynamic corresponding to a lambda creation contains a link to the >> > private method, it's i believe all you need, but i'm not sure because what >> > you're doing exactly is fuzy for me. >> > >> >> >> >> 2. At the same time, there is no standardized way to receive all >> >> ClassFileTransformers that are currently registered on the VM. this >> >> also requires calls into internal APIs that are not necessarily >> >> supported on other platforms. >> >> >> >> I fully understand the hesitation to support this from a technical >> >> point of view but in reality, people are already dependant on this >> >> feature and disallowing the instrumentation of lambda classes will >> >> only inspire work-arrounds that reduce the stability of software using >> >> this APIs that does currently work without problems for non-lambda >> >> classes. >> > >> > again, there is no such thing as a lambda class, trying to see a lambda as >> > an anonymous class will never fully work. >> > a lambda is a lambda, you have the link between the functional interface >> > and the lambda body encoded into the invokedynamic which is used to create >> > the lambda, you should not try to get more information, because as i said, >> > it may change. >> > >> >> >> >> Best regards, Rafael >> >> >> >> PS: While implementing my solution, I found that the LambdaMetafactory >> >> of the Open JDK creates private, final methods for the serialization >> >> bits. The methods should however not be final as they are already >> >> private. >> > >> > yes, private is enough but it's not a big deal. >> > >> >> >> > >> > regards, >> > R?mi >> > >> >> >> >> 2016-01-23 13:55 GMT+01:00 Remi Forax : >> >> > I agree with Vladimir, >> >> > >> >> > You should not be able to transform/redefine a VM anonymous class >> >> > because >> >> > the transformer will certainly mess up with the order of the constant >> >> > pool >> >> > entries. >> >> > >> >> > Slightly off-topic, about ASM, when you create a ClassWriter [1], you >> >> > can >> >> > pass a ClassReader of an existing class, in that case ASM copy the >> >> > constant pool from the class reader to the class writer so the constant >> >> > pool is preserved. >> >> > >> >> > R?mi >> >> > >> >> > [1] >> >> > http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 >> >> > >> >> > ----- Mail original ----- >> >> >> De: "Vladimir Ivanov" >> >> >> ?: "Rafael Winterhalter" >> >> >> Cc: "Coleen Phillimore" , >> >> >> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com >> >> >> Spitsyn" , "Daniel Daugherty" >> >> >> >> >> >> Envoy?: Vendredi 22 Janvier 2016 18:47:31 >> >> >> Objet: Re: ClassFileTransformer does not apply to anonymous classes >> >> >> >> >> >> Rafael, >> >> >> >> >> >> First of all, I'd like to agree on the terminology. There's some >> >> >> confusion there. Anonymous term is ambiguous in Java. There are >> >> >> anonymous classes on language level and there's >> >> >> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense >> >> >> classes. I prefer to call them VM anonymous classes to avoid confusion. >> >> >> >> >> >> I assume that whenever you use anonymous you assume "VM anonymous". >> >> >> >> >> >> > thank you for your response. While I completely understand your view >> >> >> > from a VM implementor's point of view, as a practicioner I would >> >> >> > recommend against it. Not being able to instrument lambda expressions >> >> >> > puts a severe limitation onto using the instrumentation API >> >> >> > alltogether. For example, a monitoring application that promises to >> >> >> > record all invocations of a method of a certain interface that only >> >> >> > works 95% of the time is not 5% less usefull but might no longer be >> >> >> > useful at all. People have build large applications based on the >> >> >> > assumption that all user application code can be instrumented and >> >> >> > such >> >> >> > a regression would hurt them. For example, until today, over 30 >> >> >> > people >> >> >> > that use my code generation framework reached out to me and reported >> >> >> > the reported behavior as a bug in my library and people are only >> >> >> > starting to migrate their applications to Java 8. The outcome is >> >> >> > simply not intuitive as using a lambda expression over an anonyous >> >> >> > inner class should not change an application's behavior. >> >> >> Can you elaborate on that point, please? >> >> >> >> >> >> I don't see any problems with instrumenting user code. Javac represents >> >> >> lambda expression body as a private static method of the enclosing >> >> >> class, which can be instrumented. VM anonymous classes are used only as >> >> >> a mechanism to glue functional interfaces and lambda expressions >> >> >> together at runtime. >> >> >> >> >> >> For example: >> >> >> static void f(Runnable r) { r.run(); } >> >> >> f(() -> {}); >> >> >> >> >> >> Test::f (7 bytes) >> >> >> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) >> >> >> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) >> >> >> >> >> >> Why do you need to instrument Test$$Lambda$1/... and not >> >> >> Test::lambda$main$0? >> >> >> >> >> >> > The currently used workaround that people use is to instrument the >> >> >> > LambdaMetaFactory class itself to apply the transformer manually when >> >> >> > it is being created. This solution is spreading as a standard >> >> >> > approach >> >> >> > and I am sure that forbidding a redefinition alltogether would only >> >> >> > inspire to other workarrounds for being able to migrate running code >> >> >> > to the next Java version. >> >> >> It doesn't sound like a workaround, but as a solution for a different >> >> >> problem. >> >> >> >> >> >> If there's a need to gather profile for every lambda expression >> >> >> instantiation site (indy call), then redefining bootstrap method is the >> >> >> right way to go IMO. It allows to intercept and customize indy call >> >> >> site >> >> >> binding. >> >> >> >> >> >> What does sound as a workaround is an attempt to instrument classes >> >> >> produced by LambdaMetaFactory. Right now, it generates a fresh VM >> >> >> anonymous class on every request, but it is an implementation detail >> >> >> and >> >> >> not a requirement. >> >> >> >> >> >> For example, LambdaMetaFactory can reuse wrappers on per-functional >> >> >> interface basis. It would provide significant footprint savings for >> >> >> lambda expression-rich code bases (usually, there are much more >> >> >> instantiations than functional interface flavors). >> >> >> >> >> >> > Furthermore, I do not think that not being able to patch constant >> >> >> > pool >> >> >> > indices in the generated code introduces any problems in practice. >> >> >> > Most real-world instrumentation only appends new constant pool >> >> >> > entries >> >> >> > without interfering with the existant pool. Rather, I would like to >> >> >> > see an exception being raised if one attempts to change the original >> >> >> > constant pool without me being able to consider this from a technical >> >> >> > perspective. I think this would serve to be sufficient for most >> >> >> > people. >> >> >> If you care only about lambda expressions, then constant pool patching >> >> >> shouldn't be a problem (AFAIR it isn't used for lambda expressions). >> >> >> But >> >> >> in a more generic case (even for java.lang.invoke purposes), the >> >> >> coupling between class file and CP patches is very tight and can be >> >> >> easily broken with benign class file transformations. Considering ASM >> >> >> library, I'm not sure that even simple instrumentations preserve >> >> >> original constant pool structure. >> >> >> >> >> >> Probably, VM can do some structural checks on CP to forbid any >> >> >> modifications except appends, but there are still aliasing problems - >> >> >> sharing doesn't work (even for string constants), so new code shouldn't >> >> >> use original CP entries. >> >> >> >> >> >> I'd prefer to avoid such increase in complexity in VM implementation. >> >> >> >> >> >> > I really hope that there is a way to allow for patching anonymously >> >> >> > loaded classes even without exposing the constant pool patches. >> >> >> From VM implementation perspective I don't see any problems with >> >> >> allowing class file redefinition for VM anonymous classes. What I want >> >> >> to do is to discourage from doing so. Again - it is very fragile and >> >> >> there's little we can do to fix that. >> >> >> >> >> >> > So far, thank you for looking into this. I am sure this a more >> >> >> > complex >> >> >> > matter than what I am able to comprehend. I appreciate that you are >> >> >> > taking the time to consider my opinion! >> >> >> >> >> >> Best regards, >> >> >> Vladimir Ivanov >> >> >> >> >> >> From sean.coffey at oracle.com Tue Jan 26 12:45:30 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Tue, 26 Jan 2016 12:45:30 +0000 Subject: RFR [9] 8148220: Update TEST.groups to include jdk/internal/math and jdk/internal/misc In-Reply-To: <9A6E8F7C-0465-41B5-A88D-246BDE11226A@oracle.com> References: <9A6E8F7C-0465-41B5-A88D-246BDE11226A@oracle.com> Message-ID: <56A76A6A.8080706@oracle.com> Looks fine to me Chris. Regards, Sean. On 26/01/2016 12:27, Chris Hegarty wrote: > The test groups, that make up jdk_core, should be updated to include > jdk/internal/math and jdk/internal/misc. These updates were overlooked > when doing re-orgaziation and cleanup in preparation for JEP 260. > > diff --git a/test/TEST.groups b/test/TEST.groups > --- a/test/TEST.groups > +++ b/test/TEST.groups > @@ -77,6 +77,7 @@ > sun/misc \ > sun/reflect \ > jdk/lambda \ > + jdk/internal/misc \ > vm > > @@ -145,7 +146,8 @@ > java/util/stream > > jdk_math = \ > - java/math > + java/math \ > + jdk/internal/math > > jdk_io = \ > java/io > > -Chris. From paul.sandoz at oracle.com Tue Jan 26 13:18:21 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 14:18:21 +0100 Subject: RFR: 8076458: java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout In-Reply-To: <56A71B6A.9090605@oracle.com> References: <56A71B6A.9090605@oracle.com> Message-ID: <8ED500E5-D39D-468D-A08D-2EE9C09B8029@oracle.com> Hi Hamlin, Conservatively I would prefer not to remove data sets if at all possible. It will affect all tests, and leaf tasks for parallel streams should have enough data to crunch on. I suspect the problem of the flatMap test is not necessarily due to the source sizes being of 1000 elements but that there are tests that substitute an element whose value is m for n elements from 0..m, which can explode things and generate lots of garbage. Have you tried executing those kinds tests when the data size is < 1000? My bet is the FlatMapOpTest will run significantly faster and you will not need to split it out. There are two ways we could consider doing this: 1) Check the size in the test method: if (data.size() < 1000) { exerciseOps(data, s -> s.flatMap(mfLt)); exerciseOps(data, s -> s.flatMap(integerRangeMapper)); exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); } 2) Include a new data provider for smaller data sets @Test(dataProvider = "StreamTestData", dataProviderClass = StreamTestDataProvider.class) public void testOps(String name, TestData.OfRef data) { Collection result = exerciseOps(data, s -> s.flatMap(mfId)); assertEquals(data.size(), result.size()); result = exerciseOps(data, s -> s.flatMap(mfNull)); assertEquals(0, result.size()); result = exerciseOps(data, s-> s.flatMap(e -> Stream.empty())); assertEquals(0, result.size()); } @Test(dataProvider = "StreamTestData.small", dataProviderClass = StreamTestDataProvider.class) public void testOpsX(String name, TestData.OfRef data) { exerciseOps(data, s -> s.flatMap(mfLt)); exerciseOps(data, s -> s.flatMap(integerRangeMapper)); exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); } I prefer the latter approach (applied to ref and primitive data sets). It?s more work, but i think the right direction. Paul. > On 26 Jan 2016, at 08:08, Hamlin Li wrote: > > Hi everyone, > > Would you please help to review the fix for bug https://bugs.openjdk.java.net/browse/JDK-8076458, java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout. > webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.00/ > > Thank you > -Hamlin From paul.sandoz at oracle.com Tue Jan 26 13:29:07 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 14:29:07 +0100 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <56A65F97.4080500@oracle.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> <56A65F97.4080500@oracle.com> Message-ID: <15AF237D-96FE-4907-8BED-57756931D1C8@oracle.com> > On 25 Jan 2016, at 18:47, Xueming Shen wrote: > > The proposed change looks fine with me. > Yes, +1 nice work. Paul. From jason_mehrens at hotmail.com Tue Jan 26 13:44:57 2016 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Tue, 26 Jan 2016 13:44:57 +0000 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: <56A133FC.4040806@oracle.com> <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> <56A2AD9D.70802@oracle.com>, Message-ID: Hi Martin, Are you sure about the change where addElement is now calling the public (non-final) version of add? I would think that we would want to avoid any type of compatibility changes with regard to Vector. Jason ________________________________________ From: core-libs-dev on behalf of Martin Buchholz Sent: Monday, January 25, 2016 6:49 PM To: Stuart Marks Cc: core-libs-dev Subject: Re: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) I already regret touching crufty old Vector, but it had the same bug to fix, and more surprisingly the same sort of performance improvement from doing the obvious port. So ... Stuart, please review also http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Vector-grow/ https://bugs.openjdk.java.net/browse/JDK-8148174 On Fri, Jan 22, 2016 at 2:30 PM, Stuart Marks wrote: > > > On 1/22/16 12:02 PM, Martin Buchholz wrote: >> >> I went "by the book" as you suggested and now throw >> InvalidObjectException when size < 0. >> (But I've been saying for a decade: if we're serious about >> Serialization, it needs to be someone's full time job) > > > "Admiral, if we go by the book, years could turn into decades." > > Thanks for the update; this looks great to me. > > s'marks From chris.hegarty at oracle.com Tue Jan 26 13:55:32 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 26 Jan 2016 13:55:32 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A3AC1D.50003@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> Message-ID: It is wonderful to see the various ideas on this thread about the longer term solution to the prompt releasing of direct buffer native memory. I do not want to obstruct that ( it is very informative ), but I?d like to warp up the review on the actual moving of Cleaner. To that end, I?ve update the webrev as per Alan?s comments and suggestion ( to extend Runnable ). http://cr.openjdk.java.net/~chegar/8148117/ -Chris. On 23 Jan 2016, at 16:36, Alan Bateman wrote: > On 23/01/2016 16:16, Chris Hegarty wrote: >> : >> >> Webrev: >> http://cr.openjdk.java.net/~chegar/8148117/ >> >> > This has to move to your patch looks okay. You might need to update the TEST.groups to ensure that the existOnThrow tests will be run by jdk_core. > > -Alan. From scolebourne at joda.org Tue Jan 26 13:57:21 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Tue, 26 Jan 2016 13:57:21 +0000 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <56A3F349.5070500@oracle.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> Message-ID: While there is no blockng reason why the concept could not be pulled up to ChronoLocalDate, the method signatures would differ. LocalDate::datesUntil(LocalDate) LocalDate::datesUntil(LocalDate, Period) ChronoLocalDate::datesUntil(ChronoLocalDate) ChronoLocalDate::datesUntil(ChronoLocalDate, ChronoPeriod) Thus, adding the ChronoLocalDate methods later will make two additional methods available on LocalDate (as they will not override). As such, I'm not convinced that the streaming methods are needed on ChronoLocalDate. The "correct" mechanism for looping with a ChronoPeriod may differ by calendar system (not sure how, but it might). And while datesUntil(ChronoLocalDate) would be safe on ChronoLocalDate, it would weaken the method signature for the common case on LocalDate. Basically, since the stream can be created manually in the rare cases of ChronoLocalDate, it seems unnecessary to worry about abstracting up. Stephen On 23 January 2016 at 21:40, Roger Riggs wrote: > Hi, > > Looks good. > > I would want to verify that it is defined such that if it later needs to be > abstracted up to ChronoLocalDate and apply to compatible Chronologies > and respective local dates such as JapaneseDate there is no conflict. > It should not be an issue since Period implements ChronoPeriod. > In the respective implementations, the estimation/computation of the > number of steps > would need to depend on the Chronology's definition of months. > > I can sponsor this enhancement. > > Thanks, Roger > > > > > On 1/20/16 10:15 AM, Stephen Colebourne wrote: > >> I'm happy with the logic and specification of this proposal. I think it >> will be a useful addition. >> >> I'll let the Oracle team chime in to do a further review. >> >> thanks >> Stephen >> >> >> >> On 16 January 2016 at 13:31, Tagir F. Valeev wrote: >> >> Hello! >>> >>> Thanks for review! Here's the updated patch: >>> http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ >>> >>> SC> The docs say that if the end date is before the start date, the >>> SC> stream is empty. While I can see just about why LongStream.range() >>> SC> works that way, I don't think this API should. The minimum is an >>> SC> exception, but it would be easy to support negative in the >>> SC> days-only case or the months-only case. The problem is where there >>> SC> are both months/years and days and those should be exceptions. >>> >>> Now datesUntil(endExclusive) throws an IllegalArgumentException if end >>> date is before start date. >>> >>> datesUntil(endExclusive, step) supports negative periods. It throws >>> IllegalArgumentException if: >>> - step is zero >>> - step.toTotalMonths() and step.getDays() both non-zero and have >>> opposite sign >>> - step is negative and end date is after start date >>> - step is positive and end date is before start date >>> >>> Otherwise it works correctly: you can use >>> LocalDate.of(2016, 1, 1) >>> .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); >>> >>> Steps like Period.of(-1, -1, -1) are also supported. >>> >>> SC> The single-arg implementation uses plusDays() with an >>> SC> incrementing number. When the performance patch goes in, the >>> SC> proposed streaming implementation will be optimal for small >>> SC> streams but less optimal for large ones. The fastest way to loop >>> SC> over a list of dates would be to manually generate them by >>> SC> incrementing the day until it exceeds the length of month, and so >>> SC> on. However, this would be serial. >>> >>> As I understand, plusDays performance patch is already pushed. >>> >>> It's possible to write custom Spliterator which would use >>> previous.plusDays(1) in tryAdvance() and jump (via >>> LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is >>> requested. However this adds at least one additional class and more >>> complexity. I guess, such optimization can be considered later as >>> separate issue when API is stabilized. >>> >>> SC> As such, I think the best way to write this, taking account of >>> SC> how plusDays() is implemented, is as follows: >>> >>> SC> LongStream.range(start.toEpochDay(), >>> SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); >>> >>> Ok, now it's done this way. >>> >>> SC> In the period-based method, it would be best to capture the case >>> SC> where totalMonths == 0 and days > 0 and forward to another method >>> SC> that ignores months. That private method can share implementation >>> SC> with the public single-arg method (passing in 1). >>> >>> This case still more complex than one-day case as it requires division >>> and multiplication. Thus I'd keep these case separately. However I >>> simplified "months = 0" path using ofEpochDay, now it should be >>> faster. >>> >>> SC> The docs for the period-based method should specify the strategy >>> SC> that produces the results (in abstract terms). This needs to cover >>> SC> that the result is equivalent to always adding to the start period >>> SC> a multiple of the period. >>> >>> I added some clarifications, please check. >>> >>> SC> Some nits: >>> SC> I prefer to avoid @link in the first sentence. Just using 'stream' >>> would be sufficient. >>> >>> Done. >>> >>> SC> The first sentence should be a summary. In this case it probably has >>> a >>> bit too much detail. >>> >>> Done. >>> >>> SC> The @return has 'values' on a new line when it could be on the same >>> line. >>> >>> I set now line length = 100 characters in my IDE. Is it ok? >>> >>> SC> If statements need braces. >>> >>> Done. >>> >>> With best regards, >>> Tagir Valeev. >>> >>> >>> > From uschindler at apache.org Tue Jan 26 14:05:22 2016 From: uschindler at apache.org (Uwe Schindler) Date: Tue, 26 Jan 2016 15:05:22 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> Message-ID: <02b701d15842$9c1cbd30$d4563790$@apache.org> Hi, looks good to me. Once we have EA builds with that I will adapt the ByteBufferIndexInput code in Lucene. One thing about the Runable: This should work perfectly fine, because we only need to make call the getCleaner() method accessible, call it and finally check if return type implements Runable (if not we fall back to pre Java 9 code). We can then cast and invoke the cleaner without any further reflection. But there is now some security related issue: The reflection on DirectBuffer should work in most environments (also with security managers), so you can get the Cleaner instance with a doPrivileged (ideally). But as this cleaner instance was previously inside sun.misc package, and you needed RuntimePermission "accessInPackage.sun.misc" to call and make the clean method accessible. But with the new patch you no longer need this permission, making it easier to invoke that method. In my opinion, you should add some other runtime permission for safety, which is checked on calling the run() method. This would make it safe, because as soon as you have SecurityManager you would need to give explicit permission to the code. Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Chris Hegarty > Sent: Tuesday, January 26, 2016 2:56 PM > To: Alan Bateman ; core-libs-dev dev at openjdk.java.net> > Subject: Re: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref > > It is wonderful to see the various ideas on this thread about the longer > term solution to the prompt releasing of direct buffer native memory. I > do not want to obstruct that ( it is very informative ), but I?d like to warp up > the review on the actual moving of Cleaner. To that end, I?ve update the > webrev as per Alan?s comments and suggestion ( to extend Runnable ). > > http://cr.openjdk.java.net/~chegar/8148117/ > > -Chris. > > On 23 Jan 2016, at 16:36, Alan Bateman wrote: > > > On 23/01/2016 16:16, Chris Hegarty wrote: > >> : > >> > >> Webrev: > >> http://cr.openjdk.java.net/~chegar/8148117/ > >> > >> > > This has to move to your patch looks okay. You might need to update the > TEST.groups to ensure that the existOnThrow tests will be run by jdk_core. > > > > -Alan. From paul.sandoz at oracle.com Tue Jan 26 14:08:23 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 15:08:23 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A3E320.2070004@redhat.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> Message-ID: > On 23 Jan 2016, at 21:31, Andrew Haley wrote: > > BTW, does anyone here know why we don't have humongous ByteBuffers > with a long index? > Probably in part because of Java arrays. IMO we need Arrays 2.0 (+ panama i suspect). Here?s some "shoot from the hip" thinking? In principle a direct ByteBuffer could bound a region larger than that which can be expressed with an int limit (i suppose a heap ByteBuffer could do the same if it holds multiple instances if byte[], although that would require more work). The direct BB can be used to access the first 2^31 - 1 values. A VarHandle could be produced that accesses the whole range since it could accept long index values and check bounds against a special long limit field. That might require some additional HotSpot work to optimize bounds checks and any loops to reduce or avoid a safe point check per iteration. Paul. From Alan.Bateman at oracle.com Tue Jan 26 14:32:43 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jan 2016 14:32:43 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <02b701d15842$9c1cbd30$d4563790$@apache.org> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <02b701d15842$9c1cbd30$d4563790$@apache.org> Message-ID: <56A7838B.7040005@oracle.com> On 26/01/2016 14:05, Uwe Schindler wrote: > Hi, > > looks good to me. Once we have EA builds with that I will adapt the ByteBufferIndexInput code in Lucene. > > One thing about the Runable: This should work perfectly fine, because we only need to make call the getCleaner() method accessible, call it and finally check if return type implements Runable (if not we fall back to pre Java 9 code). We can then cast and invoke the cleaner without any further reflection. > > But there is now some security related issue: The reflection on DirectBuffer should work in most environments (also with security managers), so you can get the Cleaner instance with a doPrivileged (ideally). But as this cleaner instance was previously inside sun.misc package, and you needed RuntimePermission "accessInPackage.sun.misc" to call and make the clean method accessible. But with the new patch you no longer need this permission, making it easier to invoke that method. > This is a private field so if you are hacking into it then you'll need ReflectPermission("suppressAccessChecks") when running with a security manager. Clearly anything with a reference to a Cleaner needs to be super careful not to let it leak to untrusted code. -Alan From uschindler at apache.org Tue Jan 26 14:43:17 2016 From: uschindler at apache.org (Uwe Schindler) Date: Tue, 26 Jan 2016 15:43:17 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A7838B.7040005@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <02b701d15842$9c1cbd30$d4563790$@apache.org> <56A7838B.7040005@oracle.com> Message-ID: <02c201d15847$e8220140$b86603c0$@apache.org> Hi, > On 26/01/2016 14:05, Uwe Schindler wrote: > > Hi, > > > > looks good to me. Once we have EA builds with that I will adapt the > ByteBufferIndexInput code in Lucene. > > > > One thing about the Runable: This should work perfectly fine, because we > only need to make call the getCleaner() method accessible, call it and finally > check if return type implements Runable (if not we fall back to pre Java 9 > code). We can then cast and invoke the cleaner without any further > reflection. > > > > But there is now some security related issue: The reflection on DirectBuffer > should work in most environments (also with security managers), so you can > get the Cleaner instance with a doPrivileged (ideally). But as this cleaner > instance was previously inside sun.misc package, and you needed > RuntimePermission "accessInPackage.sun.misc" to call and make the clean > method accessible. But with the new patch you no longer need this > permission, making it easier to invoke that method. > > > This is a private field so if you are hacking into it then you'll need > ReflectPermission("suppressAccessChecks") when running with a security > manager. Clearly anything with a reference to a Cleaner needs to be > super careful not to let it leak to untrusted code. I know and Lucene takes care of that. I just said, previously you needed both runtime permissions: "accessInPackage.sun.misc" AND "suppressAccessChecks", now only one. I just suggested to add the 2nd hurdle back somehow, as additional safety. Otherwise I am fine with the patch, was just meant as "improvement". E.g., Elasticsearch already gives *only* lucene-core.jar both permisisions, nothing else in its code!!! Uwe From Roger.Riggs at Oracle.com Tue Jan 26 14:59:39 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 26 Jan 2016 09:59:39 -0500 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> Message-ID: <56A789DB.60903@Oracle.com> Hi Chris, Looks fine. Perhaps update the copyrights. Upgrading the shell based test to java based would be good sometime too. Also, there is a more recent version of webrev [1] that provides convenient next and previous links. Thanks, Roger [1] http://hg.openjdk.java.net/code-tools/webrev/raw-file/tip/webrev.ksh On 1/26/2016 8:55 AM, Chris Hegarty wrote: > It is wonderful to see the various ideas on this thread about the longer > term solution to the prompt releasing of direct buffer native memory. I > do not want to obstruct that ( it is very informative ), but I?d like to warp up > the review on the actual moving of Cleaner. To that end, I?ve update the > webrev as per Alan?s comments and suggestion ( to extend Runnable ). > > http://cr.openjdk.java.net/~chegar/8148117/ > > -Chris. > > On 23 Jan 2016, at 16:36, Alan Bateman wrote: > >> On 23/01/2016 16:16, Chris Hegarty wrote: >>> : >>> >>> Webrev: >>> http://cr.openjdk.java.net/~chegar/8148117/ >>> >>> >> This has to move to your patch looks okay. You might need to update the TEST.groups to ensure that the existOnThrow tests will be run by jdk_core. >> >> -Alan. From Alan.Bateman at oracle.com Tue Jan 26 15:23:58 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jan 2016 15:23:58 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> Message-ID: <56A78F8E.5020402@oracle.com> On 26/01/2016 13:55, Chris Hegarty wrote: > It is wonderful to see the various ideas on this thread about the longer > term solution to the prompt releasing of direct buffer native memory. I > do not want to obstruct that ( it is very informative ), but I?d like to warp up > the review on the actual moving of Cleaner. To that end, I?ve update the > webrev as per Alan?s comments and suggestion ( to extend Runnable ). > > http://cr.openjdk.java.net/~chegar/8148117/ > > -Chris. > > This looks okay. As a defensive-in-depth then Cleaner::run can do a permission check and should ease concerns about leakage. -Alan. From paul.sandoz at oracle.com Tue Jan 26 15:27:30 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 16:27:30 +0100 Subject: API review of VarHandles In-Reply-To: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Hi Many thanks for the feedback so far. Some high-level responses: 1) I don?t think there is much we can do right now to reduce the verbosity of reflective lookup. We have discussed this at least once before in the past. We need field literals, as mentioned in the JEP, to really knock this on the head (which might lead to constant pool forms, or even a ?constant-dynamic?, and perhaps local variables to VHs that constant fold). But such support of literals is just too much work to take on at the moment. 2) Examples are needed in api notes much like there are examples for MHs. It might be possible to rearrange things so it?s more user-focused up front with suitable "Government Health Warnings". I wanted to focus on the normative aspects first. 3) Yes, it?s a sharp knife, a low-level building block, consolidating many forms of access and shape under one interface, designed for power users to avail of directly or wrap in higher-level forms as required, and leveraging the low-level method handle invocation mechanism that has been heavily optimized to constant fold and inline. Paul. > On 21 Jan 2016, at 23:42, Paul Sandoz wrote: > > Hi > > This is a request to review the VarHandles API. The code reviews and pushes will occur separately, and flow through the hs-comp repo, most likely from the bottom up first with Unsafe changes. > > The specdiff can be found here: > > http://cr.openjdk.java.net/~psandoz/jdk9/varhandles/specdiff/overview-summary.html > > (Note that specdiff renders some aspects of JavaDoc incorrectly, so just ignore any such quirks.) > > A consensus on the set of access mode methods proposed by Doug was previously discussed and reached. > > For the moment please ignore the following methods on MethodHandles: byteArrayViewVarHandle; and byteBufferViewVarHandle. It is necessary to revisit that functionality w.r.t. alignment and proposed enhancements to ByteBuffer (in discussion on valhalla-dev). > > Paul. > > From amaembo at gmail.com Tue Jan 26 15:51:43 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Tue, 26 Jan 2016 21:51:43 +0600 Subject: 8148250 Stream.limit parallel ordered performance In-Reply-To: References: <919184407.20160124173651@gmail.com> Message-ID: <1059144622.20160126215143@gmail.com> Hello! Thank you for review! Here's the issue: https://bugs.openjdk.java.net/browse/JDK-8148250 Will post complete webrev later. PS> Note that it is still the case that in almost all scenarios this PS> is likely to be a bad form of parallel stream. Well not always it's possible to estimate in advance the size of the stream. Consider that we have user-specified filter upstream which searches over the big collection (we want to return first 10 search results, order is important): data.parallelStream() .filter(element -> userQuery.test(element)) .limit(10).collect(toList()); If user query produces 10-15 results, using parallel stream is very reasonable, but if it produces millions of results it should not regress very much (at least should not become much slower than sequential version which is what we see currently). PS> I think the comment you refer to still applies but now for larger n, so we should refine it. Should we replace "regardless of the value of n" with "when n*parallelismLevel is sufficiently large"? With best regards, Tagir Valeev. PS> Paul. >> On 24 Jan 2016, at 12:36, Tagir F. Valeev wrote: >> >> Hello! >> >> I'm investigating Stream.limit() performance for parallel ordered >> streams when no SUBSIZED optimization is performed. >> >> Consider the following code, for example: >> >> AtomicInteger counter = new AtomicInteger(); >> int[] result = IntStream.range(0, 1_000_000).parallel().filter(x -> true) >> .peek(x -> counter.incrementAndGet()).limit(10).toArray(); >> System.out.println(Arrays.toString(result)); >> System.out.println(counter.get()); >> >> How much the counter.get() would print? It changes from launch to >> launch, but usually within 375000..625000. This is just insane. On my >> 4-core system parallel stream creates 16 individual tasks. I expect >> that every individual task should consume no more than 10 elements, so >> in total no more than 160 elements should be consumed in this case. >> >> Here's a patch which addresses this issue: >> http://cr.openjdk.java.net/~tvaleev/patches/limit/limit-patch.txt >> >> In the limit case non-root leaf tasks may switch to copyIntoWithCancel >> to control the count of consumed elements and do not consume more than >> necessary. >> >> This change seems to fix the issue addressed in comment (at least >> partially): >> >> // @@@ OOMEs will occur for LongStream.range(0, Long.MAX_VALUE).filter(i -> true).limit(n) >> // regardless of the value of n >> // Need to adjust the target size of splitting for the >> // SliceTask from say (size / k) to say min(size / k, 1 << 14) >> // This will limit the size of the buffers created at the leaf nodes >> // cancellation will be more aggressive cancelling later tasks >> // if the target slice size has been reached from a given task, >> // cancellation should also clear local results if any >> >> I checked with the following code: >> >> for(int n : new int[] {10, 100, 1000, 5000, 10000, 50000, 100000, 500000, 1000000}) { >> System.out.println(n); >> long[] arr = LongStream.range(0, Long.MAX_VALUE).filter(i -> true).parallel().limit(n).toArray(); >> long[] ref = LongStream.range(0, n).toArray(); >> System.out.println(Arrays.equals(arr, ref)); >> } >> >> It works correctly after applying my patch (while dies with OOME >> without patch, as comment suggests). >> >> Currently existing unit tests also pass with my patch. However I'm >> very new in the internals of parallel stream processing, so it's >> possible that I'm missing something. Please review! If this looks >> reasonable I will log an issue and write new test cases. >> >> Thank you in advance, >> With best regards, >> Tagir Valeev. >> From amaembo at gmail.com Tue Jan 26 15:57:20 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Tue, 26 Jan 2016 21:57:20 +0600 Subject: RFR 8147505: Clarification of BaseStream.onClose() behavior In-Reply-To: <757F647A-DE7A-49A3-BC0F-938B4D0E8CD2@oracle.com> References: <1666489469.20160115103543@gmail.com> <68E50067-D3FB-414F-80AD-40F188DAE5FC@oracle.com> <1616962894.20160116091332@gmail.com> <0886057.20160123163820@gmail.com> <757F647A-DE7A-49A3-BC0F-938B4D0E8CD2@oracle.com> Message-ID: <1832946052.20160126215720@gmail.com> Hello! Thanks for review! Updated webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8147505/r2/ Removed redundant test and added a comment. With best regards, Tagir Valeev. PS> Hi Tagir, PS> StreamCloseTest.java PS> ? PS> 181 try(Stream s = countTo(100).stream()) { PS> 182 s.map(x -> x).forEach(i -> {}); PS> 183 checkISE(() -> s.onClose(() -> fail("2"))); PS> 184 } PS> We don?t need this one, it?s redundant. The other performing the PS> s.close() is i think useful because i don?t recall it being tested PS> in other places like this, plus it also tests the idempotent PS> behaviour, perhaps worth a comment on this in the test. PS> Otherwise, +1 PS> Again, i will handle in bulk with your other fixes. PS> Many thanks, PS> Paul. >> On 23 Jan 2016, at 11:38, Tagir F. Valeev wrote: >> >> Hello! >> >> Here's a webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8147505/r1/ >> >> I just added a check in AbstractPipeline.onClose and the corresponding >> unit-test. To me it seems that no documentation corrections are >> necessary, because "onClose" is already documented that it's >> an intermediate operation: >> >> http://download.java.net/jdk9/docs/api/java/util/stream/BaseStream.html#onClose-java.lang.Runnable- >> >> Also it's already documented: >> >>> A stream should be operated on (invoking an intermediate or terminal >>> stream operation) only once. This rules out, for example, "forked" >>> streams, where the same source feeds two or more pipelines, or >>> multiple traversals of the same stream. A stream implementation may >>> throw IllegalStateException if it detects that the stream is being >>> reused. >> >> http://download.java.net/jdk9/docs/api/java/util/stream/Stream.html >> >> So this already implies that invoking intermediate operation "onClose" >> after stream terminal operation is not allowed and may throw >> IllegalStateException. >> >> Please review! >> >> With best regards, >> Tagir Valeev. >> >> TFV> Seems that my original letter sounds confusing due to my bad English. >> TFV> Every occurrence of "idempotent" should be read as "not idempotent". >> TFV> Sorry for this! >> >> PS>> We did not bother to throw ISEs from parallel/sequence/onClose >> PS>> primarily because they return ?this? and they are kind of >> PS>> harmless, but the undefined behaviour you point out could be shut >> PS>> down without much concern for backwards compatibility. We can >> PS>> modify them to check whether they have been linked or consumed >> PS>> (while not updating the previous stage). >> >> PS>> Wanna log an issue? >> >> TFV> Sure: >> TFV> https://bugs.openjdk.java.net/browse/JDK-8147505 >> >> TFV> Thanks, >> TFV> Tagir Valeev. >> >> PS>> Thanks, >> PS>> Paul. >> >> >>>>> On 15 Jan 2016, at 05:35, Tagir F. Valeev wrote: >>>>> >>>>> Hello! >>>>> >>>>> Current documentation for BaseStream.onClose() does not specify >>>>> explicitly how the stream would behave if additional close handlers >>>>> are registered after the stream is consumed or even closed. The >>>>> implementation actually allows registering the additional close >>>>> handlers after consumption or closure and close() method is >>>>> idempotent: newly registered handlers are perfectly called: >>>>> >>>>> Stream s = Stream.of("content"); >>>>> s = s.onClose(() -> System.out.println("A")); >>>>> s.forEach(System.out::println); >>>>> s = s.onClose(() -> System.out.println("B")); >>>>> s.close(); // prints A and B >>>>> s = s.onClose(() -> System.out.println("C")); >>>>> s.close(); // prints C >>>>> >>>>> (removing "s =" produces the same result). >>>>> >>>>> I think if such behavior is intended, it should be explicitly noted in >>>>> the specification, as third-party implementations of BaseStream >>>>> interface should provide consistent behavior. Or at least it should be >>>>> noted that some BaseStream implementations may have idempotent close() >>>>> method, so the derived AutoCloseable objects (which own the BaseStream >>>>> object) should be aware about this behavior and provide idempotent >>>>> close() method as well. Without knowing this one may write: >>>>> >>>>> class MyObject implements AutoCloseable { >>>>> private BaseStream s; >>>>> >>>>> public MyObject(BaseStream s) { >>>>> this.s = s; >>>>> } >>>>> >>>>> @Override >>>>> public void close() throws Exception { >>>>> if(s != null) { >>>>> s.close(); >>>>> s = null; >>>>> } >>>>> } >>>>> } >>>>> >>>>> Such code closes the stream only once, so newly registered handlers >>>>> will not be called if MyObject::close is called the second time. >>>>> >>>>> However, to my opinion the current behavior is weird and should be >>>>> changed in order not to allow registering new close handles (throwing >>>>> IllegalStateException) when the stream is already closed, or even >>>>> better when the stream is linked/consumed. As far as I know currently >>>>> in JDK close handlers are registered only for non-consumed stream, so >>>>> such change would not break existing JDK code (though may break some >>>>> strange third party code). It should be noted that AutoCloseable >>>>> interface discourages idempotent close() methods (though not forbids >>>>> them). >>>>> >>>>> What do you think? >>>>> >>>>> With best regards, >>>>> Tagir Valeev. >>>>> >> From brian.burkhalter at oracle.com Tue Jan 26 16:13:06 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 26 Jan 2016 08:13:06 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56A76652.5090500@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A76652.5090500@oracle.com> Message-ID: <56A79B12.7050803@oracle.com> On 1/26/16 4:28 AM, Alan Bateman wrote: > > On 26/01/2016 04:57, Mikael Vidstedt wrote: >> >> I've finally found some time to return to this and have a new version >> of the patch which looks more promising: >> >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.02/webrev/ >> >> This uses memcpy to read/write the native data and the preliminary >> benchmark numbers on linux/x64 shows the expected performance. I'll >> work on verifying that it doesn't impact other platforms negatively >> over the next days/weeks. >> >> Btw, I also played around with implementing it in pure Java, but >> there are some issues getting C2 to correctly vectorize the loop >> given the native data accesses, so the native code seems to be needed >> for now. > This looks good and maybe it was a good thing that the compiler > upgrade ran into this. We eliminated most of the native methods in > this area a few years ago but had to leave the byte swapping methods. > It would be good to eliminate those some day. I concur that this looks OK. Don't forget to update the latest copyright year to 2016. Thanks, Brian From chris.hegarty at oracle.com Tue Jan 26 16:27:51 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 26 Jan 2016 16:27:51 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A78F8E.5020402@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> Message-ID: <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> Latest webrev updated in-place: http://cr.openjdk.java.net/~chegar/8148117/ * to execute the run method requires an appropriate permission * reverted any copyright changes ( leave to a bulk update ) * updated the test to remove the script -Chris. On 26 Jan 2016, at 15:23, Alan Bateman wrote: > On 26/01/2016 13:55, Chris Hegarty wrote: >> It is wonderful to see the various ideas on this thread about the longer >> term solution to the prompt releasing of direct buffer native memory. I >> do not want to obstruct that ( it is very informative ), but I?d like to warp up >> the review on the actual moving of Cleaner. To that end, I?ve update the >> webrev as per Alan?s comments and suggestion ( to extend Runnable ). >> >> http://cr.openjdk.java.net/~chegar/8148117/ >> >> -Chris. >> >> > This looks okay. As a defensive-in-depth then Cleaner::run can do a permission check and should ease concerns about leakage. From Roger.Riggs at Oracle.com Tue Jan 26 16:36:07 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 26 Jan 2016 11:36:07 -0500 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> Message-ID: <56A7A077.5020503@Oracle.com> Hi Chris, Looks good, thanks for updating the test. One typo: "Unexpected *exist *code" Roger On 1/26/2016 11:27 AM, Chris Hegarty wrote: > Latest webrev updated in-place: > http://cr.openjdk.java.net/~chegar/8148117/ > > * to execute the run method requires an appropriate permission > * reverted any copyright changes ( leave to a bulk update ) > * updated the test to remove the script > > -Chris. > > > On 26 Jan 2016, at 15:23, Alan Bateman wrote: > >> On 26/01/2016 13:55, Chris Hegarty wrote: >>> It is wonderful to see the various ideas on this thread about the longer >>> term solution to the prompt releasing of direct buffer native memory. I >>> do not want to obstruct that ( it is very informative ), but I?d like to warp up >>> the review on the actual moving of Cleaner. To that end, I?ve update the >>> webrev as per Alan?s comments and suggestion ( to extend Runnable ). >>> >>> http://cr.openjdk.java.net/~chegar/8148117/ >>> >>> -Chris. >>> >>> >> This looks okay. As a defensive-in-depth then Cleaner::run can do a permission check and should ease concerns about leakage. > > From chris.hegarty at oracle.com Tue Jan 26 16:39:41 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 26 Jan 2016 16:39:41 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A7A077.5020503@Oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> <56A7A077.5020503@Oracle.com> Message-ID: <3D08E357-04D2-4C5D-98C3-DFBC772654C8@oracle.com> On 26 Jan 2016, at 16:36, Roger Riggs wrote: > Hi Chris, > > Looks good, thanks for updating the test. > > One typo: > "Unexpected exist > code? D?oh. Fixed in-place. -Chris > Roger > > > > On 1/26/2016 11:27 AM, Chris Hegarty wrote: >> Latest webrev updated in-place: >> >> http://cr.openjdk.java.net/~chegar/8148117/ >> >> >> * to execute the run method requires an appropriate permission >> * reverted any copyright changes ( leave to a bulk update ) >> * updated the test to remove the script >> >> -Chris. >> >> >> On 26 Jan 2016, at 15:23, Alan Bateman >> >> wrote: >> >> >>> On 26/01/2016 13:55, Chris Hegarty wrote: >>> >>>> It is wonderful to see the various ideas on this thread about the longer >>>> term solution to the prompt releasing of direct buffer native memory. I >>>> do not want to obstruct that ( it is very informative ), but I?d like to warp up >>>> the review on the actual moving of Cleaner. To that end, I?ve update the >>>> webrev as per Alan?s comments and suggestion ( to extend Runnable ). >>>> >>>> >>>> http://cr.openjdk.java.net/~chegar/8148117/ >>>> >>>> >>>> -Chris. >>>> >>>> >>>> >>> This looks okay. As a defensive-in-depth then Cleaner::run can do a permission check and should ease concerns about leakage. >>> >> >> >> > From uschindler at apache.org Tue Jan 26 16:49:26 2016 From: uschindler at apache.org (Uwe Schindler) Date: Tue, 26 Jan 2016 17:49:26 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> Message-ID: <000401d15859$87224a00$9566de00$@apache.org> Hi, API changes l and security check look good to me. I don't have time to compile and test a JDK, but I trust you that it works :-) Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: Chris Hegarty [mailto:chris.hegarty at oracle.com] > Sent: Tuesday, January 26, 2016 5:28 PM > To: Alan Bateman ; Roger Riggs > ; uschindler at apache.org > Cc: core-libs-dev > Subject: Re: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref > > Latest webrev updated in-place: > http://cr.openjdk.java.net/~chegar/8148117/ > > * to execute the run method requires an appropriate permission > * reverted any copyright changes ( leave to a bulk update ) > * updated the test to remove the script > > -Chris. > > > On 26 Jan 2016, at 15:23, Alan Bateman wrote: > > > On 26/01/2016 13:55, Chris Hegarty wrote: > >> It is wonderful to see the various ideas on this thread about the longer > >> term solution to the prompt releasing of direct buffer native memory. I > >> do not want to obstruct that ( it is very informative ), but I?d like to warp > up > >> the review on the actual moving of Cleaner. To that end, I?ve update the > >> webrev as per Alan?s comments and suggestion ( to extend Runnable ). > >> > >> http://cr.openjdk.java.net/~chegar/8148117/ > >> > >> -Chris. > >> > >> > > This looks okay. As a defensive-in-depth then Cleaner::run can do a > permission check and should ease concerns about leakage. > From paul.sandoz at oracle.com Tue Jan 26 17:49:12 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 18:49:12 +0100 Subject: 8148250 Stream.limit parallel ordered performance In-Reply-To: <1059144622.20160126215143@gmail.com> References: <919184407.20160124173651@gmail.com> <1059144622.20160126215143@gmail.com> Message-ID: <22597C98-6A73-4DA0-BFA5-CD3C0C14C30F@oracle.com> > On 26 Jan 2016, at 16:51, Tagir F. Valeev wrote: > > Hello! > > Thank you for review! Here's the issue: > https://bugs.openjdk.java.net/browse/JDK-8148250 > Will post complete webrev later. > > PS> Note that it is still the case that in almost all scenarios this > PS> is likely to be a bad form of parallel stream. > > Well not always it's possible to estimate in advance the size of the > stream. Consider that we have user-specified filter upstream which > searches over the big collection (we want to return first 10 search > results, order is important): > > data.parallelStream() > .filter(element -> userQuery.test(element)) > .limit(10).collect(toList()); > > If user query produces 10-15 results, using parallel stream is very > reasonable, but if it produces millions of results it should not > regress very much (at least should not become much slower than > sequential version which is what we see currently). I have my doubts that the cost of splitting and filtering a small number of elements concurrently will pay off in the majority of scenarios, hence the ?almost all?. It could work in cases where there is lots of data to be filtered and only a few items are reported that are proportionally spread out, or over small data and the filter operation is costly. In any case it?s good to avoid the OOME, i am very glad you found a simple way to resolve that. > > PS> I think the comment you refer to still applies but now for larger n, so we should refine it. > > Should we replace "regardless of the value of n" with "when > n*parallelismLevel is sufficiently large?? Yes, when N * P is sufficiently large e.g to pluck a number out of the air > 2^32, say Paul. From paul.sandoz at oracle.com Tue Jan 26 17:50:12 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 26 Jan 2016 18:50:12 +0100 Subject: RFR 8147505: Clarification of BaseStream.onClose() behavior In-Reply-To: <1832946052.20160126215720@gmail.com> References: <1666489469.20160115103543@gmail.com> <68E50067-D3FB-414F-80AD-40F188DAE5FC@oracle.com> <1616962894.20160116091332@gmail.com> <0886057.20160123163820@gmail.com> <757F647A-DE7A-49A3-BC0F-938B4D0E8CD2@oracle.com> <1832946052.20160126215720@gmail.com> Message-ID: <24123466-DA0F-4673-AFA9-41AC6A28FECF@oracle.com> > On 26 Jan 2016, at 16:57, Tagir F. Valeev wrote: > > Hello! > > Thanks for review! Updated webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8147505/r2/ > Removed redundant test and added a comment. > Thanks, it?s in my queue, Paul. From Alan.Bateman at oracle.com Tue Jan 26 17:53:57 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 26 Jan 2016 17:53:57 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> Message-ID: <56A7B2B5.3080804@oracle.com> On 26/01/2016 16:27, Chris Hegarty wrote: > Latest webrev updated in-place: > http://cr.openjdk.java.net/~chegar/8148117/ > > * to execute the run method requires an appropriate permission > * reverted any copyright changes ( leave to a bulk update ) > * updated the test to remove the script > > -Chris. > This version looks good to me. -Alan. From martinrb at google.com Tue Jan 26 18:10:28 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 26 Jan 2016 10:10:28 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: <56A133FC.4040806@oracle.com> <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> <56A2AD9D.70802@oracle.com> Message-ID: On Tue, Jan 26, 2016 at 5:44 AM, Jason Mehrens wrote: > Hi Martin, > > Are you sure about the change where addElement is now calling the public (non-final) version of add? I would think that we would want to avoid any type of compatibility changes with regard to Vector. Jason, you are right that we shouldn't be changing observable calls to public methods. From martinrb at google.com Tue Jan 26 18:25:00 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 26 Jan 2016 10:25:00 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: <1482625420.744530.1453811301415.JavaMail.zimbra@u-pem.fr> References: <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> <56A2AD9D.70802@oracle.com> <1482625420.744530.1453811301415.JavaMail.zimbra@u-pem.fr> Message-ID: webrev refreshed - http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Vector-grow/ On Tue, Jan 26, 2016 at 4:28 AM, Remi Forax wrote: > Hi Martin, > sorry for jumping on this conversation lately, > please don't be seduced by the dark side, Too late! I've already committed such changes to ArrayList.java. No one would do this sort of bytecode golf if it didn't have measurable effects on microbenchmarks. We've been asking hotspot maintainers to kill their method-bytecode-size heuristics for a long time, but still no sign of it happening. > I understand that the code is 1 byte bigger, but i think that the following code > final int s = this.size; > Object[] elementData = this.elementData; > if (s == elementData.length) { > elementData = grow(); > } > > is more readable than > final int s; > Object[] elementData; > if ((s = size) == (elementData = this.elementData).length) > elementData = grow(); Vector.insertElementAt is not performance critical, and not near MaxInlineSize, so I did as you ask. In core libraries readability is slightly less important than performance. From john.r.rose at oracle.com Tue Jan 26 18:32:56 2016 From: john.r.rose at oracle.com (John Rose) Date: Tue, 26 Jan 2016 10:32:56 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56A736B7.4020601@redhat.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> Message-ID: <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> On Jan 26, 2016, at 1:04 AM, Andrew Haley wrote: > > I agree that memcpy is the right thing to use. It's portable and is > inlined well on production-quality C compilers. But it is not strong enough to uphold the Java memory model, because it is allows to copy byte-wise, which can tear shorts, ints, or longs, creating illegal race states. So we try to avoid memcpy when we can. ? John From aph at redhat.com Tue Jan 26 18:48:27 2016 From: aph at redhat.com (Andrew Haley) Date: Tue, 26 Jan 2016 18:48:27 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> Message-ID: <56A7BF7B.5020006@redhat.com> On 01/26/2016 06:32 PM, John Rose wrote: > On Jan 26, 2016, at 1:04 AM, Andrew Haley wrote: >> >> I agree that memcpy is the right thing to use. It's portable and is >> inlined well on production-quality C compilers. > > But it is not strong enough to uphold the Java memory model, > because it is allows to copy byte-wise, which can tear shorts, > ints, or longs, creating illegal race states. > > So we try to avoid memcpy when we can. Yes, I see. I guess the best we can do is something like the fun and games in Unsafe.{get, put}LongUnaligned(), which always do the best they can to align everything. Andrew. From john.r.rose at oracle.com Tue Jan 26 19:04:01 2016 From: john.r.rose at oracle.com (John Rose) Date: Tue, 26 Jan 2016 11:04:01 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56A7BF7B.5020006@redhat.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> Message-ID: <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> On Jan 26, 2016, at 10:48 AM, Andrew Haley wrote: > > On 01/26/2016 06:32 PM, John Rose wrote: >> On Jan 26, 2016, at 1:04 AM, Andrew Haley wrote: >>> >>> I agree that memcpy is the right thing to use. It's portable and is >>> inlined well on production-quality C compilers. >> >> But it is not strong enough to uphold the Java memory model, >> because it is allows to copy byte-wise, which can tear shorts, >> ints, or longs, creating illegal race states. >> >> So we try to avoid memcpy when we can. > > Yes, I see. I guess the best we can do is something like the fun and > games in Unsafe.{get, put}LongUnaligned(), which always do the best > they can to align everything. Unsafe.copyMemory bottoms out to Copy::conjoint_memory_atomic. IMO that's a better starting point than memcpy. Perhaps it can be given an additional parameter (or overloading) to specify a swap size. ? John From aph at redhat.com Tue Jan 26 19:08:00 2016 From: aph at redhat.com (Andrew Haley) Date: Tue, 26 Jan 2016 19:08:00 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> Message-ID: <56A7C410.4040301@redhat.com> On 01/26/2016 07:04 PM, John Rose wrote: > Unsafe.copyMemory bottoms out to Copy::conjoint_memory_atomic. > IMO that's a better starting point than memcpy. Perhaps it can be > given an additional parameter (or overloading) to specify a swap size. OK, but conjoint_memory_atomic doesn't guarantee that destination words won't be torn if their source is misaligned: in fact it guarantees that they will will be. Andrew. From john.r.rose at oracle.com Tue Jan 26 19:13:32 2016 From: john.r.rose at oracle.com (John Rose) Date: Tue, 26 Jan 2016 11:13:32 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56A7C410.4040301@redhat.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> Message-ID: <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> On Jan 26, 2016, at 11:08 AM, Andrew Haley wrote: > > On 01/26/2016 07:04 PM, John Rose wrote: >> Unsafe.copyMemory bottoms out to Copy::conjoint_memory_atomic. >> IMO that's a better starting point than memcpy. Perhaps it can be >> given an additional parameter (or overloading) to specify a swap size. > > OK, but conjoint_memory_atomic doesn't guarantee that destination > words won't be torn if their source is misaligned: in fact it > guarantees that they will will be. That's a good point, and argues for a new function with the stronger guarantee. Actually, it would be perfectly reasonable to strengthen the guarantee on the existing function. I don't think anyone will care about the slight performance change, especially since it is probably favorable. Since it's Unsafe, they are not supposed to care, either. ? John From rafael.wth at gmail.com Tue Jan 26 19:33:34 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Tue, 26 Jan 2016 20:33:34 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <56A766AA.8050906@oracle.com> References: <56A1F24C.7050205@redhat.com> <56A23615.6090105@oracle.com> <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> <56A766AA.8050906@oracle.com> Message-ID: Hi Vladimir, I have previously explored what you suggest (instrumenting the lambda exception's method bodies) but decided against it, mainly for three reasons: 1. Instrumentation has to be done more eagerly: To avoid redefinition (and the constraints involved), the instrumentation has to be applied once the lambda's containing class is loaded. The lambda expression's instrumentation only needs to be applied once the lambda class is generated. 2. By instrumenting the containing class, one cannot add "per-instance" fields of a lambda class any more: For example, one might like to introduce a class that contains a counter for how often a "lambda object"'s method was called. This is more difficult if that counter field needs to be contained in the lambda's defining instance or class. 3. The implementation effort is a magnifold of simply redefining the LambdaMetafactory. Especially in non-commercial software where time is the biggest constraint, this is very relevant. I have also looked into the possibility of introducing a double-indirection to retain the original LambdaMetafactory but to rather introduce an intermediate factory where all invokedynamic call sites would first implement custom code and then were subsequently dispatched with a replaced method handle. This solves problem (2) but not (1, 3) of the above mentioned implications. As for production quality: It is not bad at all, I tested on several versions now and starting from u40, it seems to run quite stable despite for the problem of the wrong byte array being provided to a retransformation. Good job! Best regards, Rafael Am 26.01.2016 1:29 nachm. schrieb "Vladimir Ivanov" < vladimir.x.ivanov at oracle.com>: > what bothers me about instrumenting a lambda expression's target >> method is the difficulty of locating the method that contains the code >> and setting it into context. This is a lot of work to do since one >> needs to first locate any invokedynamic call sites and interpret the >> connection via the LambdaMetafactory. This is difficult to instrument >> without greater efforts and does not feel like a clean solution. >> >> Many real life instrumentations are based on the assumption of being >> able to instrument any class of a given subtype or other any other >> class property. For example, one would want to instrument any class >> that implements InputStream to discover a resource leak. If the >> interface is however functional and implemented as a lambda >> expression, a tool that manipulating all class files of this type >> stops working. At the same time, end users do not really understand >> the difference of their former anonymous class that is now expressed a >> lambda expression and perceive the beaviour as a regression. >> > I agree, it becomes more complex without the ability to instrument any > class: not only you have to enumerate all loaded lasses, but also inspect > all lambda expressions. It means dedicated support for lambda expressions, > but IMO the complications are far from insurmountable and makes the design > cleaner. Otherwise, you have to support reliable instrumentation of VM > anonymous classes, which is harder (again, IMO). > > Also, you still need dedicated support for lambdas to be able to > instrument them individually. Ability to instrument VM anonymous classes > doesn't cover that (only the case of instrument all classes fulfilling some > requirement). > > But I'd like to hear your opinion about that. Have you tried to instrument > lambda expression bodies instead of tweaking LambdaMetafactory > implementation? > > I have myself implemented a custom solution for my code generation >> library that instruments the LambdaMetafactory to apply class file >> transformers to these classes manually. This does however show two >> disadvantages: >> >> 1. javac creates the target method of a lambda expression as a private >> method. The only way to implement a lambda expression legally in byte >> code is by loading the generated implementation anonymously what >> requires non-public API. While it is possible to call >> Unsafe::defineAnonymousClass from the instrumented LambdaMetafactory, >> it is again required to use non-standardized APIs which might fail on >> differing implementations of the JVM. >> > No, U::dAC usage is just an implementation detail. It is not required for > lambda expression support. What is crucial is the ability to construct > appropriate method handle to the method which represents the body. That's > where invokedynamic bootstrap methods helps. Lookup object (1st argument > [1]) has enough privileges to construct method handles to any methods (even > private) in the caller. After that you are free to decide how to wrap the > method handle to the lambda body. > > 2. At the same time, there is no standardized way to receive all >> ClassFileTransformers that are currently registered on the VM. this >> also requires calls into internal APIs that are not necessarily >> supported on other platforms. >> >> I fully understand the hesitation to support this from a technical >> point of view but in reality, people are already dependant on this >> feature and disallowing the instrumentation of lambda classes will >> only inspire work-arrounds that reduce the stability of software using >> this APIs that does currently work without problems for non-lambda >> classes. >> > As your experience demonstrates, instrumentation support of VM anonymous > classes is far from production quality. If anybody heavily relies on it, > then they should face the same problems. But you are the first who stumbled > upon and reported them. So, it seems we are lucky and can decide what is > the correct behavior (and not just stick with how it works now). > > Best regards, Rafael >> >> PS: While implementing my solution, I found that the LambdaMetafactory >> of the Open JDK creates private, final methods for the serialization >> bits. The methods should however not be final as they are already >> private. >> > There's no such thing as too much safety, right? ;-) > > Best regards, > Vladimir Ivanov > > [1] public class LambdaMetafactory { > > public static CallSite metafactory( > MethodHandles.Lookup caller, > String invokedName, > MethodType invokedType, > MethodType samMethodType, > MethodHandle implMethod, > MethodType instantiatedMethodType) > > >> >> 2016-01-23 13:55 GMT+01:00 Remi Forax : >> >>> I agree with Vladimir, >>> >>> You should not be able to transform/redefine a VM anonymous class >>> because the transformer will certainly mess up with the order of the >>> constant pool entries. >>> >>> Slightly off-topic, about ASM, when you create a ClassWriter [1], you >>> can pass a ClassReader of an existing class, in that case ASM copy the >>> constant pool from the class reader to the class writer so the constant >>> pool is preserved. >>> >>> R?mi >>> >>> [1] >>> http://asm.ow2.org/asm50/javadoc/user/org/objectweb/asm/ClassWriter.html#ClassWriter%28org.objectweb.asm.ClassReader,%20int%29 >>> >>> ----- Mail original ----- >>> >>>> De: "Vladimir Ivanov" >>>> ?: "Rafael Winterhalter" >>>> Cc: "Coleen Phillimore" , >>>> core-libs-dev at openjdk.java.net, "serguei.spitsyn at oracle.com >>>> Spitsyn" , "Daniel Daugherty" < >>>> daniel.daugherty at oracle.com> >>>> Envoy?: Vendredi 22 Janvier 2016 18:47:31 >>>> Objet: Re: ClassFileTransformer does not apply to anonymous classes >>>> >>>> Rafael, >>>> >>>> First of all, I'd like to agree on the terminology. There's some >>>> confusion there. Anonymous term is ambiguous in Java. There are >>>> anonymous classes on language level and there's >>>> Unsafe.defineAnonymousClass() which produce anonymous-in-VM-sense >>>> classes. I prefer to call them VM anonymous classes to avoid confusion. >>>> >>>> I assume that whenever you use anonymous you assume "VM anonymous". >>>> >>>> thank you for your response. While I completely understand your view >>>>> from a VM implementor's point of view, as a practicioner I would >>>>> recommend against it. Not being able to instrument lambda expressions >>>>> puts a severe limitation onto using the instrumentation API >>>>> alltogether. For example, a monitoring application that promises to >>>>> record all invocations of a method of a certain interface that only >>>>> works 95% of the time is not 5% less usefull but might no longer be >>>>> useful at all. People have build large applications based on the >>>>> assumption that all user application code can be instrumented and such >>>>> a regression would hurt them. For example, until today, over 30 people >>>>> that use my code generation framework reached out to me and reported >>>>> the reported behavior as a bug in my library and people are only >>>>> starting to migrate their applications to Java 8. The outcome is >>>>> simply not intuitive as using a lambda expression over an anonyous >>>>> inner class should not change an application's behavior. >>>>> >>>> Can you elaborate on that point, please? >>>> >>>> I don't see any problems with instrumenting user code. Javac represents >>>> lambda expression body as a private static method of the enclosing >>>> class, which can be instrumented. VM anonymous classes are used only as >>>> a mechanism to glue functional interfaces and lambda expressions >>>> together at runtime. >>>> >>>> For example: >>>> static void f(Runnable r) { r.run(); } >>>> f(() -> {}); >>>> >>>> Test::f (7 bytes) >>>> @ 1 Test$$Lambda$1/791452441::run (4 bytes) inline (hot) >>>> @ 0 Test::lambda$main$0 (1 bytes) inline (hot) >>>> >>>> Why do you need to instrument Test$$Lambda$1/... and not >>>> Test::lambda$main$0? >>>> >>>> The currently used workaround that people use is to instrument the >>>>> LambdaMetaFactory class itself to apply the transformer manually when >>>>> it is being created. This solution is spreading as a standard approach >>>>> and I am sure that forbidding a redefinition alltogether would only >>>>> inspire to other workarrounds for being able to migrate running code >>>>> to the next Java version. >>>>> >>>> It doesn't sound like a workaround, but as a solution for a different >>>> problem. >>>> >>>> If there's a need to gather profile for every lambda expression >>>> instantiation site (indy call), then redefining bootstrap method is the >>>> right way to go IMO. It allows to intercept and customize indy call site >>>> binding. >>>> >>>> What does sound as a workaround is an attempt to instrument classes >>>> produced by LambdaMetaFactory. Right now, it generates a fresh VM >>>> anonymous class on every request, but it is an implementation detail and >>>> not a requirement. >>>> >>>> For example, LambdaMetaFactory can reuse wrappers on per-functional >>>> interface basis. It would provide significant footprint savings for >>>> lambda expression-rich code bases (usually, there are much more >>>> instantiations than functional interface flavors). >>>> >>>> Furthermore, I do not think that not being able to patch constant pool >>>>> indices in the generated code introduces any problems in practice. >>>>> Most real-world instrumentation only appends new constant pool entries >>>>> without interfering with the existant pool. Rather, I would like to >>>>> see an exception being raised if one attempts to change the original >>>>> constant pool without me being able to consider this from a technical >>>>> perspective. I think this would serve to be sufficient for most >>>>> people. >>>>> >>>> If you care only about lambda expressions, then constant pool patching >>>> shouldn't be a problem (AFAIR it isn't used for lambda expressions). But >>>> in a more generic case (even for java.lang.invoke purposes), the >>>> coupling between class file and CP patches is very tight and can be >>>> easily broken with benign class file transformations. Considering ASM >>>> library, I'm not sure that even simple instrumentations preserve >>>> original constant pool structure. >>>> >>>> Probably, VM can do some structural checks on CP to forbid any >>>> modifications except appends, but there are still aliasing problems - >>>> sharing doesn't work (even for string constants), so new code shouldn't >>>> use original CP entries. >>>> >>>> I'd prefer to avoid such increase in complexity in VM implementation. >>>> >>>> I really hope that there is a way to allow for patching anonymously >>>>> loaded classes even without exposing the constant pool patches. >>>>> >>>> From VM implementation perspective I don't see any problems with >>>> allowing class file redefinition for VM anonymous classes. What I want >>>> to do is to discourage from doing so. Again - it is very fragile and >>>> there's little we can do to fix that. >>>> >>>> So far, thank you for looking into this. I am sure this a more complex >>>>> matter than what I am able to comprehend. I appreciate that you are >>>>> taking the time to consider my opinion! >>>>> >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> From martinrb at google.com Tue Jan 26 20:31:26 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 26 Jan 2016 12:31:26 -0800 Subject: API review of VarHandles In-Reply-To: <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Message-ID: There's a big "expectations" effect here. j.l.invoke is "supposed to be" for making dynamic languages less slow, not for making low-level, ultra-non-dynamic operations faster. Asking the Unsafe users of the world to switch to dynamic VarHandle is like asking C programmers to rewrite their code in perl 6 ... for performance! It's the same "srsly?" feeling one gets reading """We can currently use RPerl to speed up low-magic Perl 5 code with over 300x performance gain.""" From stuart.marks at oracle.com Tue Jan 26 20:59:40 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 26 Jan 2016 12:59:40 -0800 Subject: RFR: JDK-8146568 NegativeArraySizeException in ArrayList.grow(int) In-Reply-To: References: <56A15D62.6020303@oracle.com> <56A26F0D.7010303@oracle.com> <56A2AD9D.70802@oracle.com> <1482625420.744530.1453811301415.JavaMail.zimbra@u-pem.fr> Message-ID: <56A7DE3C.1090404@oracle.com> On 1/26/16 4:28 AM, Remi Forax wrote: > please don't be seduced by the dark side, Clearly, he feels the call from light. :-) On 1/26/16 10:25 AM, Martin Buchholz wrote: > webrev refreshed - > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/Vector-grow/ Hi Martin, Thanks for the updates. Looks like Jason and Remi already looked at it, but in case you were waiting for me, I looked at it too. Looks fine. > In core libraries readability is slightly less important than performance. For some values of "slightly." :-) s'marks From brian.goetz at oracle.com Tue Jan 26 21:46:15 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 26 Jan 2016 16:46:15 -0500 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Message-ID: <56A7E927.2070909@oracle.com> I think that expectation is just out of date (if not outright mistaken.) Yes, j.l.i was originally called "java.dyn", but prior to shipping *7* we renamed it to j.l.i precisely because it had turned into a general customizable linkage mechanism that was usable far beyond dynamic languages. (To wit, Java 8 lambdas got a significant linkage/capture boost from using invokedynamic.) So to claim it's "supposed to be" only for dynamic languages represents a significant "missing of the memo." On 1/26/2016 3:31 PM, Martin Buchholz wrote: > There's a big "expectations" effect here. j.l.invoke is "supposed to > be" for making dynamic languages less slow, not for making low-level, > ultra-non-dynamic operations faster. Asking the Unsafe users of the > world to switch to dynamic VarHandle is like asking C programmers to > rewrite their code in perl 6 ... for performance! It's the same > "srsly?" feeling one gets reading """We can currently use RPerl to > speed up low-magic Perl 5 code with over 300x performance gain.""" From michael.haupt at oracle.com Tue Jan 26 22:11:14 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Tue, 26 Jan 2016 23:11:14 +0100 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> Message-ID: <95DCB2A7-D773-4601-BAAC-282AE7DC1D15@oracle.com> Hi Martin, how about seeing j.l.i as a framework for lightweight reflection on the one hand, and for method handle-based meta-programming on the other? That's clearly usable beyond the domain of dynamic language implementation. Granted, the latter remains an important application area, but there are others. For example, JEP 280 uses invokedynamic for string concatenation. In that vein, how about seeing VarHandles as a safer replacement for some of the Unsafe API? It's going to be public API, readily usable without having to enable access to the carefully tucked-away Unsafe. Best, Michael > Am 26.01.2016 um 21:31 schrieb Martin Buchholz : > > There's a big "expectations" effect here. j.l.invoke is "supposed to > be" for making dynamic languages less slow, not for making low-level, > ultra-non-dynamic operations faster. Asking the Unsafe users of the > world to switch to dynamic VarHandle is like asking C programmers to > rewrite their code in perl 6 ... for performance! It's the same > "srsly?" feeling one gets reading """We can currently use RPerl to > speed up low-magic Perl 5 code with over 300x performance gain.""" -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From mandy.chung at oracle.com Tue Jan 26 23:29:29 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 26 Jan 2016 15:29:29 -0800 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> Message-ID: <6A5E6D87-1BAA-4A99-BBEC-9E6396A9C2BB@oracle.com> > On Jan 26, 2016, at 8:27 AM, Chris Hegarty wrote: > > Latest webrev updated in-place: > http://cr.openjdk.java.net/~chegar/8148117/ > > * to execute the run method requires an appropriate permission > * reverted any copyright changes ( leave to a bulk update ) > * updated the test to remove the script Catching up on this review thread. The patch looks good. Having jdk.internal.ref.Cleaner to implement Runnable with the permission check looks reasonable. Mandy From john.r.rose at oracle.com Tue Jan 26 23:38:29 2016 From: john.r.rose at oracle.com (John Rose) Date: Tue, 26 Jan 2016 15:38:29 -0800 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: References: <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> <147644520.594985.1453798977640.JavaMail.zimbra@u-pem.fr> <710812902.735970.1453809790007.JavaMail.zimbra@u-pem.fr> Message-ID: <23E542DE-E816-4565-85E7-41D0AD05B419@oracle.com> VM anonymous classes are an implementation detail that is opaque to system components except for the lowest layers of the JDK runtime and the JVM itself. Transformers and other instrumentation should not look inside them expecting to interpose on their behavior. Ideally we should not make them visible at all, but sometimes it helps (e.g., with single stepping through BCs). VM anonymous classes may be (and are) replaced or interchanged unpredictably with similar mechanisms, such as JNI-based reflection, or indirect invocation via MemberName tokens. You can't rely on any of this meaning what you think it means, even if it appears to have a classfile structure. Even if you were able to "transform" one of these classfiles, it wouldn't necessary do what you think it should do, because its structure is a private internal-only contract of the JDK and JVM. And, as you probably have already noticed, the number and structure of these VMAC classfiles change over time. We may (at some point) replace the classes with some completely different internal representation, which, even if it is visible somehow to instrumentation, cannot meaningfully be parsed and re-implemented. Likewise, lambdas are translated into inner classes, but this also can change at any time; the metafactory API makes few or no promises as to the internal structure of the invokedynamic binding. In fact, some JVMs use special polymorphic nodes, instead of the standard inner-class translation. Suddenly, some or all of these classfiles may disappear, when the runtime begins to optimize them differently. Please don't lead your users to rely on them. I second Vladimir's suggestion, that the only sane way to interpose on lambdas is to transform the class that defines the lambda (including perhaps the parameters of the indy that create the lambda), and not dig into system internals. System internals are nothing like user code, and cannot be transformed like user code. Sorry to bear bad news, ? John From steve.drach at oracle.com Wed Jan 27 01:56:12 2016 From: steve.drach at oracle.com (Steve Drach) Date: Tue, 26 Jan 2016 17:56:12 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56A6532D.6050805@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> Message-ID: <5112CC4C-2733-4982-A340-01D010F2C8EC@oracle.com> > On Jan 25, 2016, at 8:54 AM, Alan Bateman wrote: Somehow I missed this, sorry for the delayed response. >> >> Changed to BASE, i.e. Release.BASE >> > This looks better. Release.BASE is probably okay although it still feels like Release.UNVERSIONED, esp. when it is defined as "Represents unversioned entries?. Base entries imply to me the entries that are the ?base? of the jar file. All multi-release jar files have to have a set of base entries that, as a whole, export the public API of the jar file (whether it?s multi-release or not). Versioned entries ?override? base entries. I could have said ?Represents base entries? but that seems a little circular. Actually base entries are the set of root entries minus the set of entries in the META-INF/versions directory ;-) > > I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. To me they are a resource in the root directory, but I see your point. > I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? Let me see what I can do. From ramanand.patil at oracle.com Wed Jan 27 04:51:14 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Tue, 26 Jan 2016 20:51:14 -0800 (PST) Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale In-Reply-To: <575e40e1-9041-424a-8057-f8c289e032ae@default> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> <575e40e1-9041-424a-8057-f8c289e032ae@default> Message-ID: Hi all, Please help me in reviewing this test fix. Regards, Ramanand. From: Ramanand Patil Sent: Monday, January 25, 2016 1:05 PM To: i18n-dev at openjdk.java.net Cc: core-libs-dev at openjdk.java.net Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale Hi all, Please review the trivial test bug fix for: https://bugs.openjdk.java.net/browse/JDK-8147912 Bug Description: Since the test case "parseWithZoneWithoutOffset" is using hard code as input data it will fail on some non-English locales (de_DE locale). Webrev: http://cr.openjdk.java.net/~rpatil/8147912/webrev.00 Fix: Even though hardcoded data is not preferred in compatibility test cases, this case was exception. English is provided as the default locale data for DateTimeFormatter in this testcase. Regards, Ramanand. From masayoshi.okutsu at oracle.com Wed Jan 27 05:53:13 2016 From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu) Date: Wed, 27 Jan 2016 14:53:13 +0900 Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale In-Reply-To: References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> <575e40e1-9041-424a-8057-f8c289e032ae@default> Message-ID: <56A85B49.4040704@oracle.com> Looks OK to me. But I'd like some java.time people to review this change to see if the intention of this test is to run only in English. Thanks, Masayoshi On 1/27/2016 1:51 PM, Ramanand Patil wrote: > Hi all, > > > > Please help me in reviewing this test fix. > > > > Regards, > > Ramanand. > > From: Ramanand Patil > Sent: Monday, January 25, 2016 1:05 PM > To: i18n-dev at openjdk.java.net > Cc: core-libs-dev at openjdk.java.net > Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale > > > > Hi all, > > Please review the trivial test bug fix for: https://bugs.openjdk.java.net/browse/JDK-8147912 > > Bug Description: Since the test case "parseWithZoneWithoutOffset" is using hard code as input data it will fail on some non-English locales (de_DE locale). > > Webrev: http://cr.openjdk.java.net/~rpatil/8147912/webrev.00 > > Fix: Even though hardcoded data is not preferred in compatibility test cases, this case was exception. English is provided as the default locale data for DateTimeFormatter in this testcase. > > Regards, > Ramanand. From ramanand.patil at oracle.com Wed Jan 27 06:33:42 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Tue, 26 Jan 2016 22:33:42 -0800 (PST) Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale In-Reply-To: <56A85B49.4040704@oracle.com> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> <575e40e1-9041-424a-8057-f8c289e032ae@default> <56A85B49.4040704@oracle.com> Message-ID: Hi Masayoshi, Thank you for review. This test was contributed by me for bug HYPERLINK "https://bugs.openjdk.java.net/browse/JDK-8066982"JDK-8066982. Running this only in English should be Ok as per me, since this was added just to test one of the parsing scenario when Zone information is available without Offset. Anyway, I will wait for java.time people to review this change. Regards, Ramanand. -----Original Message----- From: Masayoshi Okutsu Sent: Wednesday, January 27, 2016 11:23 AM To: Ramanand Patil; i18n-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Subject: Re: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale Looks OK to me. But I'd like some java.time people to review this change to see if the intention of this test is to run only in English. Thanks, Masayoshi On 1/27/2016 1:51 PM, Ramanand Patil wrote: > Hi all, > > > > Please help me in reviewing this test fix. > > > > Regards, > > Ramanand. > > From: Ramanand Patil > Sent: Monday, January 25, 2016 1:05 PM > To: HYPERLINK "mailto:i18n-dev at openjdk.java.net"i18n-dev at openjdk.java.net > Cc: HYPERLINK "mailto:core-libs-dev at openjdk.java.net"core-libs-dev at openjdk.java.net > Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" > of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on > de_DE locale > > > > Hi all, > > Please review the trivial test bug fix for: > https://bugs.openjdk.java.net/browse/JDK-8147912 > > Bug Description: Since the test case "parseWithZoneWithoutOffset" is using hard code as input data it will fail on some non-English locales (de_DE locale). > > Webrev: http://cr.openjdk.java.net/~rpatil/8147912/webrev.00 > > Fix: Even though hardcoded data is not preferred in compatibility test cases, this case was exception. English is provided as the default locale data for DateTimeFormatter in this testcase. > > Regards, > Ramanand. From paul.sandoz at oracle.com Wed Jan 27 08:44:36 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 27 Jan 2016 09:44:36 +0100 Subject: API review of VarHandles In-Reply-To: <56A7E927.2070909@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <282FAEC5-CF12-43C8-A58B-E16FDAC3C9B7@oracle.com> <56A7E927.2070909@oracle.com> Message-ID: <39841106-6BC8-4CA3-9748-8DFB6F56C08E@oracle.com> > On 26 Jan 2016, at 22:46, Brian Goetz wrote: > > I think that expectation is just out of date (if not outright mistaken.) Yes, j.l.i was originally called "java.dyn", but prior to shipping *7* we renamed it to j.l.i precisely because it had turned into a general customizable linkage mechanism that was usable far beyond dynamic languages. (To wit, Java 8 lambdas got a significant linkage/capture boost from using invokedynamic.) So to claim it's "supposed to be" only for dynamic languages represents a significant "missing of the memo.? > Furthermore, Java has always been quite dynamic under the covers of the language. See also Aleksey?s recent indy string concat enhancements. In future this area is likely to be leveraged further. And i know Remi?s is always on the look out to expose indy up to into the language :-) and with good reason. Paul. > > On 1/26/2016 3:31 PM, Martin Buchholz wrote: >> There's a big "expectations" effect here. j.l.invoke is "supposed to >> be" for making dynamic languages less slow, not for making low-level, >> ultra-non-dynamic operations faster. Asking the Unsafe users of the >> world to switch to dynamic VarHandle is like asking C programmers to >> rewrite their code in perl 6 ... for performance! It's the same >> "srsly?" feeling one gets reading """We can currently use RPerl to >> speed up low-magic Perl 5 code with over 300x performance gain.""" > From rafael.wth at gmail.com Wed Jan 27 08:44:45 2016 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Wed, 27 Jan 2016 09:44:45 +0100 Subject: ClassFileTransformer does not apply to anonymous classes In-Reply-To: <23E542DE-E816-4565-85E7-41D0AD05B419@oracle.com> References: <56A24DD4.2080705@oracle.com> <56A26B33.2010308@oracle.com> <1423885283.2117833.1453553742207.JavaMail.zimbra@u-pem.fr> <147644520.594985.1453798977640.JavaMail.zimbra@u-pem.fr> <710812902.735970.1453809790007.JavaMail.zimbra@u-pem.fr> <23E542DE-E816-4565-85E7-41D0AD05B419@oracle.com> Message-ID: Hi John, thank you for your answer. As I said before, I do understand the concept behind resolving lambda expressions at runtime to not represent "real classes". I do however still disagree with your assertion. On the opposite, I argue that the fact that nobody can expect a certain form of implementation for a lambda expression makes chaging this form a viable solution in the first place. After the LambdaMetafactory is instrumented, I know that the returned class site that produces a lambda creates a "plain old Java class" and can be instrumented. I also know that I can redefine/retransform such classes without breaking anything. The only glitch in this substitution of the LambdaMetafactory is that I still need to use the Unsafe API and that such classes might no longer be retransformable after some VM update due to loading the class via Unsafe::defineAnonymousClass. Therefore, I suggest to make classes retransformable only if their constant pool is not patched, i.e. the third argument is set to null. This would suffice to allow for the described workarround that people already use. Finally, I want to stress that there is already a lot of code out there that uses instrumentation. Until Java 8, any class of a Java application could be transformed in a canonical manner. Having changed this behavior is a breaking change to a certain degree since people cannot longer update their applications without a change in behavior when lambda expressions are involved. And I do not argue that instrumenting lambdas is a good idea in the first place but existing code bases already implement a lot of bad ideas that people need to work with. Refactorings of existing code are always a consideration of cost and the harder instrumenting lambdas gets, the uglier the hacks will become and this is to nobody's benefit. This is why I hope that you can somehow support this functionality after all. Thank you for considering this! Best regards, Rafael 2016-01-27 0:38 GMT+01:00 John Rose : > VM anonymous classes are an implementation detail that is > opaque to system components except for the lowest layers of > the JDK runtime and the JVM itself. Transformers and other > instrumentation should not look inside them expecting to interpose > on their behavior. Ideally we should not make them visible at all, > but sometimes it helps (e.g., with single stepping through BCs). > > VM anonymous classes may be (and are) replaced or interchanged > unpredictably with similar mechanisms, such as JNI-based > reflection, or indirect invocation via MemberName tokens. > > You can't rely on any of this meaning what you think it means, > even if it appears to have a classfile structure. Even if you > were able to "transform" one of these classfiles, it wouldn't > necessary do what you think it should do, because its structure > is a private internal-only contract of the JDK and JVM. > > And, as you probably have already noticed, the number and > structure of these VMAC classfiles change over time. > We may (at some point) replace the classes with some > completely different internal representation, which, > even if it is visible somehow to instrumentation, cannot > meaningfully be parsed and re-implemented. > > Likewise, lambdas are translated into inner classes, but > this also can change at any time; the metafactory API > makes few or no promises as to the internal structure > of the invokedynamic binding. In fact, some JVMs use > special polymorphic nodes, instead of the standard > inner-class translation. Suddenly, some or all of these > classfiles may disappear, when the runtime begins to > optimize them differently. > > Please don't lead your users to rely on them. > > I second Vladimir's suggestion, that the only sane way > to interpose on lambdas is to transform the class that > defines the lambda (including perhaps the parameters > of the indy that create the lambda), and not dig into > system internals. System internals are nothing like > user code, and cannot be transformed like user code. > > Sorry to bear bad news, > > ? John From scolebourne at joda.org Wed Jan 27 10:26:54 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 27 Jan 2016 10:26:54 +0000 Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale In-Reply-To: <56A85B49.4040704@oracle.com> References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> <575e40e1-9041-424a-8057-f8c289e032ae@default> <56A85B49.4040704@oracle.com> Message-ID: >From the name of the test and looking at the code, running in English only is fine. Stephen On 27 January 2016 at 05:53, Masayoshi Okutsu wrote: > Looks OK to me. But I'd like some java.time people to review this change > to see if the intention of this test is to run only in English. > > Thanks, > Masayoshi > > > On 1/27/2016 1:51 PM, Ramanand Patil wrote: > >> Hi all, >> >> >> Please help me in reviewing this test fix. >> >> >> Regards, >> >> Ramanand. >> >> From: Ramanand Patil >> Sent: Monday, January 25, 2016 1:05 PM >> To: i18n-dev at openjdk.java.net >> Cc: core-libs-dev at openjdk.java.net >> Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of >> java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale >> >> >> Hi all, >> Please review the trivial test bug fix for: >> https://bugs.openjdk.java.net/browse/JDK-8147912 >> Bug Description: Since the test case "parseWithZoneWithoutOffset" is >> using hard code as input data it will fail on some non-English locales >> (de_DE locale). >> Webrev: http://cr.openjdk.java.net/~rpatil/8147912/webrev.00 >> Fix: Even though hardcoded data is not preferred in compatibility test >> cases, this case was exception. English is provided as the default locale >> data for DateTimeFormatter in this testcase. >> Regards, >> Ramanand. >> > > From ivan at azulsystems.com Wed Jan 27 13:31:21 2016 From: ivan at azulsystems.com (Ivan Krylov) Date: Wed, 27 Jan 2016 16:31:21 +0300 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() Message-ID: <56A8C6A9.8080705@azulsystems.com> Hello, Earlier there was a discussion on this mail alias about the spin loop hint proposal [1]. Based on the feedback from that discussion some changes were incorporated and the JEP has been filed [2]. There seems to be a consensus on the API side. The JEP is now in a draft state and I hope this JEP will get targeted for java 9 shortly. Please review the upcoming API changes: http://cr.openjdk.java.net/~ikrylov/8147844.jdk.00/ For the reference, the new generated JavaDoc for j.l.Runtime class and the new method http://ivankrylov.github.io/onspinwait/api/java/lang/Runtime.html#onSpinWait-- Thanks, Ivan 1 - http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-October/thread.html#35613 2 - https://bugs.openjdk.java.net/browse/JDK-8147832. From andrej.golovnin at gmail.com Wed Jan 27 13:48:39 2016 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Wed, 27 Jan 2016 14:48:39 +0100 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: <56A8C6A9.8080705@azulsystems.com> References: <56A8C6A9.8080705@azulsystems.com> Message-ID: Hi Ivan, > Please review the upcoming API changes: > http://cr.openjdk.java.net/~ikrylov/8147844.jdk.00/ I think the semicolon at the end of the line 887 is not needed, e.g: 887 public static void onSpinWait() {}; should be 887 public static void onSpinWait() {} Best regards, Andrej Golovnin From aleksey.shipilev at oracle.com Wed Jan 27 13:55:48 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 27 Jan 2016 16:55:48 +0300 Subject: RFR (L) JEP 280: Indify String Concatenation (integration) Message-ID: <56A8CC64.3080500@oracle.com> Hi again, This is a formal pre-integration review thread for JEP 280 ("Indify String Concatenation") integration: http://openjdk.java.net/jeps/280 The JEP is Targeted, the CCC is approved, the code reviews and pre-integration checks are clean. Code changes are happening simultaneously in four components: a) (M) javac changes that emit indy: http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ b) (L) JDK changes with StringConcatFactory and friends, plus fixing the regression tests that do not expect additional indys: http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ c) (XS) Build changes that force emitting the "legacy" inline StringBuilder concat in a few cases (e.g. when pre-JDK 9 bytecode is expected): http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ d) (XS) HotSpot changes that fix a GC regression test that now allocates some metaspace from within the test methods having a string concatenation: http://cr.openjdk.java.net/~shade/8085796/webrev.hs.00/ These changes were already reviewed by multiple people, and so I would like to keep the comments only for serious breaking issues at this point. (Note that this thread cross-posts over several mailing lists: bike-shedding discussion would get multiplied a lot!) Formal acknowledgements from Reviewers would be appreciated. Pending no show-stopper comments, I'd like to push this through jdk9/dev in 24 hours. Thanks, -Aleksey From ivan at azulsystems.com Wed Jan 27 14:10:04 2016 From: ivan at azulsystems.com (Ivan Krylov) Date: Wed, 27 Jan 2016 17:10:04 +0300 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: References: <56A8C6A9.8080705@azulsystems.com> Message-ID: <56A8CFBC.608@azulsystems.com> Indeed, thanks! New webrev http://cr.openjdk.java.net/~ikrylov/8147844.jdk.01/ On 27/01/2016 16:48, Andrej Golovnin wrote: > Hi Ivan, > >> Please review the upcoming API changes: >> http://cr.openjdk.java.net/~ikrylov/8147844.jdk.00/ > I think the semicolon at the end of the line 887 is not needed, e.g: > > 887 public static void onSpinWait() {}; > > should be > > 887 public static void onSpinWait() {} > > Best regards, > Andrej Golovnin From Roger.Riggs at Oracle.com Wed Jan 27 15:13:44 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 27 Jan 2016 10:13:44 -0500 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> Message-ID: <56A8DEA8.7000501@Oracle.com> Hi Stephen, On 1/26/2016 8:57 AM, Stephen Colebourne wrote: > While there is no blocking reason why the concept could not be pulled up to > ChronoLocalDate, the method signatures would differ. > > LocalDate::datesUntil(LocalDate) > LocalDate::datesUntil(LocalDate, Period) > > ChronoLocalDate::datesUntil(ChronoLocalDate) > ChronoLocalDate::datesUntil(ChronoLocalDate, ChronoPeriod) > > Thus, adding the ChronoLocalDate methods later will make two additional > methods available on LocalDate (as they will not override). Since all the calendars are built on the same 24hour days and epochDays, the computations result would be the same regardless of the Chronology. The existing LocalDate.until, compareTo, isBefore, isAfter, isEqual methods already use the ChronoLocalDate argument type to avoid having double the signatures. Modifying the type of the argument to be ChronoLocalDate would not modify the semantics and would make it possible to avoid extra methods in the future. I recommend changing the argument to ChronoLocalDate be consistent with the existing until method to keep the option open for a possible addition to ChronoLocalDate > > As such, I'm not convinced that the streaming methods are needed on > ChronoLocalDate. The "correct" mechanism for looping with a ChronoPeriod > may differ by calendar system (not sure how, but it might). And while > datesUntil(ChronoLocalDate) would be safe on ChronoLocalDate, it would > weaken the method signature for the common case on LocalDate. > > Basically, since the stream can be created manually in the rare cases of > ChronoLocalDate, it seems unnecessary to worry about abstracting up. Yes, there is a work around. Roger > > Stephen > > > > On 23 January 2016 at 21:40, Roger Riggs wrote: > >> Hi, >> >> Looks good. >> >> I would want to verify that it is defined such that if it later needs to be >> abstracted up to ChronoLocalDate and apply to compatible Chronologies >> and respective local dates such as JapaneseDate there is no conflict. >> It should not be an issue since Period implements ChronoPeriod. >> In the respective implementations, the estimation/computation of the >> number of steps >> would need to depend on the Chronology's definition of months. >> >> I can sponsor this enhancement. >> >> Thanks, Roger >> >> >> >> >> On 1/20/16 10:15 AM, Stephen Colebourne wrote: >> >>> I'm happy with the logic and specification of this proposal. I think it >>> will be a useful addition. >>> >>> I'll let the Oracle team chime in to do a further review. >>> >>> thanks >>> Stephen >>> >>> >>> >>> On 16 January 2016 at 13:31, Tagir F. Valeev wrote: >>> >>> Hello! >>>> Thanks for review! Here's the updated patch: >>>> http://cr.openjdk.java.net/~tvaleev/webrev/8146218/r2/ >>>> >>>> SC> The docs say that if the end date is before the start date, the >>>> SC> stream is empty. While I can see just about why LongStream.range() >>>> SC> works that way, I don't think this API should. The minimum is an >>>> SC> exception, but it would be easy to support negative in the >>>> SC> days-only case or the months-only case. The problem is where there >>>> SC> are both months/years and days and those should be exceptions. >>>> >>>> Now datesUntil(endExclusive) throws an IllegalArgumentException if end >>>> date is before start date. >>>> >>>> datesUntil(endExclusive, step) supports negative periods. It throws >>>> IllegalArgumentException if: >>>> - step is zero >>>> - step.toTotalMonths() and step.getDays() both non-zero and have >>>> opposite sign >>>> - step is negative and end date is after start date >>>> - step is positive and end date is before start date >>>> >>>> Otherwise it works correctly: you can use >>>> LocalDate.of(2016, 1, 1) >>>> .datesUntil(LocalDate.of(2015, 1, 1), Period.ofMonths(-1)); >>>> >>>> Steps like Period.of(-1, -1, -1) are also supported. >>>> >>>> SC> The single-arg implementation uses plusDays() with an >>>> SC> incrementing number. When the performance patch goes in, the >>>> SC> proposed streaming implementation will be optimal for small >>>> SC> streams but less optimal for large ones. The fastest way to loop >>>> SC> over a list of dates would be to manually generate them by >>>> SC> incrementing the day until it exceeds the length of month, and so >>>> SC> on. However, this would be serial. >>>> >>>> As I understand, plusDays performance patch is already pushed. >>>> >>>> It's possible to write custom Spliterator which would use >>>> previous.plusDays(1) in tryAdvance() and jump (via >>>> LocalDate.of(startEpochDay+n)) in trySplit() if parallel processing is >>>> requested. However this adds at least one additional class and more >>>> complexity. I guess, such optimization can be considered later as >>>> separate issue when API is stabilized. >>>> >>>> SC> As such, I think the best way to write this, taking account of >>>> SC> how plusDays() is implemented, is as follows: >>>> >>>> SC> LongStream.range(start.toEpochDay(), >>>> SC> end.toEpochDay()).mapToObj(LocalDate::ofEpochDay); >>>> >>>> Ok, now it's done this way. >>>> >>>> SC> In the period-based method, it would be best to capture the case >>>> SC> where totalMonths == 0 and days > 0 and forward to another method >>>> SC> that ignores months. That private method can share implementation >>>> SC> with the public single-arg method (passing in 1). >>>> >>>> This case still more complex than one-day case as it requires division >>>> and multiplication. Thus I'd keep these case separately. However I >>>> simplified "months = 0" path using ofEpochDay, now it should be >>>> faster. >>>> >>>> SC> The docs for the period-based method should specify the strategy >>>> SC> that produces the results (in abstract terms). This needs to cover >>>> SC> that the result is equivalent to always adding to the start period >>>> SC> a multiple of the period. >>>> >>>> I added some clarifications, please check. >>>> >>>> SC> Some nits: >>>> SC> I prefer to avoid @link in the first sentence. Just using 'stream' >>>> would be sufficient. >>>> >>>> Done. >>>> >>>> SC> The first sentence should be a summary. In this case it probably has >>>> a >>>> bit too much detail. >>>> >>>> Done. >>>> >>>> SC> The @return has 'values' on a new line when it could be on the same >>>> line. >>>> >>>> I set now line length = 100 characters in my IDE. Is it ok? >>>> >>>> SC> If statements need braces. >>>> >>>> Done. >>>> >>>> With best regards, >>>> Tagir Valeev. >>>> >>>> >>>> From Roger.Riggs at Oracle.com Wed Jan 27 15:18:33 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 27 Jan 2016 10:18:33 -0500 Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale In-Reply-To: References: <50c6e11a-b026-4e8e-aa7e-e9f7498fc805@default> <575e40e1-9041-424a-8057-f8c289e032ae@default> <56A85B49.4040704@oracle.com> Message-ID: <56A8DFC9.4040307@Oracle.com> +1 On 1/27/2016 5:26 AM, Stephen Colebourne wrote: > From the name of the test and looking at the code, running in English only > is fine. > Stephen > > On 27 January 2016 at 05:53, Masayoshi Okutsu > wrote: > >> Looks OK to me. But I'd like some java.time people to review this change >> to see if the intention of this test is to run only in English. >> >> Thanks, >> Masayoshi >> >> >> On 1/27/2016 1:51 PM, Ramanand Patil wrote: >> >>> Hi all, >>> >>> >>> Please help me in reviewing this test fix. >>> >>> >>> Regards, >>> >>> Ramanand. >>> >>> From: Ramanand Patil >>> Sent: Monday, January 25, 2016 1:05 PM >>> To: i18n-dev at openjdk.java.net >>> Cc: core-libs-dev at openjdk.java.net >>> Subject: RFR: JDK-8147912: test "parseWithZoneWithoutOffset" of >>> java/time/tck/java/time/format/TCKDTFParsedInstant.java fail on de_DE locale >>> >>> >>> Hi all, >>> Please review the trivial test bug fix for: >>> https://bugs.openjdk.java.net/browse/JDK-8147912 >>> Bug Description: Since the test case "parseWithZoneWithoutOffset" is >>> using hard code as input data it will fail on some non-English locales >>> (de_DE locale). >>> Webrev: http://cr.openjdk.java.net/~rpatil/8147912/webrev.00 >>> Fix: Even though hardcoded data is not preferred in compatibility test >>> cases, this case was exception. English is provided as the default locale >>> data for DateTimeFormatter in this testcase. >>> Regards, >>> Ramanand. >>> >> From Alan.Bateman at oracle.com Wed Jan 27 15:20:58 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 27 Jan 2016 15:20:58 +0000 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: <56A8CFBC.608@azulsystems.com> References: <56A8C6A9.8080705@azulsystems.com> <56A8CFBC.608@azulsystems.com> Message-ID: <56A8E05A.7040500@oracle.com> On 27/01/2016 14:10, Ivan Krylov wrote: > Indeed, thanks! > New webrev http://cr.openjdk.java.net/~ikrylov/8147844.jdk.01/ > Can you add @since 9 too? -Alan. From scolebourne at joda.org Wed Jan 27 15:30:01 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 27 Jan 2016 15:30:01 +0000 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <56A8DEA8.7000501@Oracle.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> <56A8DEA8.7000501@Oracle.com> Message-ID: On 27 January 2016 at 15:13, Roger Riggs wrote: > On 1/26/2016 8:57 AM, Stephen Colebourne wrote: >> Thus, adding the ChronoLocalDate methods later will make two additional >> methods available on LocalDate (as they will not override). > > Since all the calendars are built on the same 24hour days and epochDays, the computations > result would be the same regardless of the Chronology. > > The existing LocalDate.until, compareTo, isBefore, isAfter, isEqual methods already use the > ChronoLocalDate argument type to avoid having double the signatures. > > Modifying the type of the argument to be ChronoLocalDate would not modify the semantics > and would make it possible to avoid extra methods in the future. > > I recommend changing the argument to ChronoLocalDate be consistent with the existing > until method to keep the option open for a possible addition to ChronoLocalDate The LocalDate::datesUntil(ChronoLocalDate) method internals would be unaffected as it operates off toEpochDay(). Worth noting that an abstraction on the ChronoLocalDate interface would have to return Stream. A LocalDate::datesUntil(ChronoLocalDate, Period) method would however contain a mixture of Chrono and ISO specific types. Given how the internals of the method depend on access to Period specific concepts abstracting to ChronoPeriod would not be pleasant (if possible) As such, this signature seems unwise. But that gives two types of signature - an abstracted one and a specific one: LocalDate::datesUntil(ChronoLocalDate) LocalDate::datesUntil(LocalDate, Period) Again, it isn't clear that is better. Stephen From paul.sandoz at oracle.com Wed Jan 27 16:31:21 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 27 Jan 2016 17:31:21 +0100 Subject: RFR (L) JEP 280: Indify String Concatenation (integration) In-Reply-To: <56A8CC64.3080500@oracle.com> References: <56A8CC64.3080500@oracle.com> Message-ID: <7E17DDBD-9795-49FA-B83B-DF0F93C13374@oracle.com> I think it quite reasonable to push everything to jdk9/dev. Paul. > On 27 Jan 2016, at 14:55, Aleksey Shipilev wrote: > > Hi again, > > This is a formal pre-integration review thread for JEP 280 ("Indify > String Concatenation") integration: > http://openjdk.java.net/jeps/280 > > The JEP is Targeted, the CCC is approved, the code reviews and > pre-integration checks are clean. > > Code changes are happening simultaneously in four components: > > a) (M) javac changes that emit indy: > http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ > > b) (L) JDK changes with StringConcatFactory and friends, plus fixing > the regression tests that do not expect additional indys: > http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ > > c) (XS) Build changes that force emitting the "legacy" inline > StringBuilder concat in a few cases (e.g. when pre-JDK 9 bytecode is > expected): > http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ > > d) (XS) HotSpot changes that fix a GC regression test that now > allocates some metaspace from within the test methods having a string > concatenation: > http://cr.openjdk.java.net/~shade/8085796/webrev.hs.00/ > > These changes were already reviewed by multiple people, and so I would > like to keep the comments only for serious breaking issues at this > point. (Note that this thread cross-posts over several mailing lists: > bike-shedding discussion would get multiplied a lot!) > > Formal acknowledgements from Reviewers would be appreciated. Pending no > show-stopper comments, I'd like to push this through jdk9/dev in 24 hours. > > Thanks, > -Aleksey > From nadeesh.tv at oracle.com Wed Jan 27 16:34:36 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Wed, 27 Jan 2016 22:04:36 +0530 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: <56A65328.1080207@Oracle.com> References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> Message-ID: <56A8F19C.8090509@oracle.com> Hi all, Thanks for the suggestions. Please see the updated webrev http://cr.openjdk.java.net/~ntv/8141452/webrev.01/ Regards, Nadeesh TV On 1/25/2016 10:24 PM, Roger Riggs wrote: > Hi Stephen, Nadeesh, > > TimeUnit.toChronoUnit is a static method. It seems redundant to have > to pass an instance to a static method of its type. > cu = TimeUnit.toChronoUnit(TimeUnit.SECONDS); > > Instead of: > TimeUnit tu = TimeUnit.SECONDS; > ChronoUnit cu = tu.toChronoUnit(); > > > Minor edits please: > > in @param and @return use the type name when referring to the type. > For example, TimeUnit vs timeUnit (the parameter). > > in @throws, use the parameter name instead of "the unit"; > For example, > + * @throws IllegalArgumentException if timeUnit cannot be converted > > Thanks, Roger > > > On 1/25/2016 11:06 AM, nadeesh tv wrote: >> Hi all, >> >> Please see the updated webrev >> http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >> >> -- >> Thanks and Regards, >> Nadeesh TV >> >> >> On 1/25/2016 9:01 PM, Stephen Colebourne wrote: >>> Typo "TimeUnitequivalent" >>> Otherwise looks good. >>> thanks >>> Stephen >>> >>> >>> >>> On 25 January 2016 at 15:25, nadeesh tv wrote: >>> >>>> Hi all, >>>> >>>> Please review a fix for conversion between Chronounit and Timeunit >>>> >>>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >>>> >>>> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>> >>>> -- >>>> Thanks and Regards, >>>> Nadeesh TV >>>> >>>> >> > -- Thanks and Regards, Nadeesh TV From vladimir.x.ivanov at oracle.com Wed Jan 27 16:47:22 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 27 Jan 2016 19:47:22 +0300 Subject: RFR (L) JEP 280: Indify String Concatenation (integration) In-Reply-To: <56A8CC64.3080500@oracle.com> References: <56A8CC64.3080500@oracle.com> Message-ID: <56A8F49A.4050503@oracle.com> JDK changes looks good! Best regards, Vladimir Ivanov On 1/27/16 4:55 PM, Aleksey Shipilev wrote: > Hi again, > > This is a formal pre-integration review thread for JEP 280 ("Indify > String Concatenation") integration: > http://openjdk.java.net/jeps/280 > > The JEP is Targeted, the CCC is approved, the code reviews and > pre-integration checks are clean. > > Code changes are happening simultaneously in four components: > > a) (M) javac changes that emit indy: > http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ > > b) (L) JDK changes with StringConcatFactory and friends, plus fixing > the regression tests that do not expect additional indys: > http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ > > c) (XS) Build changes that force emitting the "legacy" inline > StringBuilder concat in a few cases (e.g. when pre-JDK 9 bytecode is > expected): > http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ > > d) (XS) HotSpot changes that fix a GC regression test that now > allocates some metaspace from within the test methods having a string > concatenation: > http://cr.openjdk.java.net/~shade/8085796/webrev.hs.00/ > > These changes were already reviewed by multiple people, and so I would > like to keep the comments only for serious breaking issues at this > point. (Note that this thread cross-posts over several mailing lists: > bike-shedding discussion would get multiplied a lot!) > > Formal acknowledgements from Reviewers would be appreciated. Pending no > show-stopper comments, I'd like to push this through jdk9/dev in 24 hours. > > Thanks, > -Aleksey > From Roger.Riggs at Oracle.com Wed Jan 27 16:49:53 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 27 Jan 2016 11:49:53 -0500 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> <56A8DEA8.7000501@Oracle.com> Message-ID: <56A8F531.5000703@Oracle.com> Hi Stephen, Tagir, On 1/27/2016 10:30 AM, Stephen Colebourne wrote: > On 27 January 2016 at 15:13, Roger Riggs wrote: >> On 1/26/2016 8:57 AM, Stephen Colebourne wrote: >>> Thus, adding the ChronoLocalDate methods later will make two additional >>> methods available on LocalDate (as they will not override). >> Since all the calendars are built on the same 24hour days and epochDays, the computations >> result would be the same regardless of the Chronology. >> >> The existing LocalDate.until, compareTo, isBefore, isAfter, isEqual methods already use the >> ChronoLocalDate argument type to avoid having double the signatures. >> >> Modifying the type of the argument to be ChronoLocalDate would not modify the semantics >> and would make it possible to avoid extra methods in the future. >> >> I recommend changing the argument to ChronoLocalDate be consistent with the existing >> until method to keep the option open for a possible addition to ChronoLocalDate > The LocalDate::datesUntil(ChronoLocalDate) method internals would be > unaffected as it operates off toEpochDay(). Worth noting that an > abstraction on the ChronoLocalDate interface would have to return > Stream. Right, Interestingly, none of the tests explicitly depend on the return type of Stream and only use methods that are in ChronoLocalDate. (Based on a quick prototype). But its enough to suggest that there should be some additional test or use of the compile time types. A Stream would be inconvenient and counter intuitive. That's enough of a reason for me to keep the current signatures. Thanks for the comments, Roger > > A LocalDate::datesUntil(ChronoLocalDate, Period) method would however > contain a mixture of Chrono and ISO specific types. Given how the > internals of the method depend on access to Period specific concepts > abstracting to ChronoPeriod would not be pleasant (if possible) As > such, this signature seems unwise. > > But that gives two types of signature - an abstracted one and a specific one: > LocalDate::datesUntil(ChronoLocalDate) > LocalDate::datesUntil(LocalDate, Period) > > Again, it isn't clear that is better. > Stephen From ivan at azulsystems.com Wed Jan 27 17:53:10 2016 From: ivan at azulsystems.com (Ivan Krylov) Date: Wed, 27 Jan 2016 20:53:10 +0300 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: <56A8E05A.7040500@oracle.com> References: <56A8C6A9.8080705@azulsystems.com> <56A8CFBC.608@azulsystems.com> <56A8E05A.7040500@oracle.com> Message-ID: <56A90406.7060107@azulsystems.com> Updated to http://cr.openjdk.java.net/~ikrylov/8147844.jdk.02/ The sample JavaDoc has been updated too: http://ivankrylov.github.io/onspinwait/api/java/lang/Runtime.html#onSpinWait-- Alan, Thank you. On 27/01/2016 18:20, Alan Bateman wrote: > > > On 27/01/2016 14:10, Ivan Krylov wrote: >> Indeed, thanks! >> New webrev http://cr.openjdk.java.net/~ikrylov/8147844.jdk.01/ >> > Can you add @since 9 too? > > -Alan. From rachel.protacio at oracle.com Wed Jan 27 23:02:36 2016 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Wed, 27 Jan 2016 18:02:36 -0500 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 Message-ID: <56A94C8C.9050306@oracle.com> Hello! Small but important change for review: updating the JNI_VERSION and in so doing, changing the format from JNI_VERSION_1_x to JNI_VERSION_x_y (see code/bug for details). Bug: https://bugs.openjdk.java.net/browse/JDK-8145098 hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot/ jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk/ I grep'ed through the code to find references to the current JNI version and believe I have caught all the ones that needed changing, plus the fact that all these tests pass: * local hotspot jtreg tests * my own sample JNI test to print and visually inspect the version (essentially what is performed by the updated hotspot/test/native_sanity/JniVersion.java test) * jck vm tests * local rbt colcated and noncolocated tests, especially for the purpose of hitting tonga/src/nsk/share/jvmti tests where JNI_GetVersion() is used Thank you, Rachel From steve.drach at oracle.com Wed Jan 27 23:37:15 2016 From: steve.drach at oracle.com (Steve Drach) Date: Wed, 27 Jan 2016 15:37:15 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56A6532D.6050805@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> Message-ID: <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> > I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. From steve.drach at oracle.com Thu Jan 28 00:35:58 2016 From: steve.drach at oracle.com (Steve Drach) Date: Wed, 27 Jan 2016 16:35:58 -0800 Subject: RFR: 8147607: Remove test library dependency on sun.security.tools.jarsigner.Main Message-ID: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> Please review a small change to the zipfs test library. Issue: https://bugs.openjdk.java.net/browse/JDK-8147607 Webrev: http://cr.openjdk.java.net/~sdrach/8147607/webrev This change uses the public jdk.security.jarsigner.JarSigner API rather than the API exposed by sun.security.tools.jarsigner. From mikael.vidstedt at oracle.com Thu Jan 28 01:13:57 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 27 Jan 2016 17:13:57 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> Message-ID: <56A96B55.7050301@oracle.com> Just an FYI: I'm working on moving all of this to the Hotspot Copy class and bridging to it via jdk.internal.misc.Unsafe, removing Bits.c altogether. The implementation is working, and the preliminary performance numbers beat the pants off of any of the suggested Bits.c implementations (yay!). I'm currently in the progress of getting some unit tests in place for it all to make sure it covers all the corner cases and then I'll run some real benchmarks to see if it actually lives up to the expectations. Cheers, Mikael On 2016-01-26 11:13, John Rose wrote: > On Jan 26, 2016, at 11:08 AM, Andrew Haley wrote: >> On 01/26/2016 07:04 PM, John Rose wrote: >>> Unsafe.copyMemory bottoms out to Copy::conjoint_memory_atomic. >>> IMO that's a better starting point than memcpy. Perhaps it can be >>> given an additional parameter (or overloading) to specify a swap size. >> OK, but conjoint_memory_atomic doesn't guarantee that destination >> words won't be torn if their source is misaligned: in fact it >> guarantees that they will will be. > That's a good point, and argues for a new function with the > stronger guarantee. Actually, it would be perfectly reasonable > to strengthen the guarantee on the existing function. I don't > think anyone will care about the slight performance change, > especially since it is probably favorable. Since it's Unsafe, > they are not supposed to care, either. > > ? John From amaembo at gmail.com Thu Jan 28 04:28:26 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Thu, 28 Jan 2016 10:28:26 +0600 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <56A8F531.5000703@Oracle.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> <56A8DEA8.7000501@Oracle.com> <56A8F531.5000703@Oracle.com> Message-ID: <1615990576.20160128102826@gmail.com> Hello! It should be noted that there's already a precedent in JDK where method returning stream is subclassed and returns the stream of more concrete objects. I'm speaking about ZipFile and JarFile: public class ZipFile { public Stream stream() {...} } public class JarFile extends ZipFile { @Override public Stream stream() {} } Such generic stream declaration adds some difficulties for ZipFile users. For example, consider this question: http://stackoverflow.com/q/31455188/4856258 So in general I would like to avoid Stream. With best regards, Tagir Valeev. RR> Hi Stephen, Tagir, RR> On 1/27/2016 10:30 AM, Stephen Colebourne wrote: >> On 27 January 2016 at 15:13, Roger Riggs wrote: >>> On 1/26/2016 8:57 AM, Stephen Colebourne wrote: >>>> Thus, adding the ChronoLocalDate methods later will make two additional >>>> methods available on LocalDate (as they will not override). >>> Since all the calendars are built on the same 24hour days and epochDays, the computations >>> result would be the same regardless of the Chronology. >>> >>> The existing LocalDate.until, compareTo, isBefore, isAfter, isEqual methods already use the >>> ChronoLocalDate argument type to avoid having double the signatures. >>> >>> Modifying the type of the argument to be ChronoLocalDate would not modify the semantics >>> and would make it possible to avoid extra methods in the future. >>> >>> I recommend changing the argument to ChronoLocalDate be consistent with the existing >>> until method to keep the option open for a possible addition to ChronoLocalDate >> The LocalDate::datesUntil(ChronoLocalDate) method internals would be >> unaffected as it operates off toEpochDay(). Worth noting that an >> abstraction on the ChronoLocalDate interface would have to return >> Stream. RR> Right, Interestingly, none of the tests explicitly depend on the return RR> type of Stream RR> and only use methods that are in ChronoLocalDate. (Based on a quick RR> prototype). RR> But its enough to suggest that there should be some additional test or RR> use of the compile time types. RR> A Stream would be inconvenient and counter intuitive. RR> That's enough of a reason for me to keep the current signatures. RR> Thanks for the comments, Roger >> >> A LocalDate::datesUntil(ChronoLocalDate, Period) method would however >> contain a mixture of Chrono and ISO specific types. Given how the >> internals of the method depend on access to Period specific concepts >> abstracting to ChronoPeriod would not be pleasant (if possible) As >> such, this signature seems unwise. >> >> But that gives two types of signature - an abstracted one and a specific one: >> LocalDate::datesUntil(ChronoLocalDate) >> LocalDate::datesUntil(LocalDate, Period) >> >> Again, it isn't clear that is better. >> Stephen From roger.riggs at oracle.com Thu Jan 28 05:00:57 2016 From: roger.riggs at oracle.com (Roger Riggs) Date: Thu, 28 Jan 2016 00:00:57 -0500 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <1615990576.20160128102826@gmail.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> <56A8DEA8.7000501@Oracle.com> <56A8F531.5000703@Oracle.com> <1615990576.20160128102826@gmail.com> Message-ID: <56A9A089.6@oracle.com> Hi Tagir, After the discussion, I agree. The only addition I would like to see in the tests to add or modify a test so that the it explicitly requires the Stream at compile time. None of the new tests check that the result type signatures are as expected. Thanks, Roger On 1/27/16 11:28 PM, Tagir F. Valeev wrote: > Hello! > > It should be noted that there's already a precedent in JDK where > method returning stream is subclassed and returns the stream of more > concrete objects. I'm speaking about ZipFile and JarFile: > > public class ZipFile { > public Stream stream() {...} > } > > public class JarFile extends ZipFile { > @Override > public Stream stream() {} > } > > Such generic stream declaration adds some difficulties for ZipFile > users. For example, consider this question: > http://stackoverflow.com/q/31455188/4856258 > So in general I would like to avoid Stream. > > With best regards, > Tagir Valeev. > > RR> Hi Stephen, Tagir, > > RR> On 1/27/2016 10:30 AM, Stephen Colebourne wrote: >>> On 27 January 2016 at 15:13, Roger Riggs wrote: >>>> On 1/26/2016 8:57 AM, Stephen Colebourne wrote: >>>>> Thus, adding the ChronoLocalDate methods later will make two additional >>>>> methods available on LocalDate (as they will not override). >>>> Since all the calendars are built on the same 24hour days and epochDays, the computations >>>> result would be the same regardless of the Chronology. >>>> >>>> The existing LocalDate.until, compareTo, isBefore, isAfter, isEqual methods already use the >>>> ChronoLocalDate argument type to avoid having double the signatures. >>>> >>>> Modifying the type of the argument to be ChronoLocalDate would not modify the semantics >>>> and would make it possible to avoid extra methods in the future. >>>> >>>> I recommend changing the argument to ChronoLocalDate be consistent with the existing >>>> until method to keep the option open for a possible addition to ChronoLocalDate >>> The LocalDate::datesUntil(ChronoLocalDate) method internals would be >>> unaffected as it operates off toEpochDay(). Worth noting that an >>> abstraction on the ChronoLocalDate interface would have to return >>> Stream. > RR> Right, Interestingly, none of the tests explicitly depend on the return > RR> type of Stream > RR> and only use methods that are in ChronoLocalDate. (Based on a quick > RR> prototype). > > RR> But its enough to suggest that there should be some additional test or > RR> use of the compile time types. > > RR> A Stream would be inconvenient and counter intuitive. > RR> That's enough of a reason for me to keep the current signatures. > > RR> Thanks for the comments, Roger > > >>> A LocalDate::datesUntil(ChronoLocalDate, Period) method would however >>> contain a mixture of Chrono and ISO specific types. Given how the >>> internals of the method depend on access to Period specific concepts >>> abstracting to ChronoPeriod would not be pleasant (if possible) As >>> such, this signature seems unwise. >>> >>> But that gives two types of signature - an abstracted one and a specific one: >>> LocalDate::datesUntil(ChronoLocalDate) >>> LocalDate::datesUntil(LocalDate, Period) >>> >>> Again, it isn't clear that is better. >>> Stephen From david.holmes at oracle.com Thu Jan 28 05:41:37 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 28 Jan 2016 15:41:37 +1000 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: <56A8C6A9.8080705@azulsystems.com> References: <56A8C6A9.8080705@azulsystems.com> Message-ID: <56A9AA11.4050706@oracle.com> HI Ivan, On 27/01/2016 11:31 PM, Ivan Krylov wrote: > Hello, > > Earlier there was a discussion on this mail alias about the spin loop > hint proposal [1]. Based on the feedback from that discussion some > changes were incorporated and the JEP has been filed [2]. There seems to > be a consensus on the API side. The JEP is now in a draft state and I > hope this JEP will get targeted for java 9 shortly. The discussion in [1] continued in: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037063.html but ended abruptly. In particular Mark's query as to why this moved from Thread to Runtime was seemingly left unanswered. Thanks, David > Please review the upcoming API changes: > http://cr.openjdk.java.net/~ikrylov/8147844.jdk.00/ > > For the reference, the new generated JavaDoc for j.l.Runtime class and > the new method > http://ivankrylov.github.io/onspinwait/api/java/lang/Runtime.html#onSpinWait-- > > > Thanks, > > Ivan > > 1 - > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-October/thread.html#35613 > > 2 - https://bugs.openjdk.java.net/browse/JDK-8147832. From paul.sandoz at oracle.com Thu Jan 28 08:14:23 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 Jan 2016 09:14:23 +0100 Subject: RFR: 8147607: Remove test library dependency on sun.security.tools.jarsigner.Main In-Reply-To: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> References: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> Message-ID: <5EF91586-4747-45F9-A7AB-F8A82C545848@oracle.com> > On 28 Jan 2016, at 01:35, Steve Drach wrote: > > Please review a small change to the zipfs test library. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8147607 > Webrev: http://cr.openjdk.java.net/~sdrach/8147607/webrev > > This change uses the public jdk.security.jarsigner.JarSigner API rather than the API exposed by sun.security.tools.jarsigner. > I am not terribly familiar with the APIs, but looks ok to me. Paul. From ivan.gerasimov at oracle.com Thu Jan 28 08:44:08 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 28 Jan 2016 11:44:08 +0300 Subject: RFR: 8079136: Accessing a nested sublist leads to StackOverflowError In-Reply-To: <55486EA5.8000009@oracle.com> References: <55486EA5.8000009@oracle.com> Message-ID: <56A9D4D7.5040506@oracle.com> Hello everyone! I'd like to respin the discussion. The previous attempt can be found here: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033159.html BUGURL: https://bugs.openjdk.java.net/browse/JDK-8079136 WEBREV: http://cr.openjdk.java.net/~igerasim/8079136/05/webrev/ Here's the summary of the proposed changes: 1) Sublist of an AbstractList (AbstractList.SubList class) now maintains a link to the root AbstractList, and not only to the immediate parent list. This allows to perform modifications on the chain of sub-lists in a loop instead of using recursion (which, in particular, helps avoid the stack-overflow problem). The sublist is still backed by its parent list, in sense that all the modifications are correctly reflected in the backing list (as well as in all the grand parents the sublist, if any), so the fix does not violate the existing specification. 2) It is proposed to update the spec of AbstractList.subList() in the @implSpec section by removing the words about private fields. With the fix, some of those private fields are removed. 3) A similar change is proposed for the ArrayList.SubList class. 4) Two regression tests are added: NestedSubList.java - demonstrates a stack-overflow problem when dealing with a long chain of sublists, SubList.java - tests basic functionality of sub-lists, which helps us make sure nothing is broken with the proposed change. If people agree that the fix is good, I'll file a CCC request for changing the spec of AbstractList.subList(). Sincerely yours, Ivan From Alan.Bateman at oracle.com Thu Jan 28 09:05:22 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 28 Jan 2016 09:05:22 +0000 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56A94C8C.9050306@oracle.com> References: <56A94C8C.9050306@oracle.com> Message-ID: <56A9D9D2.2030208@oracle.com> On 27/01/2016 23:02, Rachel Protacio wrote: > Hello! > > Small but important change for review: updating the JNI_VERSION and in > so doing, changing the format from JNI_VERSION_1_x to JNI_VERSION_x_y > (see code/bug for details). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8145098 > > hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot/ > jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk/ It would be good if Iris or someone on JEP 223 would comment on this. In JEP 223 then the version number doesn't have trailing zeros and I'm wondering if this should be JNI_VERSION_9 rather than JNI_VERSION_9_0. I brought up the issue of the native interfaces on verona-dev recently [1] and I think Iris was open to having the JEP document this so that there is one document of truth on the version changes. Also just to mention that we have draft updates to the JVM TI spec in the jake forest. As part of that work then we've optimistically rev'ed the JVM TI version so that jvmti.h defines JVMTI_VERSION_9. We should at least be consistent so that both native interfaces are _9 or _9_0. -Alan. [1] http://mail.openjdk.java.net/pipermail/verona-dev/2015-December/000324.html From chris.hegarty at oracle.com Thu Jan 28 09:32:55 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 28 Jan 2016 09:32:55 +0000 Subject: RFR: 8147607: Remove test library dependency on sun.security.tools.jarsigner.Main In-Reply-To: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> References: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> Message-ID: On 28 Jan 2016, at 00:35, Steve Drach wrote: > Please review a small change to the zipfs test library. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8147607 > Webrev: http://cr.openjdk.java.net/~sdrach/8147607/webrev > > This change uses the public jdk.security.jarsigner.JarSigner API rather than the API exposed by sun.security.tools.jarsigner. Thanks for doing this Steve. Your changes look fine. -Chris. From paul.sandoz at oracle.com Thu Jan 28 09:46:54 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 Jan 2016 10:46:54 +0100 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> References: <561ED8BF.40901@oracle.com> <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! ! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> Message-ID: <56BB24AD-ED91-41B4-9A79-CF068E4DDA86@oracle.com> +1 Paul. > On 28 Jan 2016, at 00:37, Steve Drach wrote: > > >> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? > > I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. > From paul.sandoz at oracle.com Thu Jan 28 10:04:56 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 28 Jan 2016 11:04:56 +0100 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <1615990576.20160128102826@gmail.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> <56A8DEA8.7000501@Oracle.com> <56A8F531.5000703@Oracle.com> <1615990576.20160128102826@gmail.com> Message-ID: <3B6500F7-122C-4554-98B2-14377ED8BAE0@oracle.com> As with many rules there are cases where one may break them for other overriding reasons. I think this is one of those cases for ZipFile/JarFile.stream(), since for many usages the signature of the stream will not pop out into an explicit type. I have not looked at the specific case for streams and java.time. Regarding the StackOverflow issue with compiler errors, i believe this is fixed in later releases of JDK 8 (at least from 1.8.0_51) and 9-ea. Paul. > On 28 Jan 2016, at 05:28, Tagir F. Valeev wrote: > > Hello! > > It should be noted that there's already a precedent in JDK where > method returning stream is subclassed and returns the stream of more > concrete objects. I'm speaking about ZipFile and JarFile: > > public class ZipFile { > public Stream stream() {...} > } > > public class JarFile extends ZipFile { > @Override > public Stream stream() {} > } > > Such generic stream declaration adds some difficulties for ZipFile > users. For example, consider this question: > http://stackoverflow.com/q/31455188/4856258 > So in general I would like to avoid Stream. > > With best regards, > Tagir Valeev. > > RR> Hi Stephen, Tagir, > > RR> On 1/27/2016 10:30 AM, Stephen Colebourne wrote: >>> On 27 January 2016 at 15:13, Roger Riggs wrote: >>>> On 1/26/2016 8:57 AM, Stephen Colebourne wrote: >>>>> Thus, adding the ChronoLocalDate methods later will make two additional >>>>> methods available on LocalDate (as they will not override). >>>> Since all the calendars are built on the same 24hour days and epochDays, the computations >>>> result would be the same regardless of the Chronology. >>>> >>>> The existing LocalDate.until, compareTo, isBefore, isAfter, isEqual methods already use the >>>> ChronoLocalDate argument type to avoid having double the signatures. >>>> >>>> Modifying the type of the argument to be ChronoLocalDate would not modify the semantics >>>> and would make it possible to avoid extra methods in the future. >>>> >>>> I recommend changing the argument to ChronoLocalDate be consistent with the existing >>>> until method to keep the option open for a possible addition to ChronoLocalDate >>> The LocalDate::datesUntil(ChronoLocalDate) method internals would be >>> unaffected as it operates off toEpochDay(). Worth noting that an >>> abstraction on the ChronoLocalDate interface would have to return >>> Stream. > RR> Right, Interestingly, none of the tests explicitly depend on the return > RR> type of Stream > RR> and only use methods that are in ChronoLocalDate. (Based on a quick > RR> prototype). > > RR> But its enough to suggest that there should be some additional test or > RR> use of the compile time types. > > RR> A Stream would be inconvenient and counter intuitive. > RR> That's enough of a reason for me to keep the current signatures. > > RR> Thanks for the comments, Roger > > >>> >>> A LocalDate::datesUntil(ChronoLocalDate, Period) method would however >>> contain a mixture of Chrono and ISO specific types. Given how the >>> internals of the method depend on access to Period specific concepts >>> abstracting to ChronoPeriod would not be pleasant (if possible) As >>> such, this signature seems unwise. >>> >>> But that gives two types of signature - an abstracted one and a specific one: >>> LocalDate::datesUntil(ChronoLocalDate) >>> LocalDate::datesUntil(LocalDate, Period) >>> >>> Again, it isn't clear that is better. >>> Stephen > From miroslav.kos at oracle.com Thu Jan 28 10:40:34 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Thu, 28 Jan 2016 11:40:34 +0100 Subject: RFR [9]: 8145104: NPE is thrown when JAXBContextFactory implementation is specified in system property, 8145112: newInstance(String, ClassLoader): java.lang.JAXBException should not be wrapped as expected according to spec In-Reply-To: <56938E7F.1010800@oracle.com> References: <567953BF.5010206@oracle.com> <56938E7F.1010800@oracle.com> Message-ID: <56A9F022.4010706@oracle.com> Hi Lance, this is the RFR Alan was mentioning. Thanks a lot! Miran On 11/01/16 12:14, Miroslav Kos wrote: > Ping ... anybody? > > On 22/12/15 14:44, Miroslav Kos wrote: >> Hi everybody, >> I'd like to ask for reviewing following patch. It fixes two issues: >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8145104 >> NPE is thrown when JAXBContextFactory implementation is specified in >> system property >> >> In short, it's a regression of recent changes in JAXB-AP. The old >> version of API is using static methods to create JAXBContext >> instance, where as the new configuration facility uses defined >> interface. The methods are called via core reflection, so when >> invoking non-static method of spi implemetation, instance of the spi >> must be created first. This fixes NPE which occured in certain >> scenarios. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8145112 >> newInstance(String, ClassLoader): java.lang.JAXBException should not >> be wrapped as expected according to spec >> >> There are still some changes in javadoc being discussed, but anyway >> this double-wrapping exception into JAXBException doesn't make any >> sense. JAXBException thrown from JAXB spi provider is re-thrown as is. >> >> Fix: http://cr.openjdk.java.net/~mkos/8145104/jaxws.01/ >> Tests: http://cr.openjdk.java.net/~mkos/8145104/jdk.02/ >> >> Added 2 new jtreg tests, the tests added also to standalone project. >> >> Thanks >> Miran >> > From uschindler at apache.org Thu Jan 28 11:28:53 2016 From: uschindler at apache.org (Uwe Schindler) Date: Thu, 28 Jan 2016 12:28:53 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <000401d15859$87224a00$9566de00$@apache.org> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> <000401d15859$87224a00$9566de00$@apache.org> Message-ID: <012801d159bf$14096280$3c1c2780$@apache.org> Hi, Could you update the JIRA issue (https://bugs.openjdk.java.net/browse/JDK-8132928) and JEP 260 (http://openjdk.java.net/jeps/260) about the change we discussed here (jdk.internal.ref.Cleaner now implements Runable to support Lucene and similar apps needing to hack into cleaner), mainly in the section "open issues"? Maybe also add a comment to the source at the "@Override public void run()" decl, so it may not get lost by later refactorings (if somebody thinks "WTF!?") The update of above docs would be fine for us to have some reference point to cite when we implement the changed Hack. BTW, we already have a preliminary patch for Lucene (untested): https://issues.apache.org/jira/secure/attachment/12784516/LUCENE-6989.patch Thanks for taking care of this issue. Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: Uwe Schindler [mailto:uschindler at apache.org] > Sent: Tuesday, January 26, 2016 5:49 PM > To: 'Chris Hegarty' ; 'Alan Bateman' > ; 'Roger Riggs' ; > uschindler at apache.org > Cc: 'core-libs-dev' > Subject: RE: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref > > Hi, > > API changes l and security check look good to me. I don't have time to > compile and test a JDK, but I trust you that it works :-) > > Uwe > > ----- > Uwe Schindler > uschindler at apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > > > -----Original Message----- > > From: Chris Hegarty [mailto:chris.hegarty at oracle.com] > > Sent: Tuesday, January 26, 2016 5:28 PM > > To: Alan Bateman ; Roger Riggs > > ; uschindler at apache.org > > Cc: core-libs-dev > > Subject: Re: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref > > > > Latest webrev updated in-place: > > http://cr.openjdk.java.net/~chegar/8148117/ > > > > * to execute the run method requires an appropriate permission > > * reverted any copyright changes ( leave to a bulk update ) > > * updated the test to remove the script > > > > -Chris. > > > > > > On 26 Jan 2016, at 15:23, Alan Bateman > wrote: > > > > > On 26/01/2016 13:55, Chris Hegarty wrote: > > >> It is wonderful to see the various ideas on this thread about the longer > > >> term solution to the prompt releasing of direct buffer native memory. I > > >> do not want to obstruct that ( it is very informative ), but I?d like to warp > > up > > >> the review on the actual moving of Cleaner. To that end, I?ve update the > > >> webrev as per Alan?s comments and suggestion ( to extend Runnable ). > > >> > > >> http://cr.openjdk.java.net/~chegar/8148117/ > > >> > > >> -Chris. > > >> > > >> > > > This looks okay. As a defensive-in-depth then Cleaner::run can do a > > permission check and should ease concerns about leakage. > > From lance.andersen at oracle.com Thu Jan 28 14:43:16 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 28 Jan 2016 09:43:16 -0500 Subject: RFR [9]: 8145104: NPE is thrown when JAXBContextFactory implementation is specified in system property, 8145112: newInstance(String, ClassLoader): java.lang.JAXBException should not be wrapped as expected according to spec In-Reply-To: <56A9F022.4010706@oracle.com> References: <567953BF.5010206@oracle.com> <56938E7F.1010800@oracle.com> <56A9F022.4010706@oracle.com> Message-ID: Hi Miran, Sorry for the delay in the review. Lost it over the holidays? Overall it looks good. I probably would have used TestNG but I understand these are not just openjdk specific tests so all is good. Please remember to bump the copyright date from 2015 to 2016 Best Lance On Jan 28, 2016, at 5:40 AM, Miroslav Kos wrote: > Hi Lance, > this is the RFR Alan was mentioning. > > Thanks a lot! > Miran > > > On 11/01/16 12:14, Miroslav Kos wrote: >> Ping ... anybody? >> >> On 22/12/15 14:44, Miroslav Kos wrote: >>> Hi everybody, >>> I'd like to ask for reviewing following patch. It fixes two issues: >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8145104 >>> NPE is thrown when JAXBContextFactory implementation is specified in system property >>> >>> In short, it's a regression of recent changes in JAXB-AP. The old version of API is using static methods to create JAXBContext instance, where as the new configuration facility uses defined interface. The methods are called via core reflection, so when invoking non-static method of spi implemetation, instance of the spi must be created first. This fixes NPE which occured in certain scenarios. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8145112 >>> newInstance(String, ClassLoader): java.lang.JAXBException should not be wrapped as expected according to spec >>> >>> There are still some changes in javadoc being discussed, but anyway this double-wrapping exception into JAXBException doesn't make any sense. JAXBException thrown from JAXB spi provider is re-thrown as is. >>> >>> Fix: http://cr.openjdk.java.net/~mkos/8145104/jaxws.01/ >>> Tests: http://cr.openjdk.java.net/~mkos/8145104/jdk.02/ >>> >>> Added 2 new jtreg tests, the tests added also to standalone project. >>> >>> Thanks >>> Miran >>> >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From weijun.wang at oracle.com Thu Jan 28 15:16:04 2016 From: weijun.wang at oracle.com (Wang Weijun) Date: Thu, 28 Jan 2016 23:16:04 +0800 Subject: RFR: 8147607: Remove test library dependency on sun.security.tools.jarsigner.Main In-Reply-To: References: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> Message-ID: Shouldn't you also include the FileOuputStream in try-with-resources? --Max > On Jan 28, 2016, at 5:32 PM, Chris Hegarty wrote: > > > On 28 Jan 2016, at 00:35, Steve Drach wrote: > >> Please review a small change to the zipfs test library. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8147607 >> Webrev: http://cr.openjdk.java.net/~sdrach/8147607/webrev >> >> This change uses the public jdk.security.jarsigner.JarSigner API rather than the API exposed by sun.security.tools.jarsigner. > > Thanks for doing this Steve. Your changes look fine. > > -Chris. From chris.hegarty at oracle.com Thu Jan 28 15:17:52 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 28 Jan 2016 15:17:52 +0000 Subject: RFR: 8147607: Remove test library dependency on sun.security.tools.jarsigner.Main In-Reply-To: References: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> Message-ID: <06A1611E-5A53-481F-B39E-E07164C4468B@oracle.com> On 28 Jan 2016, at 15:16, Wang Weijun wrote: > Shouldn't you also include the FileOuputStream in try-with-resources? It is. Do you need to refresh your browers? -Chris. > --Max > >> On Jan 28, 2016, at 5:32 PM, Chris Hegarty wrote: >> >> >> On 28 Jan 2016, at 00:35, Steve Drach wrote: >> >>> Please review a small change to the zipfs test library. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8147607 >>> Webrev: http://cr.openjdk.java.net/~sdrach/8147607/webrev >>> >>> This change uses the public jdk.security.jarsigner.JarSigner API rather than the API exposed by sun.security.tools.jarsigner. >> >> Thanks for doing this Steve. Your changes look fine. >> >> -Chris. > From chris.hegarty at oracle.com Thu Jan 28 15:30:39 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 28 Jan 2016 15:30:39 +0000 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <012801d159bf$14096280$3c1c2780$@apache.org> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <56A3AC1D.50003@oracle.com> <56A78F8E.5020402@oracle.com> <7757C7A2-B908-409C-8AAD-246E0EDD8FDD@oracle.com> <000401d15859$87224a00$9566de00$@apache.org> <012801d159bf$14096280$3c1c2780$@apache.org> Message-ID: Uwe, On 28 Jan 2016, at 11:28, Uwe Schindler wrote: > Hi, > > Could you update the JIRA issue (https://bugs.openjdk.java.net/browse/JDK-8132928) and JEP 260 (http://openjdk.java.net/jeps/260) about the change we discussed here (jdk.internal.ref.Cleaner now implements Runable to support Lucene and similar apps needing to hack into cleaner), mainly in the section "open issues?? You are still in unsupported waters. You are using reflection and setAccessible(true) to gain access to the internals of direct buffer. The refactoring we are doing in Cleaner as part of this issue, has a positive side-effect, from a library maintainers point of view ( less reflective code and static usage of internal types), but this is not to be confused with an agreed upon supported interface. It is just to keep things working, as we refactor / move some internal types in the JDK. Longer term a standard Java SE API should replace this. JEP 260 will, for the present time, continue to list sun.misc.Cleaner as an open issue, until we are satisfied that there is no critical need for it, outside of the direct/mapped buffer usage. > Maybe also add a comment to the source at the "@Override public void run()" decl, so it may not get lost by later refactorings (if somebody thinks "WTF!?") > > The update of above docs would be fine for us to have some reference point to cite when we implement the changed Hack. BTW, we already have a preliminary patch for Lucene (untested): https://issues.apache.org/jira/secure/attachment/12784516/LUCENE-6989.patch Thanks for doing this Uwe. I really appreciate your feedback and prompt replies on this topic. -Chris. > Thanks for taking care of this issue. > Uwe > > ----- > Uwe Schindler > uschindler at apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > >> -----Original Message----- >> From: Uwe Schindler [mailto:uschindler at apache.org] >> Sent: Tuesday, January 26, 2016 5:49 PM >> To: 'Chris Hegarty' ; 'Alan Bateman' >> ; 'Roger Riggs' ; >> uschindler at apache.org >> Cc: 'core-libs-dev' >> Subject: RE: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref >> >> Hi, >> >> API changes l and security check look good to me. I don't have time to >> compile and test a JDK, but I trust you that it works :-) >> >> Uwe >> >> ----- >> Uwe Schindler >> uschindler at apache.org >> ASF Member, Apache Lucene PMC / Committer >> Bremen, Germany >> http://lucene.apache.org/ >> >>> -----Original Message----- >>> From: Chris Hegarty [mailto:chris.hegarty at oracle.com] >>> Sent: Tuesday, January 26, 2016 5:28 PM >>> To: Alan Bateman ; Roger Riggs >>> ; uschindler at apache.org >>> Cc: core-libs-dev >>> Subject: Re: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref >>> >>> Latest webrev updated in-place: >>> http://cr.openjdk.java.net/~chegar/8148117/ >>> >>> * to execute the run method requires an appropriate permission >>> * reverted any copyright changes ( leave to a bulk update ) >>> * updated the test to remove the script >>> >>> -Chris. >>> >>> >>> On 26 Jan 2016, at 15:23, Alan Bateman >> wrote: >>> >>>> On 26/01/2016 13:55, Chris Hegarty wrote: >>>>> It is wonderful to see the various ideas on this thread about the longer >>>>> term solution to the prompt releasing of direct buffer native memory. I >>>>> do not want to obstruct that ( it is very informative ), but I?d like to warp >>> up >>>>> the review on the actual moving of Cleaner. To that end, I?ve update the >>>>> webrev as per Alan?s comments and suggestion ( to extend Runnable ). >>>>> >>>>> http://cr.openjdk.java.net/~chegar/8148117/ >>>>> >>>>> -Chris. >>>>> >>>>> >>>> This looks okay. As a defensive-in-depth then Cleaner::run can do a >>> permission check and should ease concerns about leakage. >>> > > From gil at azul.com Thu Jan 28 16:12:48 2016 From: gil at azul.com (Gil Tene) Date: Thu, 28 Jan 2016 16:12:48 +0000 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: <56A9AA11.4050706@oracle.com> References: <56A8C6A9.8080705@azulsystems.com> <56A9AA11.4050706@oracle.com> Message-ID: > On Jan 27, 2016, at 9:41 PM, David Holmes wrote: > > HI Ivan, > > On 27/01/2016 11:31 PM, Ivan Krylov wrote: >> Hello, >> >> Earlier there was a discussion on this mail alias about the spin loop >> hint proposal [1]. Based on the feedback from that discussion some >> changes were incorporated and the JEP has been filed [2]. There seems to >> be a consensus on the API side. The JEP is now in a draft state and I >> hope this JEP will get targeted for java 9 shortly. > > The discussion in [1] continued in: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037063.html > > but ended abruptly. In particular Mark's query as to why this moved from Thread to Runtime was seemingly left unanswered. The thread continued, but it looks like due to cross-posting with concurrency-interest and people replying on the thread dropping the cores-libs-dev recipient somehow. See continuations of the thread here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 and here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580 Mark's question on why this was moved from Thread to Runtime is discussed in detail there. An easy summary in a single message body can be found here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/014587.html . Also, Doug addressed questions about the brief/bland wording choice ("typically improve performance", which we then switched to "The runtime may may take action to improve the performance" at Brian's suggestion) here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/014578.html HTH. > > Thanks, > David > >> Please review the upcoming API changes: >> http://cr.openjdk.java.net/~ikrylov/8147844.jdk.00/ >> >> For the reference, the new generated JavaDoc for j.l.Runtime class and >> the new method >> http://ivankrylov.github.io/onspinwait/api/java/lang/Runtime.html#onSpinWait-- >> >> >> Thanks, >> >> Ivan >> >> 1 - >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-October/thread.html#35613 >> >> 2 - https://bugs.openjdk.java.net/browse/JDK-8147832. From maurizio.cimadamore at oracle.com Thu Jan 28 16:34:36 2016 From: maurizio.cimadamore at oracle.com (Maurizio Cimadamore) Date: Thu, 28 Jan 2016 16:34:36 +0000 Subject: RFR (L) JEP 280: Indify String Concatenation (integration) In-Reply-To: <56A8CC64.3080500@oracle.com> References: <56A8CC64.3080500@oracle.com> Message-ID: <56AA431C.4010008@oracle.com> Langtools changes look fine to me (as discussed on previous review thread). Thanks! Maurizio On 27/01/16 13:55, Aleksey Shipilev wrote: > Hi again, > > This is a formal pre-integration review thread for JEP 280 ("Indify > String Concatenation") integration: > http://openjdk.java.net/jeps/280 > > The JEP is Targeted, the CCC is approved, the code reviews and > pre-integration checks are clean. > > Code changes are happening simultaneously in four components: > > a) (M) javac changes that emit indy: > http://cr.openjdk.java.net/~shade/8085796/webrev.langtools.05/ > > b) (L) JDK changes with StringConcatFactory and friends, plus fixing > the regression tests that do not expect additional indys: > http://cr.openjdk.java.net/~shade/8085796/webrev.jdk.08/ > > c) (XS) Build changes that force emitting the "legacy" inline > StringBuilder concat in a few cases (e.g. when pre-JDK 9 bytecode is > expected): > http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ > > d) (XS) HotSpot changes that fix a GC regression test that now > allocates some metaspace from within the test methods having a string > concatenation: > http://cr.openjdk.java.net/~shade/8085796/webrev.hs.00/ > > These changes were already reviewed by multiple people, and so I would > like to keep the comments only for serious breaking issues at this > point. (Note that this thread cross-posts over several mailing lists: > bike-shedding discussion would get multiplied a lot!) > > Formal acknowledgements from Reviewers would be appreciated. Pending no > show-stopper comments, I'd like to push this through jdk9/dev in 24 hours. > > Thanks, > -Aleksey > From mark.reinhold at oracle.com Thu Jan 28 16:51:31 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Thu, 28 Jan 2016 08:51:31 -0800 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: References: <56A8C6A9.8080705@azulsystems.com> <56A9AA11.4050706@oracle.com> Message-ID: <20160128085131.63077988eggemoggin.niobe.net> 2016/1/28 8:12 -0800, Gil Tene : > On Jan 27, 2016, at 9:41 PM, David Holmes wrote: >> On 27/01/2016 11:31 PM, Ivan Krylov wrote: >>> Earlier there was a discussion on this mail alias about the spin loop >>> hint proposal [1]. Based on the feedback from that discussion some >>> changes were incorporated and the JEP has been filed [2]. There seems to >>> be a consensus on the API side. The JEP is now in a draft state and I >>> hope this JEP will get targeted for java 9 shortly. >> >> The discussion in [1] continued in: >> >> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037063.html >> >> but ended abruptly. In particular Mark's query as to why this moved >> from Thread to Runtime was seemingly left unanswered. > > The thread continued, but it looks like due to cross-posting with > concurrency-interest and people replying on the thread dropping the > cores-libs-dev recipient somehow. I was wondering what happened to that thread ... > See continuations of the thread > here: > http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 > and here: > http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580 > > Mark's question on why this was moved from Thread to Runtime is > discussed in detail there. An easy summary in a single message body > can be found here: > http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/014587.html > . So that we have a self-contained record for posterity in the OpenJDK mail archive, can someone please summarize the reasoning to this list, core-libs-dev? I also suggested that this single method doesn't really need a JEP. You can do it that way if you really want to, but it will take a bit more time. - Mark From gil at azul.com Thu Jan 28 17:01:32 2016 From: gil at azul.com (Gil Tene) Date: Thu, 28 Jan 2016 17:01:32 +0000 Subject: RFR(XS): 8147844: new method j.l.Runtime.onSpinWait() In-Reply-To: <20160128085131.63077988eggemoggin.niobe.net> References: <56A8C6A9.8080705@azulsystems.com> <56A9AA11.4050706@oracle.com> <20160128085131.63077988eggemoggin.niobe.net> Message-ID: > On Jan 28, 2016, at 8:51 AM, mark.reinhold at oracle.com wrote: > > 2016/1/28 8:12 -0800, Gil Tene : >> On Jan 27, 2016, at 9:41 PM, David Holmes wrote: >>> On 27/01/2016 11:31 PM, Ivan Krylov wrote: >>>> Earlier there was a discussion on this mail alias about the spin loop >>>> hint proposal [1]. Based on the feedback from that discussion some >>>> changes were incorporated and the JEP has been filed [2]. There seems to >>>> be a consensus on the API side. The JEP is now in a draft state and I >>>> hope this JEP will get targeted for java 9 shortly. >>> >>> The discussion in [1] continued in: >>> >>> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037063.html >>> >>> but ended abruptly. In particular Mark's query as to why this moved >>> from Thread to Runtime was seemingly left unanswered. >> >> The thread continued, but it looks like due to cross-posting with >> concurrency-interest and people replying on the thread dropping the >> cores-libs-dev recipient somehow. > > I was wondering what happened to that thread ... > >> See continuations of the thread >> here: >> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 >> and here: >> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580 >> >> Mark's question on why this was moved from Thread to Runtime is >> discussed in detail there. An easy summary in a single message body >> can be found here: >> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/014587.html >> . > > So that we have a self-contained record for posterity in the OpenJDK > mail archive, can someone please summarize the reasoning to this list, > core-libs-dev? Good point. I will follow up in the original thread on core-libs-dev with a summary and pointers to the pother discussion. > > I also suggested that this single method doesn't really need a JEP. > You can do it that way if you really want to, but it will take a bit > more time. One reason we wanted to go through the JEP process is to add to the community-originated and community-lead stats in JEPs. I think it's worth the effort to do that, partly to encourage others to run through the same with larger scope things. > > - Mark From sean.mullan at oracle.com Thu Jan 28 17:25:00 2016 From: sean.mullan at oracle.com (Sean Mullan) Date: Thu, 28 Jan 2016 12:25:00 -0500 Subject: RFR: 8147607: Remove test library dependency on sun.security.tools.jarsigner.Main In-Reply-To: <06A1611E-5A53-481F-B39E-E07164C4468B@oracle.com> References: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> <06A1611E-5A53-481F-B39E-E07164C4468B@oracle.com> Message-ID: <56AA4EEC.3030409@oracle.com> On 01/28/2016 10:17 AM, Chris Hegarty wrote: > On 28 Jan 2016, at 15:16, Wang Weijun wrote: > >> Shouldn't you also include the FileOuputStream in try-with-resources? > > It is. Do you need to refresh your browers? If you read the code quickly, I missed it too, maybe it would be better to put the opening brace on a separate line, i.e.: try (ZipFile in = new ZipFile("multi-release.jar"); FileOutputStream os = new FileOutputStream("signed-multi-release.jar")) { js.sign(in, os); } ... no strong opinion on that though. --Sean > > -Chris. > >> --Max >> >>> On Jan 28, 2016, at 5:32 PM, Chris Hegarty wrote: >>> >>> >>> On 28 Jan 2016, at 00:35, Steve Drach wrote: >>> >>>> Please review a small change to the zipfs test library. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8147607 >>>> Webrev: http://cr.openjdk.java.net/~sdrach/8147607/webrev >>>> >>>> This change uses the public jdk.security.jarsigner.JarSigner API rather than the API exposed by sun.security.tools.jarsigner. >>> >>> Thanks for doing this Steve. Your changes look fine. >>> >>> -Chris. >> > From gil at azul.com Thu Jan 28 17:25:28 2016 From: gil at azul.com (Gil Tene) Date: Thu, 28 Jan 2016 17:25:28 +0000 Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal In-Reply-To: <20151207093440.607909@eggemoggin.niobe.net> References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com> <561428BC.3060806@cs.oswego.edu> <015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com> <56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu> <54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com> <561FC6F9.1080701@cs.oswego.edu> <25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com> <562BC968.70603@cs.oswego.edu> <8917A25C-4A16-4291-B145-FA7180E09F78@azul.com> <4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com> <20151207093440.607909@eggemoggin.niobe.net> Message-ID: This thread seems to have "hopped away" to the concurrency-interest list in mid-Dec-2015. This posting is intended to capture a summary of reasoning and some of the discussion there so that we have it in the record in core-libs-dev. Mostly by including the contents of several posts in the continuations of the original thread. See thread continuations here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 and here: http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580 Summary: On the reasoning for moving from Thread.SpinLoopHint to Runtime.onSpinWait: From Gil Tene, on Tue, Dec 1, 2015 at 3:45 PM > ? > Thread.spinLoopHint() was my first choice as well. But I was swayed by strong arguments against "hint" in the method name. Including the lack of "action" and ambiguity about roles. We looked at various names that were either clear and way too long or way too short or implementation specific (or narrowing), like skip, pause, relax, etc. > > Given the spec we agreed on, the name we were looking for was something that would be equivalent to the obvious expectations from something named as elaborately as: > > maybeYouShouldTryToImproveThePerformanceOfTheSpinWaitLoopMakingThisCall(), with the receiver being either the thread or the runtime. > > The "maybe you should try" part is important because doing nothing is a valid option, and accidentally failing to achieve the goal is probably ok, but consistently working in the opposite direction of the goal would be "surprising behavior". The "...making this call" part is important because of ambiguities around roles and actions (the call is not expected to spin, or wait, it's the caller that is doing those things). > > Given the natural way we'd describe what the options are for the receiver in plain language, it became clear that Runtime fit better: we naturally say "the runtime may..." and "indicate to the runtime...", not "the thread may" or "indicate to the thread...". In addition, some of the implementation possibilities (e.g. switch this thread to spin on a dedicated core) may involve actions that are natural runtime actions but far outside of the scope of what Thread might do. > > With an event delivery paradigm ("I'm in a spin wait loop, you may want to do something about that") Runtime.onSpinWait() fits common naming conventions and roles. It's also readable enough to understand that the Runtime is being told that a spin wait is going on. And in that sense, it is just as expressive as spinLoopHint(), while actually following a naming convention. We left the "try to improve the performance" to the spec/JavaDoc because it was very hard to fit in the name. From Martin Thompson, on Tue, Dec 1, 2015 at 9:13 AM: > ... > The "on" prefix was suggested as the caller is notifying the runtime that it is in a spin-wait loop. This allows the runtime the option of reacting to the event, or not, and allows flexibility in how it chooses to react to this information. The method is not waiting; the method is notifying that the caller is waiting. > > Yes, but we don't have Runtime.onGC() or Runtime.onRunFinalization(), and both of those are documented as "suggesting" the VM perform those actions. spinLoopHint() sounded much better than what's proposed here, and carries the suggestion/hint/optionality that's desired. IMHO, onSpinWait() is the least appealing options proposed thus far. > > System.gc() and Runtime.runFinalizersOnExit(boolean) are clear instructions to the system/runtime to do something in an imperative fashion. The onSpinWait() is a declarative statement about the state of the current thread. I can understand a dislike to a name. We all have personal taste on this but I don't think you are comparing similar semantics. > > Consider the runtime as an object. You send it a message via a method. What exactly is spinLoopHint() telling the Runtime what to do? It is applying an event but not expressing it via any convention. "spinLoopHint()" works for me on a Thread to an extent. We should express intent or declare status for this. Hints don't fit comfortably in a programming model. > > The actual naming does not matter so much as this will only be used by minority of programmers. I'll prepare to be flamed on that :-) However it is desperately needed and anything that makes it slip the date to make Java 9 would be such a let down. Some discussion about naming choices, aiming the API at the target user, and what we expect people might do across platforms: From Gil Tene, on Tue, Dec 1, 2015 at 4:35 PM: > ? > On Dec 1, 2015, at 8:02 AM, Vitaly Davidovich wrote: > >> See my earlier suggestion about keeping target user of this API in mind and the level of this API (i.e. low). > > In this sense (limited target audience that is very technical) Thread.foobar(), Thread.spinLoopHint(), Thread.relax(), and Runtime.onSpinWait() are all equivalent. > > The naming debate is mostly driven by longer term thinking, wanting to avoid unforeseen consequences, and s wish for readability. Which are the things that seem to drive most naming debates and make naming one of those hard things to do in CS. > > Given that it's already cost a month of elapsed time, I'd like to try to push forward with what we ended up with. > >> As for doing nothing in the implementation, OK. But I bet if someone using this API finds out it does nothing on their platform they're going to stop using this API and go back to manual hacks that have at least some chance of achieving desired results. So while I understand the general motivation of not leaking implementation details, I don't agree this API is the appropriate place for that. > > The main driver for this JEP is the fact that in Java today, there are no good manual hack options, and on platforms that do support enhanced-performance spinning, spin wait loops written in Java remain inferior (in performance and reaction time) to ones written in other languages. > > I expect spin-loop writers to naturally use Runtime.onSpinWait() across platforms, accepting that it may do nothing on some (e.g. On platforms where no HW support for better spinning performance exists) and improve spinning performance on others. E.g. I fully expect the various disruptor waitFor() variants will still exist, but half of them will probably end up using Runtime.onSpinWait() for enhanced performance, and would do nothing different (in their Java code) for x86, Aarch64, Aarch32, Power, SPARC, or MIPS. And some discussions on the term "relax": From Gil Tene, on Tue, Dec 1, 2015 at 4:19 PM: > On Dec 1, 2015, at 7:32 AM, Vitaly Davidovich wrote: > >> On why "relax" (or other "this is how it will be achieved" naming options) are not a good fit IMO: >> >> But relax, although a common term for this scenario, is still vague enough as to not indicate exactly how it does this, so I don't see any leakage of implementation details. Moreover, the niche target users of this API are likely going to know exactly what they want at the machine instruction level on the platform(s) they care about. This isn't some widely applicable general purpose API -- it's a low level API, and low level APIs should not be intentionally vague. I'm almost certain that 99% of people using this API will know, either ahead of time or by researching, exactly what happens under the hood. When trying to control things at this level, implementation details start to matter. > > The goal of the call is not to relax the CPU. It is to improve the performance of the spin loop. > > This makes goal the opposite of what "relax" suggests. While the spec rules (and clearly says that performance improvement is the goal), we'd probably agree that names like "chill()", "slowDown()", or "dontBotherWithSpeed()" would be a bad fit. "Relax" fits right in with those... > > The fact that the performance-enhancing goal can actually be technically achieved in some specific CPUs by relaxing their execution in specific ways is surprising in itself. It makes for interesting conversation ("wow, who knew that relaxing the aggressiveness of speculation might actually make the reaction time faster?"), but the fact that it's interesting and surprising is a good indication that it is a bad name choice. > > There are various ways to relax CPUs that are wrong to use for the goal of the call, but would be natural for something called "relax". These include deeper cstates in x86, variations of MWAIT with short timeouts on various CPUs, and even instructions named "pause" on some platforms that consistently hurt performance but help save power. These make it clear that if we used "relax" as a name in this spin wait loop performance context, we actually mean "relaxInA2015x86PauseInstructionSense()"... And what is probably a good summary of the whole back and forth on naming this thing: From Gil Tene, on Tue, Dec 1, 2015 at 4:38 PM: > On Dec 1, 2015, at 8:29 AM, Vitaly Davidovich wrote: > >> I suggested relax because it's a commonly used name for this effect. With no prior art in this area, I'd agree that relax can be misinterpreted. But I'm just as happy with your original spinLoopHint on the Thread class, to be honest. > > I liked it too. But was willing to give up on it given the strong arguments and opinions against it, and in the interest of getting something acceptable done. > >> I just don't like onSpinWait() on the Runtime class. > > But can you live with it? Trust me, it grows on you... ;-). Especially when you end up defending it in an e-mail thread. > On Dec 7, 2015, at 9:34 AM, mark.reinhold at oracle.com wrote: > > 2015/11/30 6:58 -0800, gil at azul.com: >> Update: After some significant back-and-forth between Doug and I on >> naming and JavaDoc'ing, and with Martin (Thompson) stepping in to >> help, we have what we think is a good spec and name selection for this >> thing. We're proposing to add a new static method to the Runtime >> class: >> >> class Runtime { /... >> /** >> * Method signifying that the caller is momentarily unable to >> * progress until the occurrence of one or more actions of one or >> * more other activities. When invoked within each iteration, this >> * method typically improves performance of spin wait loop >> * constructions. >> */ >> public static void onSpinWait() {}; >> } > > I'm glad to see some agreement here, but I'm puzzled by the proposal > to to place this in the Runtime class. What's the reasoning for that? > Wouldn't this more naturally be placed in java.lang.Thread? > > Also, I don't think this single method needs a JEP, but you're welcome > to do it that way if you really want to. > > - Mark From steve.drach at oracle.com Thu Jan 28 17:51:52 2016 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 28 Jan 2016 09:51:52 -0800 Subject: RFR: 8147607: Remove test library dependency on sun.security.tools.jarsigner.Main In-Reply-To: <56AA4EEC.3030409@oracle.com> References: <86173278-4C27-4C19-956E-2D64BE2B3A4E@oracle.com> <06A1611E-5A53-481F-B39E-E07164C4468B@oracle.com> <56AA4EEC.3030409@oracle.com> Message-ID: <1BA984CF-DEE2-41E9-991F-0A0CF7100C26@oracle.com> >>> Shouldn't you also include the FileOuputStream in try-with-resources? >> >> It is. Do you need to refresh your browers? > > If you read the code quickly, I missed it too, maybe it would be better to put the opening brace on a separate line, i.e.: > > try (ZipFile in = new ZipFile("multi-release.jar"); > FileOutputStream os = new FileOutputStream("signed-multi-release.jar")) > { > js.sign(in, os); > } Done. See http://cr.openjdk.java.net/~sdrach/8147607/changeset From vitalyd at gmail.com Thu Jan 28 18:29:25 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 28 Jan 2016 13:29:25 -0500 Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal In-Reply-To: References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com> <561428BC.3060806@cs.oswego.edu> <015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com> <56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu> <54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com> <561FC6F9.1080701@cs.oswego.edu> <25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com> <562BC968.70603@cs.oswego.edu> <8917A25C-4A16-4291-B145-FA7180E09F78@azul.com> <4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com> <20151207093440.607909@eggemoggin.niobe.net> Message-ID: So it still hasn't grown on me Gil! :) In particular, placing the method on Runtime just doesn't make sense to me, even with the "we're notifying the runtime we're spinning" sentiment. If that's the "bar" for putting things on Runtime, then a lot more things could (would) go in there; Thread.yield --> informing runtime I'd like to yield, Thread.sleep --> informing runtime I'd like to sleep, LockSupport:park --> informing runtime I'd like to park, and so on. The things that make sense on runtime are invoking GC and finalization, which are truly runtime services. This here is not a runtime service. IMO. But, I'm more interested in the functionality -- you can place it on ArrayList for all I care at that point :). On Thu, Jan 28, 2016 at 12:25 PM, Gil Tene wrote: > This thread seems to have "hopped away" to the concurrency-interest list > in mid-Dec-2015. This posting is intended to capture a summary of reasoning > and some of the discussion there so that we have it in the record in > core-libs-dev. Mostly by including the contents of several posts in the > continuations of the original thread. > > See thread continuations here: > http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 > and here: > http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580 > > Summary: > > On the reasoning for moving from Thread.SpinLoopHint to Runtime.onSpinWait: > > From Gil Tene, on Tue, Dec 1, 2015 at 3:45 PM > > ? > > Thread.spinLoopHint() was my first choice as well. But I was swayed by > strong arguments against "hint" in the method name. Including the lack of > "action" and ambiguity about roles. We looked at various names that were > either clear and way too long or way too short or implementation specific > (or narrowing), like skip, pause, relax, etc. > > > > Given the spec we agreed on, the name we were looking for was something > that would be equivalent to the obvious expectations from something named > as elaborately as: > > > > > maybeYouShouldTryToImproveThePerformanceOfTheSpinWaitLoopMakingThisCall(), > with the receiver being either the thread or the runtime. > > > > The "maybe you should try" part is important because doing nothing is a > valid option, and accidentally failing to achieve the goal is probably ok, > but consistently working in the opposite direction of the goal would be > "surprising behavior". The "...making this call" part is important because > of ambiguities around roles and actions (the call is not expected to spin, > or wait, it's the caller that is doing those things). > > > > Given the natural way we'd describe what the options are for the > receiver in plain language, it became clear that Runtime fit better: we > naturally say "the runtime may..." and "indicate to the runtime...", not > "the thread may" or "indicate to the thread...". In addition, some of the > implementation possibilities (e.g. switch this thread to spin on a > dedicated core) may involve actions that are natural runtime actions but > far outside of the scope of what Thread might do. > > > > With an event delivery paradigm ("I'm in a spin wait loop, you may want > to do something about that") Runtime.onSpinWait() fits common naming > conventions and roles. It's also readable enough to understand that the > Runtime is being told that a spin wait is going on. And in that sense, it > is just as expressive as spinLoopHint(), while actually following a naming > convention. We left the "try to improve the performance" to the > spec/JavaDoc because it was very hard to fit in the name. > > From Martin Thompson, on Tue, Dec 1, 2015 at 9:13 AM: > > ... > > The "on" prefix was suggested as the caller is notifying the runtime > that it is in a spin-wait loop. This allows the runtime the option of > reacting to the event, or not, and allows flexibility in how it chooses to > react to this information. The method is not waiting; the method is > notifying that the caller is waiting. > > > > Yes, but we don't have Runtime.onGC() or Runtime.onRunFinalization(), > and both of those are documented as "suggesting" the VM perform those > actions. spinLoopHint() sounded much better than what's proposed here, and > carries the suggestion/hint/optionality that's desired. IMHO, onSpinWait() > is the least appealing options proposed thus far. > > > > System.gc() and Runtime.runFinalizersOnExit(boolean) are clear > instructions to the system/runtime to do something in an imperative > fashion. The onSpinWait() is a declarative statement about the state of the > current thread. I can understand a dislike to a name. We all have personal > taste on this but I don't think you are comparing similar semantics. > > > > Consider the runtime as an object. You send it a message via a method. > What exactly is spinLoopHint() telling the Runtime what to do? It is > applying an event but not expressing it via any convention. > "spinLoopHint()" works for me on a Thread to an extent. We should express > intent or declare status for this. Hints don't fit comfortably in a > programming model. > > > > The actual naming does not matter so much as this will only be used by > minority of programmers. I'll prepare to be flamed on that :-) However it > is desperately needed and anything that makes it slip the date to make Java > 9 would be such a let down. > > > > Some discussion about naming choices, aiming the API at the target user, > and what we expect people might do across platforms: > > From Gil Tene, on Tue, Dec 1, 2015 at 4:35 PM: > > ? > > On Dec 1, 2015, at 8:02 AM, Vitaly Davidovich wrote: > > > >> See my earlier suggestion about keeping target user of this API in mind > and the level of this API (i.e. low). > > > > In this sense (limited target audience that is very technical) > Thread.foobar(), Thread.spinLoopHint(), Thread.relax(), and > Runtime.onSpinWait() are all equivalent. > > > > The naming debate is mostly driven by longer term thinking, wanting to > avoid unforeseen consequences, and s wish for readability. Which are the > things that seem to drive most naming debates and make naming one of those > hard things to do in CS. > > > > Given that it's already cost a month of elapsed time, I'd like to try to > push forward with what we ended up with. > > > >> As for doing nothing in the implementation, OK. But I bet if someone > using this API finds out it does nothing on their platform they're going to > stop using this API and go back to manual hacks that have at least some > chance of achieving desired results. So while I understand the general > motivation of not leaking implementation details, I don't agree this API is > the appropriate place for that. > > > > The main driver for this JEP is the fact that in Java today, there are > no good manual hack options, and on platforms that do support > enhanced-performance spinning, spin wait loops written in Java remain > inferior (in performance and reaction time) to ones written in other > languages. > > > > I expect spin-loop writers to naturally use Runtime.onSpinWait() across > platforms, accepting that it may do nothing on some (e.g. On platforms > where no HW support for better spinning performance exists) and improve > spinning performance on others. E.g. I fully expect the various disruptor > waitFor() variants will still exist, but half of them will probably end up > using Runtime.onSpinWait() for enhanced performance, and would do nothing > different (in their Java code) for x86, Aarch64, Aarch32, Power, SPARC, or > MIPS. > > > And some discussions on the term "relax": > > From Gil Tene, on Tue, Dec 1, 2015 at 4:19 PM: > > On Dec 1, 2015, at 7:32 AM, Vitaly Davidovich wrote: > > > >> On why "relax" (or other "this is how it will be achieved" naming > options) are not a good fit IMO: > >> > >> But relax, although a common term for this scenario, is still vague > enough as to not indicate exactly how it does this, so I don't see any > leakage of implementation details. Moreover, the niche target users of > this API are likely going to know exactly what they want at the machine > instruction level on the platform(s) they care about. This isn't some > widely applicable general purpose API -- it's a low level API, and low > level APIs should not be intentionally vague. I'm almost certain that 99% > of people using this API will know, either ahead of time or by researching, > exactly what happens under the hood. When trying to control things at this > level, implementation details start to matter. > > > > The goal of the call is not to relax the CPU. It is to improve the > performance of the spin loop. > > > > This makes goal the opposite of what "relax" suggests. While the spec > rules (and clearly says that performance improvement is the goal), we'd > probably agree that names like "chill()", "slowDown()", or > "dontBotherWithSpeed()" would be a bad fit. "Relax" fits right in with > those... > > > > The fact that the performance-enhancing goal can actually be technically > achieved in some specific CPUs by relaxing their execution in specific ways > is surprising in itself. It makes for interesting conversation ("wow, who > knew that relaxing the aggressiveness of speculation might actually make > the reaction time faster?"), but the fact that it's interesting and > surprising is a good indication that it is a bad name choice. > > > > There are various ways to relax CPUs that are wrong to use for the goal > of the call, but would be natural for something called "relax". These > include deeper cstates in x86, variations of MWAIT with short timeouts on > various CPUs, and even instructions named "pause" on some platforms that > consistently hurt performance but help save power. These make it clear that > if we used "relax" as a name in this spin wait loop performance context, we > actually mean "relaxInA2015x86PauseInstructionSense()"... > > And what is probably a good summary of the whole back and forth on naming > this thing: > > From Gil Tene, on Tue, Dec 1, 2015 at 4:38 PM: > > On Dec 1, 2015, at 8:29 AM, Vitaly Davidovich wrote: > > > >> I suggested relax because it's a commonly used name for this effect. > With no prior art in this area, I'd agree that relax can be > misinterpreted. But I'm just as happy with your original spinLoopHint on > the Thread class, to be honest. > > > > I liked it too. But was willing to give up on it given the strong > arguments and opinions against it, and in the interest of getting something > acceptable done. > > > >> I just don't like onSpinWait() on the Runtime class. > > > > But can you live with it? Trust me, it grows on you... ;-). Especially > when you end up defending it in an e-mail thread. > > > > > > On Dec 7, 2015, at 9:34 AM, mark.reinhold at oracle.com wrote: > > > > 2015/11/30 6:58 -0800, gil at azul.com: > >> Update: After some significant back-and-forth between Doug and I on > >> naming and JavaDoc'ing, and with Martin (Thompson) stepping in to > >> help, we have what we think is a good spec and name selection for this > >> thing. We're proposing to add a new static method to the Runtime > >> class: > >> > >> class Runtime { /... > >> /** > >> * Method signifying that the caller is momentarily unable to > >> * progress until the occurrence of one or more actions of one or > >> * more other activities. When invoked within each iteration, this > >> * method typically improves performance of spin wait loop > >> * constructions. > >> */ > >> public static void onSpinWait() {}; > >> } > > > > I'm glad to see some agreement here, but I'm puzzled by the proposal > > to to place this in the Runtime class. What's the reasoning for that? > > Wouldn't this more naturally be placed in java.lang.Thread? > > > > Also, I don't think this single method needs a JEP, but you're welcome > > to do it that way if you really want to. > > > > - Mark > > From daniel.daugherty at oracle.com Thu Jan 28 18:37:10 2016 From: daniel.daugherty at oracle.com (Daniel D. Daugherty) Date: Thu, 28 Jan 2016 11:37:10 -0700 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56A94C8C.9050306@oracle.com> References: <56A94C8C.9050306@oracle.com> Message-ID: <56AA5FD6.4020400@oracle.com> On 1/27/16 4:02 PM, Rachel Protacio wrote: > Hello! > > Small but important change for review: updating the JNI_VERSION and in > so doing, changing the format from JNI_VERSION_1_x to JNI_VERSION_x_y > (see code/bug for details). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8145098 > > hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot/ src/share/vm/prims/jni.cpp No comments. src/share/vm/prims/jni.h No comments. src/share/vm/runtime/thread.cpp No comments. test/native_sanity/JniVersion.java No comments. > jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk/ src/java.base/share/native/include/jni.h No comments. Meta-comment not specific to this bug. It sure would be nice if finally had "one source of truth" for "jni.h" (and friends). Perhaps in the top-level repo... :-) Thumbs up! On Alan's comments about JNI_VERSION_9_0 versus JNI_VERSION_9, my personal preference is for trailing zero version. There was recently a comment on hotspot-dev at o.j.n and verona-dev at o.j.n about the missing trailing zeros. Subject line is: Version special case '9' http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-January/021525.html Dan > > I grep'ed through the code to find references to the current JNI > version and believe I have caught all the ones that needed changing, > plus the fact that all these tests pass: > > * local hotspot jtreg tests > * my own sample JNI test to print and visually inspect the version > (essentially what is performed by the updated > hotspot/test/native_sanity/JniVersion.java test) > * jck vm tests > * local rbt colcated and noncolocated tests, especially for the > purpose of hitting tonga/src/nsk/share/jvmti tests where > JNI_GetVersion() is used > > Thank you, > Rachel From xueming.shen at oracle.com Thu Jan 28 19:09:02 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 28 Jan 2016 11:09:02 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> References: <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> Message-ID: <56AA674E.8040805@oracle.com> Steve, What's the purpose of having a dedicated "JarFile.runtimeVersioned"? Based on its only usages at ln#356 and #381, it appears, shouldn't getVersion() simply returns Release.valueOf(version)? sherman On 01/27/2016 03:37 PM, Steve Drach wrote: >> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? > I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. > From peter.levart at gmail.com Thu Jan 28 19:14:23 2016 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 28 Jan 2016 20:14:23 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> <56A50588.20000@gmail.com> Message-ID: <56AA688F.2080305@gmail.com> Hi, On 01/25/2016 08:32 PM, Gil Tene wrote: > I assume your goal here is to get the resources released with the next newgen collections (following a close()), rather than wait for an oldgen (if the resource was held by an old object). That's a cool thing. > > With that in mind, you can replace the repeated periodic polling/flipping/allocation and external calling changeGuard() with a simple internal GC-detecor that would call changeGuard() allocate a new guard only once per newgen GC cycle. > > This can take the form of adding a simple GCDetector inside your implementation: > > private class GCDetector { > @Override > protected void finalize() throws Throwable { > GCDetector detector = new GCDetector(); > changeGuard(); > Reference.reachabilityFence(detector); > } > } > > // The reason to use finalize here instead of a phantom ref > // based cleaner is that it would trigger immediately after the cycle, > // rather than potentially take an extra cycle to trigger. > // This can be done with a weakRef based cleaner instead > // (but probably when one is added to the JDK, otherwise you'd > // need your own polling thread and logic here...). Good idea. This will adapt the rate of guard changing to the rate of GC and there's no need for special executor. A phantom-ref cleaner would be equally prompt as finalize(). When there's a finalze() method on the referent, finalize() method is invoked 1st, then the FinalReference to the referent is cleared and only after that the PhantomReference to the same referent can be processed in the next GC cycle. But when there's no final() method on the referent, PhantomReference is processed right away. Also, when a finalize() method creates new finalizable object that has a finalize() method which creates new finalizable object that has a finalize() method ... public class Test { static class GCDetector { @Override protected void finalize() throws Throwable { System.out.println("GC detected: " + this); new GCDetector(); } } public static void main(String[] args) throws Exception { System.runFinalizersOnExit(true); new GCDetector(); } } ...and you set to run finalizers on exit, you get a never-ending loop and VM doesn't exit: ... GC detected: Test$GCDetector at 1088249a GC detected: Test$GCDetector at 7e73ecc GC detected: Test$GCDetector at 4da49a96 GC detected: Test$GCDetector at 667a971f GC detected: Test$GCDetector at 3787d3be ... So a phantom-ref cleaner would work better here. The function of Deallocator and guard-changer can even be merged in the same object, so guard-changing and reference tracking can be performed by the same Cleaner(s) like in: http://cr.openjdk.java.net/~plevart/misc/CloseableMemory/v2/CloseableMemory.java Regards, Peter > You'll need to allocate a single instance of GCDetector during construction of a CloseableMemory, without retaining a reference to it after construction. This will start a finalize-triggering chain per instance, with the chain "ticking" once per newgen cycle. > > If you want to avoid having one of these (coming and going on each GC cycle) per CloseableMemory instance, you can use a common static detector and a registration mechanism (where each registered instance would have it's changeGuard() method call? > > ? Gil. > > >> On Jan 24, 2016, at 9:10 AM, Peter Levart wrote: >> >> Hi, >> >> I had an idea recently on how to expedite the collection of an object. It is simple - just don't let it live long. >> >> Here's a concept prototype: >> >> http://cr.openjdk.java.net/~plevart/misc/CloseableMemory/CloseableMemory.java >> >> The overhead of the check in access methods (getByte()/setByte()) amounts to one volatile read of an oop variable that changes once per say 5 to 10 seconds. That's the period a special guard object is alive. It's reachability is tracked by the GC and extends to the end of each access method (using Reference.reachabilityFence). Every few seconds, the guard object is changed with new fresh one so that the chance of the guard and its tracking Cleaner being promoted to old generation is very low. >> >> Could something like that enable a low-overhead CloseableMappedByteBuffer? >> >> Regards, Peter >> >> On 01/23/2016 09:31 PM, Andrew Haley wrote: >>> On 23/01/16 20:01, Uwe Schindler wrote: >>> >>>> It depends how small! If the speed is still somewhere between Java 8 >>>> ByteBuffer performance and the recent Hotspot improvements in Java >>>> 9, I agree with trying it out. But some volatile memory access on >>>> every access is a no-go. The code around ByteBufferIndexInput in >>>> Lucene is the most performance-critical critical code, because on >>>> every search query or sorting there is all the work happening in >>>> there (millions of iterations with positional ByteBuffer.get* >>>> calls). As ByteBuffers are limited to 2 GiB, we also need lots of >>>> hairy code to work around that limitation! >>> Yes, I see that code. It would be helpful if there were a >>> self-contained but realistic benchmark using that code. That way, >>> some simple experiments would allow changes to be measured. >>> >>>> If you look at ByteBufferIndexInput's code you will see that we >>>> simply do stuff like trying to read from one bytebuffer and only if >>>> we catch an BufferUnderflowException we fall back to handling buffer >>>> switches: Instead of checking bounds on every access, we have >>>> fallback code only happening on exceptions. E.g. if you are 3 bytes >>>> before end of one buffer slice and read a long, it will throw >>>> BufferUnderflow. When this happens the code will fall back to read >>>> byte by byte from 2 different buffers and reassemble the long): >>> I'm surprised you don't see painful deoptimization traps when that >>> happens. I suppose it's rare enough that you don't care. There's a >>> new group of methods in JDK9 called Objects.checkIndex() which are >>> intended to provide a very efficient way to do bounds checks. It might >>> be interesting to see if they work well with ByteBufferIndexInput: >>> that's an important use case. >>> >>> BTW, does anyone here know why we don't have humongous ByteBuffers >>> with a long index? >>> >>> Andrew. From iris.clark at oracle.com Thu Jan 28 19:16:52 2016 From: iris.clark at oracle.com (Iris Clark) Date: Thu, 28 Jan 2016 11:16:52 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <5695A31D.1050405@oracle.com> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <5695A31D.1050405@oracle.com> Message-ID: Hi, Joe. > I suggest changing the class-level javadoc discussing comparisons > to defer to the various compare methods. It would be helpful to > not which compare methods are consistent with equals and which > are not. I've reworked the class and method javadoc for the comparison methods so that comparison details exist in only one place. I believe that this resolves the remaining spec issue to get an initial implementation of jdk.Version into jdk9/dev. After I've enhanced the regular expression as suggested elsewhere in this thread I'll post the updated webrev. Thanks, iris From joel.borggren.franck at gmail.com Thu Jan 28 19:17:34 2016 From: joel.borggren.franck at gmail.com (=?UTF-8?Q?Joel_Borggr=C3=A9n=2DFranck?=) Date: Thu, 28 Jan 2016 20:17:34 +0100 Subject: [PATCH] 8140633: Method.getAnnotatedReturnType() returns empty annotations if return type is an array In-Reply-To: References: Message-ID: Hi, Thanks for the patch, but this isn't a bug. If you look in the Java Virtual Machine Specification 4.7.20.2. The type_path structure [1] you can see that you haven't annotated the array but the component type String. You can expand your test slightly to see this: public class AnnotationTest { @Target({ ElementType.TYPE_PARAMETER, ElementType.TYPE_USE }) @Retention(RetentionPolicy.RUNTIME) public @interface MyAnnotation { String value(); } public interface ToImplement { public @MyAnnotation("sayHello2") String @MyAnnotation("sayHello1") [] sayHello2(); } public static void main(String[] args) throws Exception { for (Method m : ToImplement.class.getMethods()) { AnnotatedType returnType = m.getAnnotatedReturnType(); System.out.println(Arrays.asList(returnType.getAnnotations())); AnnotatedType t2 = ((AnnotatedArrayType)returnType).getAnnotatedGenericComponentType(); System.out.println(Arrays.asList(t2.getAnnotations())); } } } I've closed the bug. [1] https://docs.oracle.com/javase/specs/jvms/se8/html/jvms-4.html#jvms-4.7.20.2 cheers /Joel On Tue, Jan 12, 2016 at 3:11 PM, Sergey Ustimenko wrote: > Hello, > > I've recently was working on the bugfix for > https://bugs.openjdk.java.net/browse/JDK-8140633 and as a result i got that > small fix. Also I've adopted presented in bugtracker test and done jtreg > test. > > Here is patch. This is my first contribution try, so im probably missing > something. > > # HG changeset patch > # User fdesu > # Date 1452597274 -10800 > # Tue Jan 12 14:14:34 2016 +0300 > # Node ID c7a11642c1a9bb38d558e8b237afce223c586d14 > # Parent d09282af3b521f751a4b4d5056134877f4cd1b0e > 8140633: Bugfix for Method.getAnnotatedReturnType() > > diff --git > a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java > b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java > --- > a/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java > +++ > b/src/java.base/share/classes/sun/reflect/annotation/AnnotatedTypeFactory.java > @@ -90,7 +90,7 @@ > > public static LocationInfo nestingForType(Type type, LocationInfo > addTo) { > if (isArray(type)) > - return addTo; > + return addTo.pushArray(); > if (type instanceof Class) { > Class clz = (Class)type; > if (clz.getEnclosingClass() == null) > diff --git > a/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java > b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java > new file mode 100644 > --- /dev/null > +++ b/test/java/lang/reflect/Method/MethodAnnotatedReturnTypeTest.java > @@ -0,0 +1,58 @@ > +/* > + * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights > reserved. > + * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > + * > + * This code is free software; you can redistribute it and/or modify it > + * under the terms of the GNU General Public License version 2 only, as > + * published by the Free Software Foundation. > + * > + * This code is distributed in the hope that it will be useful, but WITHOUT > + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or > + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License > + * version 2 for more details (a copy is included in the LICENSE file that > + * accompanied this code). > + * > + * You should have received a copy of the GNU General Public License > version > + * 2 along with this work; if not, write to the Free Software Foundation, > + * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA. > + * > + * Please contact Oracle, 500 Oracle Parkway, Redwood Shores, CA 94065 USA > + * or visit www.oracle.com if you need additional information or have any > + * questions. > + */ > + > +import java.lang.annotation.*; > +import java.lang.reflect.AnnotatedType; > +import java.lang.reflect.Method; > +import java.util.List; > + > +/* > + * @test > + * @bug 8140633 > + * @summary Ensure that Method.getAnnotatedReturnType() not returns empty > annotations when > + * type is an array > + * @author Sergey Ustimenko > + */ > +public class MethodAnnotatedReturnTypeTest { > + > + @Target({ ElementType.TYPE_PARAMETER, ElementType.TYPE_USE }) > + @Retention(RetentionPolicy.RUNTIME) > + public @interface MyAnnotation { > + String value(); > + } > + > + public interface ToImplement { > + @MyAnnotation("sayHello") String sayHello(); > + @MyAnnotation("sayHello1") List sayHello1(); > + @MyAnnotation("sayHello2") String[] sayHello2(); > + } > + > + public static void main(String[] args) { > + for (Method m : ToImplement.class.getMethods()) { > + AnnotatedType returnType = m.getAnnotatedReturnType(); > + if(returnType.getAnnotations().length == 0) > + throw new > RuntimeException("Method.getAnnotatedReturnType() returned an empty > array!"); > + } > + } > + > +} > \ No newline at end of file From steve.drach at oracle.com Thu Jan 28 19:28:59 2016 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 28 Jan 2016 11:28:59 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AA674E.8040805@oracle.com> References: <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AA! 674E.8040805@oracle.com> Message-ID: <3581220C-8998-444B-825E-AFB171A1A799@oracle.com> > What's the purpose of having a dedicated "JarFile.runtimeVersioned?? Consistency, so getVersion returns the same version as the version set when JarFile was constructed. > Based on > its only usages at ln#356 and #381, it appears, shouldn't getVersion() simply > returns Release.valueOf(version)? That might be confusing. Also, having it consistent helps with testing. > > sherman > > On 01/27/2016 03:37 PM, Steve Drach wrote: >>> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? >> I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. >> > From peter.levart at gmail.com Thu Jan 28 19:42:59 2016 From: peter.levart at gmail.com (Peter Levart) Date: Thu, 28 Jan 2016 20:42:59 +0100 Subject: RFR [9] 8148117: Move sun.misc.Cleaner to jdk.internal.ref In-Reply-To: <56A649E2.6020002@oracle.com> References: <05524CDA-3990-479E-90E0-A0201A0AD554@oracle.com> <003701d15613$7ce20690$76a613b0$@apache.org> <56A3D54F.1070300@redhat.com> <004301d15618$cd340f80$679c2e80$@apache.org> <56A3E320.2070004@redhat.com> <56A50588.20000@gmail.com> <56A623B9.9090305@oracle.com> <56A63EE1.4080100@gmail.com> <56A649E2.6020002@oracle.com> Message-ID: <56AA6F43.3010700@gmail.com> On 01/25/2016 05:14 PM, Alan Bateman wrote: > > > On 25/01/2016 15:27, Peter Levart wrote: >> >> But let me ask something. Doesn't GC processing require (at least in >> some phases) that user threads be stopped in a safepoint? What >> happens when a user thread is blocked by kernel on memory access? >> Such thread can't be stopped in a safepoint. Safepoint can't begin, >> so GC can't proceed and therefore can't promote objects to old >> generation at that time. Am I right? If yes, then time does not pass >> for objects while a user thread is blocked on memory access, so they >> won't get promoted to old gen any time sooner after the user thread >> is unblocked. > With memory mapping then it's very possible that a memory access to > stall waiting for pages to be faulted in. So yes, I assume any > STW/safepoint is going to be stall. On the other hand then threads > doing file or socket I/O on these buffers will be in native (maybe > blocked, maybe not) and so aren't an issue. They will safepoint on > return or if they attempt to re-enter. > > -Alan Ah, yes. I/O with direct buffers. All the places that use sun.nio.ch.DirectBuffer.address() to obtain address of DB and then access the memory directly would also have to obtain current guard from DB and keep it until after the last access to the memory. There are 28 such usages in JDK. Usages with blocking I/O could use a different strategy. They could use reference counting directly (invoke increment/decrement on the Deallocator's guardCount and deallocate when it drops to 0). I/O is usually invoked for bulk transfers, so adding an atomic increment and decrement of an int variable would not present too much overhead for such usages, I think. Regards, Peter From joel.borggren.franck at gmail.com Thu Jan 28 19:50:07 2016 From: joel.borggren.franck at gmail.com (=?UTF-8?Q?Joel_Borggr=C3=A9n=2DFranck?=) Date: Thu, 28 Jan 2016 20:50:07 +0100 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: Message-ID: Hi, Thanks for the update. I think it makes sense to expand testing slightly, testing all three parsing clauses that you fixed, and for all three of them also checking that a "thing" after the broken item is parsed correctly. But, the reason we didn't fix this the last two times we looked at it (that I am aware of) is the compatibility story. In order to fix his you need to figure out two things: - Is this is a spec change, that is, can we get away with throwing an AnnotationFormatError where we would now do? Should we clarify the spec? - Since this is a behaviorally incompatible change, how big is the impact? This is of course a hard question to answer, but if one could do a corpus analysis over a large code base and look for catches of ArrayStoreExceptions when reflecting over annotations, that could be useful. If it turns out that "a lot" of users have adopted to this bug, perhaps it isn't worth fixing? On the other hand I can imagine that this is so uncommon that no one catches either type of error. I can't help you with the internal processes at Oracle anymore, but if you are interested in doing a compatibility analysis I can make sure it reaches the right people. cheers /Joel On Thu, Jan 7, 2016 at 7:59 PM, Liam Miller-Cushon wrote: > Hi Joel - > > On Tue, Jan 5, 2016 at 1:16 PM, Joel Borggr?n-Franck > wrote: >> >> I think you need to do skipMemberValue the correct number of times so >> that the cursor in buf is correct, or? > > > Thanks for the catch. I added your test case, and the exception proxy is now > stored and returned after the entire array is read. > >> >> Also, in MissingArrayElementTest, why not just rethrow t? > > > Done. > > Thanks! From xueming.shen at oracle.com Thu Jan 28 20:05:24 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 28 Jan 2016 12:05:24 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AA674E.8040805@oracle.com> References: <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AA674E.8040805@oracle.com> Message-ID: <56AA7484.1020608@oracle.com> Steve, 78 *

    Class loaders that utilize {@code JarFile} to load classes from the 79 * contents of {@code JarFile} entries should construct the {@code JarFile} 80 * by invoking the {@link JarFile#JarFile(File, boolean, int, Release)} 81 * constructor with the value {@code Release.RUNTIME} assigned to the last 82 * argument. This assures that classes compatible with the major 83 * version of the running JVM are loaded from multi-release jar files. What should be the expected result in scenario that the "multi-release" jar file actually puts all latest version entries in the base and early version entries in the "versions/n" directory? For example, in jdk 10, someone generates a mr-jar file with all classes for jdk 10 at root, but a set of jdk9 implementation/ version entries under "versions/9" (assume we don't have class major version compatible issue, both compiled with early version of javac). Those jdk9 version classes will always get picked, even open with "Relese.VERSION_10"? 59 * "META-INF/versions" directory. The versioned entries are partitioned by the 60 * major version of Java platform releases, starting with release 9. A 61 * versioned entry, with a version {@code n}, {@code 8 < n}, in the 62 * "META-INF/versions/{n}" directory overrides the base entry as well as any 63 * entry with a version number {@code i} where {@code 8 < i < n}. So the "versions/{n} always overrides the "base entry", as actually we don't have any info regarding the "version" of those base entries. Maybe I'm missing something here? -Sherman On 01/28/2016 11:09 AM, Xueming Shen wrote: > Steve, > > What's the purpose of having a dedicated "JarFile.runtimeVersioned"? Based on > its only usages at ln#356 and #381, it appears, shouldn't getVersion() simply > returns Release.valueOf(version)? > > sherman > > On 01/27/2016 03:37 PM, Steve Drach wrote: >>> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? >> I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. >> > From xueming.shen at oracle.com Thu Jan 28 20:26:58 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 28 Jan 2016 12:26:58 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <3581220C-8998-444B-825E-AFB171A1A799@oracle.com> References: <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AA674E.8040805@oracle.com> <3581220C-8998-444B-825E-AFB171A1A799@oracle.com> Message-ID: <56AA7992.8000002@oracle.com> On 01/28/2016 11:28 AM, Steve Drach wrote: >> What's the purpose of having a dedicated "JarFile.runtimeVersioned?? > Consistency, so getVersion returns the same version as the version set when JarFile was constructed. > >> Based on >> its only usages at ln#356 and #381, it appears, shouldn't getVersion() simply >> returns Release.valueOf(version)? > That might be confusing. Also, having it consistent helps with testing. > 144 private final int version; 145 private final boolean runtimeVersioned; ... 355 this.version = MULTI_RELEASE_FORCED ? RUNTIME_VERSION : version.value(); 356 this.runtimeVersioned = version == Release.RUNTIME; ... 381 return runtimeVersioned ? Release.RUNTIME : Release.valueOf(version); What I meant is that if "version" is final and RUNTIME_VERSION is static final, why do we need the middle-man "runtimeVersioned". And it appears the implementation is doing at 381 is always actually to return Release.valueOf(version)? It returns Release.valueOf(version) if "runtimeVersioned" is false. But in case of "runtimeVersioned" is true, it means "version == Release.RUNTIME" at ln#356, so you can also return Release.valueOf(version)? I'm confused, what did I miss? sherman >> sherman >> >> On 01/27/2016 03:37 PM, Steve Drach wrote: >>>> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? >>> I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. >>> From xueming.shen at oracle.com Thu Jan 28 20:29:22 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Thu, 28 Jan 2016 12:29:22 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AA7992.8000002@oracle.com> References: <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AA674E.8040805@oracle.com> <3581220C-8998-444B-825E-AFB171A1A799@oracle.com> <56AA7992.8000002@oracle.com> Message-ID: <56AA7A22.10602@oracle.com> On 01/28/2016 12:26 PM, Xueming Shen wrote: > On 01/28/2016 11:28 AM, Steve Drach wrote: >>> What's the purpose of having a dedicated "JarFile.runtimeVersioned?? >> Consistency, so getVersion returns the same version as the version set when JarFile was constructed. >> >>> Based on >>> its only usages at ln#356 and #381, it appears, shouldn't getVersion() simply >>> returns Release.valueOf(version)? >> That might be confusing. Also, having it consistent helps with testing. >> > > 144 private final int version; > 145 private final boolean runtimeVersioned; > ... > 355 this.version = MULTI_RELEASE_FORCED ? RUNTIME_VERSION : version.value(); > 356 this.runtimeVersioned = version == Release.RUNTIME; > ... > 381 return runtimeVersioned ? Release.RUNTIME : Release.valueOf(version); > > What I meant is that if "version" is final and RUNTIME_VERSION is static final, why > do we need the middle-man "runtimeVersioned". And it appears the implementation > is doing at 381 is always actually to return Release.valueOf(version)? > > It returns Release.valueOf(version) if "runtimeVersioned" is false. But in case of > "runtimeVersioned" is true, it means "version == Release.RUNTIME" at ln#356, > so you can also return Release.valueOf(version)? I'm confused, what did I miss? my bad. the "version" at #356 is not that final one. > > sherman > > > >>> sherman >>> >>> On 01/27/2016 03:37 PM, Steve Drach wrote: >>>>> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? >>>> I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. >>>> > From Roger.Riggs at Oracle.com Thu Jan 28 21:28:27 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 28 Jan 2016 16:28:27 -0500 Subject: RFR:8146218: Producing streams in java.time? In-Reply-To: <56A9A089.6@oracle.com> References: <851620702.20151210223156@gmail.com> <1461285505.20151226222913@gmail.com> <1448206903.20160116193150@gmail.com> <56A3F349.5070500@oracle.com> <56A8DEA8.7000501@Oracle.com> <56A8F531.5000703@Oracle.com> <1615990576.20160128102826@gmail.com> <56A9A089.6@oracle.com> Message-ID: <56AA87FB.5080600@Oracle.com> Hi, I added a test to confirm the return type as Stream. If there are no more comments, will push it tomorrow. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-time-stream-8146218/ Roger On 1/28/2016 12:00 AM, Roger Riggs wrote: > Hi Tagir, > > After the discussion, I agree. > > The only addition I would like to see in the tests to add or modify > a test so that the it explicitly requires the Stream at > compile time. > None of the new tests check that the result type signatures are > as expected. > > Thanks, Roger > > On 1/27/16 11:28 PM, Tagir F. Valeev wrote: >> Hello! >> >> It should be noted that there's already a precedent in JDK where >> method returning stream is subclassed and returns the stream of more >> concrete objects. I'm speaking about ZipFile and JarFile: >> >> public class ZipFile { >> public Stream stream() {...} >> } >> >> public class JarFile extends ZipFile { >> @Override >> public Stream stream() {} >> } >> >> Such generic stream declaration adds some difficulties for ZipFile >> users. For example, consider this question: >> http://stackoverflow.com/q/31455188/4856258 >> So in general I would like to avoid Stream. >> >> With best regards, >> Tagir Valeev. >> >> RR> Hi Stephen, Tagir, >> >> RR> On 1/27/2016 10:30 AM, Stephen Colebourne wrote: >>>> On 27 January 2016 at 15:13, Roger Riggs >>>> wrote: >>>>> On 1/26/2016 8:57 AM, Stephen Colebourne wrote: >>>>>> Thus, adding the ChronoLocalDate methods later will make two >>>>>> additional >>>>>> methods available on LocalDate (as they will not override). >>>>> Since all the calendars are built on the same 24hour days and >>>>> epochDays, the computations >>>>> result would be the same regardless of the Chronology. >>>>> >>>>> The existing LocalDate.until, compareTo, isBefore, isAfter, >>>>> isEqual methods already use the >>>>> ChronoLocalDate argument type to avoid having double the signatures. >>>>> >>>>> Modifying the type of the argument to be ChronoLocalDate would not >>>>> modify the semantics >>>>> and would make it possible to avoid extra methods in the future. >>>>> >>>>> I recommend changing the argument to ChronoLocalDate be consistent >>>>> with the existing >>>>> until method to keep the option open for a possible addition to >>>>> ChronoLocalDate >>>> The LocalDate::datesUntil(ChronoLocalDate) method internals would be >>>> unaffected as it operates off toEpochDay(). Worth noting that an >>>> abstraction on the ChronoLocalDate interface would have to return >>>> Stream. >> RR> Right, Interestingly, none of the tests explicitly depend on the >> return >> RR> type of Stream >> RR> and only use methods that are in ChronoLocalDate. (Based on a quick >> RR> prototype). >> >> RR> But its enough to suggest that there should be some additional >> test or >> RR> use of the compile time types. >> >> RR> A Stream would be inconvenient and counter >> intuitive. >> RR> That's enough of a reason for me to keep the current signatures. >> >> RR> Thanks for the comments, Roger >> >> >>>> A LocalDate::datesUntil(ChronoLocalDate, Period) method would however >>>> contain a mixture of Chrono and ISO specific types. Given how the >>>> internals of the method depend on access to Period specific concepts >>>> abstracting to ChronoPeriod would not be pleasant (if possible) As >>>> such, this signature seems unwise. >>>> >>>> But that gives two types of signature - an abstracted one and a >>>> specific one: >>>> LocalDate::datesUntil(ChronoLocalDate) >>>> LocalDate::datesUntil(LocalDate, Period) >>>> >>>> Again, it isn't clear that is better. >>>> Stephen > From steve.drach at oracle.com Thu Jan 28 21:31:12 2016 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 28 Jan 2016 13:31:12 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AA7484.1020608@oracle.com> References: <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AA674E.8040805@oracle.com> <56AA! 7484.1020608@oracle.com> Message-ID: > 78 *

    Class loaders that utilize {@code JarFile} to load classes from the > 79 * contents of {@code JarFile} entries should construct the {@code JarFile} > 80 * by invoking the {@link JarFile#JarFile(File, boolean, int, Release)} > 81 * constructor with the value {@code Release.RUNTIME} assigned to the last > 82 * argument. This assures that classes compatible with the major > 83 * version of the running JVM are loaded from multi-release jar files. > > What should be the expected result in scenario that the "multi-release" jar > file actually puts all latest version entries in the base and early version entries > in the "versions/n" directory? For example, in jdk 10, someone generates > a mr-jar file with all classes for jdk 10 at root, but a set of jdk9 implementation/ > version entries under "versions/9" (assume we don't have class major version > compatible issue, both compiled with early version of javac). Those jdk9 version > classes will always get picked, even open with "Relese.VERSION_10?? There really is no way to compel a jar file creator to make a valid multi-release jar file, and there is no way to check a multi-release jar file to assure it?s valid. While one might be tempted to use the class file version as an indicator of what release it belongs to, that is not necessarily accurate or meaningful. Likewise, resource files can not be partitioned into release version based on content. While the new jar tool will have the capability to assure versioned classes have the same public api as base entry classes and that identical classes aren?t stored in both the base and versioned sections, it too depends on the input being correctly classified by the jar tool user. Garbage in, garbage out. > > > 59 * "META-INF/versions" directory. The versioned entries are partitioned by the > 60 * major version of Java platform releases, starting with release 9. A > 61 * versioned entry, with a version {@code n}, {@code 8 < n}, in the > 62 * "META-INF/versions/{n}" directory overrides the base entry as well as any > 63 * entry with a version number {@code i} where {@code 8 < i < n}. > > So the "versions/{n} always overrides the "base entry", as actually we don't > have any info regarding the "version" of those base entries. Maybe I'm missing > something here? You are not missing anything. If one constructs a JarFile with a Release other than Release.BASE, a base entry will be overridden by a versioned entry if one is found. The bottom line is that the creator of a multi-release jar file must take responsibility for creating a valid multi-release jar. Jar tool will help point out some inconsistencies but can?t actually determine if an entry has been assigned to the right release version. And, of course, one does not need jar tool to construct a multi-release jar file ? see jdk/test/lib/testlibrary/java/util/jar/CreateMultiReleaseTestJars.java > > -Sherman > > > > > On 01/28/2016 11:09 AM, Xueming Shen wrote: >> Steve, >> >> What's the purpose of having a dedicated "JarFile.runtimeVersioned"? Based on >> its only usages at ln#356 and #381, it appears, shouldn't getVersion() simply >> returns Release.valueOf(version)? >> >> sherman >> >> On 01/27/2016 03:37 PM, Steve Drach wrote: >>>> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? >>> I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. >>> >> > From steve.drach at oracle.com Thu Jan 28 21:53:12 2016 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 28 Jan 2016 13:53:12 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AA7992.8000002@oracle.com> References: <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D! .6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AA674E.8040805@oracle.com> <3581220C-8998-444B-825E-AFB171A1A799@oracle.com> <56AA! 7992.8000002@oracle.com> Message-ID: >>> What's the purpose of having a dedicated "JarFile.runtimeVersioned?? >> Consistency, so getVersion returns the same version as the version set when JarFile was constructed. >> >>> Based on >>> its only usages at ln#356 and #381, it appears, shouldn't getVersion() simply >>> returns Release.valueOf(version)? >> That might be confusing. Also, having it consistent helps with testing. >> > > 144 private final int version; > 145 private final boolean runtimeVersioned; > ... > 355 this.version = MULTI_RELEASE_FORCED ? RUNTIME_VERSION : version.value(); > 356 this.runtimeVersioned = version == Release.RUNTIME; Here ?version? is a local argument and not equal to this.version. It can take on the values Release.BASE, Release.VERSION_9, and Release.RUNTIME. In the future we will see additional values like Release.VERSION_10 and Release.VERSION_11, etc. but this.version can only have values of 8 and 9, and in the future 10, 11, etc. this.version is a private implementation value and not publicly available. > ... > 381 return runtimeVersioned ? Release.RUNTIME : Release.valueOf(version); the private method Release.valueOf(version) can only return Release.BASE and Release.VERSION_9 > > What I meant is that if "version" is final and RUNTIME_VERSION is static final, why > do we need the middle-man "runtimeVersioned?. So that getVersion will return Release.RUNTIME if the JarFile object was constructed with a Release.RUNTIME argument. > And it appears the implementation > is doing at 381 is always actually to return Release.valueOf(version)? Except that it will return Release.RUNTIME if the private boolean runtimeVersioned is true > > It returns Release.valueOf(version) if "runtimeVersioned" is false. But in case of > "runtimeVersioned" is true, it means "version == Release.RUNTIME" at ln#356, Yes, that what it means, keeping in mind the local variable version is not equal to this.version > so you can also return Release.valueOf(version)? You could but then one could construct a JarFile object with Release.RUNTIME as it?s version, but getVersion would return Release.VERSION_9. That?s what I meant by consistency, or in this case inconsistency. > I'm confused, what did I miss? I think you are focussing on a minor implementation detail. Internally versions are small integer values, but externally, publicly, they are values of the Release Enum. > > sherman > > > >>> sherman >>> >>> On 01/27/2016 03:37 PM, Steve Drach wrote: >>>>> I'm still wondering about the phrase "root entry" as it continues to give the impression (to me anyway) that it's a resource in the root directory. I think "root" works in the JEP because it deals with simple resources like A.class and B.class that are in the root directory but it's confusing when there resources with a slash in the name. Add to this is the META-INF/versions/ directories which are roots for the version specific resources. I think part of the confusion is that the first mention of "root entry" is in the second paragraph where it has "overrides the unversioned root entry" without defining what it means. In summary, I'm wondering whether you would be up for change the terminology so that "root entry" isn't in the javadoc? >>>> I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. >>>> > From mark.sheppard at oracle.com Fri Jan 29 01:10:36 2016 From: mark.sheppard at oracle.com (Mark Sheppard) Date: Fri, 29 Jan 2016 01:10:36 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <56A20F09.2090904@oracle.com> References: <568FF6A5.4080700@oracle.com> <569FB2D3.6080607@oracle.com> <569FCF34.8020401@oracle.com> <56A11938.8040804@oracle.com> <56A20F09.2090904@oracle.com> Message-ID: <56AABC0C.6090306@oracle.com> Hi, based on feedback from reviews the current version of this fix can be found at http://cr.openjdk.java.net/~msheppar/8144144/webrev.05/ please oblige and review regards Mark On 22/01/2016 11:14, Se?n Coffey wrote: > > On 21/01/16 17:45, Mark Sheppard wrote: >> Thanks Sean >> >> i can revert the isClosed(), I put them in to make the access consistent > I'd question if the synchronized isClosed() and setClosed(boolean) > helper methods are needed at all. The volatile closed boolean variable > should be enough to manage this state flag. Either way, I don't think > the synchronization around the closed variable is root cause of this > issue and performance issues could be addressed in another change if > necessary (playing safe) > > regards, > Sean. > >> >> regards >> Mark >> >> On 20/01/2016 18:17, Se?n Coffey wrote: >>> Hi Mark, >>> >>> SelectorImpl.java: >>> >>> line 125, could you use a 2 arg method call to dprint. It'll print >>> the stacktrace instead : >>> dprint(".close: selector.close: " + t); --> dprint(".close: >>> selector.close", t); >>> >>> The "while (!isClosed()) " change introduces a new hot lock on >>> closed variable. Hopefully, it won't impact performance too much. >>> >>> Looks good to me otherwise. >>> >>> Regards, >>> Sean. >>> >>> On 20/01/16 16:16, Mark Sheppard wrote: >>>> Hi, >>>> an update has been made to the webrev >>>> >>>> http://cr.openjdk.java.net/~msheppar/8144144/webrev.03/ >>>> >>>> an anomaly was found in the select loop of the SelectorImpl.run() >>>> method >>>> some defensive programming, for selector null references, have been >>>> added, also. >>>> >>>> regards >>>> Mark >>>> >>>> On 08/01/2016 17:49, Mark Sheppard wrote: >>>>> Hi >>>>> please oblige and review the following changes >>>>> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ >>>>> >>>>> which addresses the issue >>>>> https://bugs.openjdk.java.net/browse/JDK-8144144 >>>>> >>>>> the changes ensure that an Acceptor and its associated >>>>> ServerSocket/ServerSocketChannel >>>>> are closed. >>>>> >>>>> regards >>>>> Mark >>>> >>> >> > From amaembo at gmail.com Fri Jan 29 04:16:08 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 29 Jan 2016 10:16:08 +0600 Subject: RFR: 8079136: Accessing a nested sublist leads to StackOverflowError In-Reply-To: <56A9D4D7.5040506@oracle.com> References: <55486EA5.8000009@oracle.com> <56A9D4D7.5040506@oracle.com> Message-ID: <85988769.20160129101608@gmail.com> Hello! AbstractList::subListRangeCheck could be replaced with new Objects::checkFromToIndex for consistency with other methods. Also AbstractList::rangeCheck could be replaced with Obejcts::checkIndex. The same for ArrayList class. Otherwise looks good to me. With best regards, Tagir Valeev. IG> Hello everyone! IG> I'd like to respin the discussion. IG> The previous attempt can be found here: IG> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033159.html IG> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8079136 IG> WEBREV: http://cr.openjdk.java.net/~igerasim/8079136/05/webrev/ IG> Here's the summary of the proposed changes: IG> 1) IG> Sublist of an AbstractList (AbstractList.SubList class) now maintains a IG> link to the root AbstractList, and not only to the immediate parent list. IG> This allows to perform modifications on the chain of sub-lists in a loop IG> instead of using recursion (which, in particular, helps avoid the IG> stack-overflow problem). IG> The sublist is still backed by its parent list, in sense that all the IG> modifications are correctly reflected in the backing list (as well as in IG> all the grand parents the sublist, if any), so the fix does not violate IG> the existing specification. IG> 2) IG> It is proposed to update the spec of AbstractList.subList() in the IG> @implSpec section by removing the words about private fields. IG> With the fix, some of those private fields are removed. IG> 3) IG> A similar change is proposed for the ArrayList.SubList class. IG> 4) IG> Two regression tests are added: IG> NestedSubList.java - demonstrates a stack-overflow problem when dealing IG> with a long chain of sublists, IG> SubList.java - tests basic functionality of sub-lists, which helps us IG> make sure nothing is broken with the proposed change. IG> If people agree that the fix is good, I'll file a CCC request for IG> changing the spec of AbstractList.subList(). IG> Sincerely yours, IG> Ivan From amaembo at gmail.com Fri Jan 29 04:32:24 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 29 Jan 2016 10:32:24 +0600 Subject: ArrayList.subList().spliterator() is not late-binding Message-ID: <27476765.20160129103224@gmail.com> Hello! When reviewing 8079136 I noticed that ArrayList.subList().spliterator() is not late-binding: List list = new ArrayList<>(Arrays.asList(1,2,3,4)).subList(0, 3); Stream s = list.stream(); list.add(5); s.findFirst(); --> Exception in thread "main" java.util.ConcurrentModificationException This works correctly for other list implementation (for example, replacing ArrayList with LinkedList makes this code working). As Collection.spliterator() spec says: > In order to preserve expected laziness behavior for the stream() and > parallelStream() methods, spliterators should either have the > characteristic of IMMUTABLE or CONCURRENT, or be late-binding. If > none of these is practical, the overriding class should describe the > spliterator's documented policy of binding and structural > interference, and should override the stream() and parallelStream() > methods to create streams using a Supplier of the spliterator None of these is true for ArrayList.subList().spliterator(): it's not IMMUTABLE, not CONCURRENT, not late-binding, the binding policy is not documented and stream()/parallelStream() methods are not overridden. Is this an issue? Actually to my opinion late-binding spliterator behavior is a mistake. Nobody cares about it as nobody modifies the source between stream creation and terminal operation call. On the other hand, this requirement adds implementation complexity and I already found the second late-binding problem in existing code (the first one was with Pattern.splitAsStream). Also it's not documented that Stream.concat may bind the late-binding source, but nobody cares anyways. Life would be easier without late-binding spliterators. With best regards, Tagir Valeev. From paul.sandoz at oracle.com Fri Jan 29 09:02:55 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jan 2016 10:02:55 +0100 Subject: ArrayList.subList().spliterator() is not late-binding In-Reply-To: <27476765.20160129103224@gmail.com> References: <27476765.20160129103224@gmail.com> Message-ID: Hi Tagir, Last-binding while adding some complexity to the implementation is an important property, as it is to fail-fast on a best effort basis. When one has mutable collections that can be structurally modified and lazy views over those collections (Streams/Iterators) then those views really need to act on the most up to date structure and not a snapshot (structural or otherwise) when the view was created. Otherwise, it is a potential source of bugs producing incorrect results or exceptions, or another form of complexity for the implementation to maintain integrity under such conditions. We took a short-cut here to reuse the existing top-level spliterator in a non-late-binding manner by specifying the absolute fence. We should fix it. Glancing at the code i believe there is a simple fix. When -1 is passed as the fence to the ArrayList.ArrayListSpliterator then the fence is created lazily and should be the result of offset + list.size e.g. in the constructor: hi = fence = offset + lst.size; The forEach impl also needs updating. ? Sublists are also tricky beasts. See documentation on List.subList: * The semantics of the list returned by this method become undefined if * the backing list (i.e., this list) is structurally modified in * any way other than via the returned list. (Structural modifications are * those that change the size of this list, or otherwise perturb it in such * a fashion that iterations in progress may yield incorrect results.) Paul. > On 29 Jan 2016, at 05:32, Tagir F. Valeev wrote: > > Hello! > > When reviewing 8079136 I noticed that > ArrayList.subList().spliterator() is not late-binding: > > List list = new ArrayList<>(Arrays.asList(1,2,3,4)).subList(0, 3); > Stream s = list.stream(); > list.add(5); > s.findFirst(); > --> Exception in thread "main" java.util.ConcurrentModificationException > > This works correctly for other list implementation (for example, > replacing ArrayList with LinkedList makes this code working). As > Collection.spliterator() spec says: > >> In order to preserve expected laziness behavior for the stream() and >> parallelStream() methods, spliterators should either have the >> characteristic of IMMUTABLE or CONCURRENT, or be late-binding. If >> none of these is practical, the overriding class should describe the >> spliterator's documented policy of binding and structural >> interference, and should override the stream() and parallelStream() >> methods to create streams using a Supplier of the spliterator > > None of these is true for ArrayList.subList().spliterator(): it's not > IMMUTABLE, not CONCURRENT, not late-binding, the binding policy is not > documented and stream()/parallelStream() methods are not overridden. > > Is this an issue? > > Actually to my opinion late-binding spliterator behavior is a mistake. > Nobody cares about it as nobody modifies the source between stream > creation and terminal operation call. On the other hand, this > requirement adds implementation complexity and I already found the > second late-binding problem in existing code (the first one was with > Pattern.splitAsStream). Also it's not documented that Stream.concat > may bind the late-binding source, but nobody cares anyways. Life would > be easier without late-binding spliterators. > > With best regards, > Tagir Valeev. > From huaming.li at oracle.com Fri Jan 29 12:43:03 2016 From: huaming.li at oracle.com (Hamlin Li) Date: Fri, 29 Jan 2016 20:43:03 +0800 Subject: RFR: 8076458: java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout In-Reply-To: <8ED500E5-D39D-468D-A08D-2EE9C09B8029@oracle.com> References: <56A71B6A.9090605@oracle.com> <8ED500E5-D39D-468D-A08D-2EE9C09B8029@oracle.com> Message-ID: <56AB5E57.50906@oracle.com> Hi Paul, Sorry for delayed response, have been occupied by other higher priority task. Thanks for your review, I agree with you that your second approach is better. New webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.01/ Below are times cost for different ops: total :169.996 testOps only :108.988 testIntOps only :23.865 testLongOps only :22.326 testDoubleOps only :16.944 so, I build small data providers for each of them. Thank you -Hamlin On 2016/1/26 21:18, Paul Sandoz wrote: > Hi Hamlin, > > Conservatively I would prefer not to remove data sets if at all possible. It will affect all tests, and leaf tasks for parallel streams should have enough data to crunch on. > > I suspect the problem of the flatMap test is not necessarily due to the source sizes being of 1000 elements but that there are tests that substitute an element whose value is m for n elements from 0..m, which can explode things and generate lots of garbage. > > Have you tried executing those kinds tests when the data size is < 1000? > > My bet is the FlatMapOpTest will run significantly faster and you will not need to split it out. > > There are two ways we could consider doing this: > > 1) Check the size in the test method: > > if (data.size() < 1000) { > exerciseOps(data, s -> s.flatMap(mfLt)); > exerciseOps(data, s -> s.flatMap(integerRangeMapper)); > exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); > } > > 2) Include a new data provider for smaller data sets > > @Test(dataProvider = "StreamTestData", dataProviderClass = StreamTestDataProvider.class) > public void testOps(String name, TestData.OfRef data) { > Collection result = exerciseOps(data, s -> s.flatMap(mfId)); > assertEquals(data.size(), result.size()); > > result = exerciseOps(data, s -> s.flatMap(mfNull)); > assertEquals(0, result.size()); > > result = exerciseOps(data, s-> s.flatMap(e -> Stream.empty())); > assertEquals(0, result.size()); > } > > @Test(dataProvider = "StreamTestData.small", dataProviderClass = StreamTestDataProvider.class) > public void testOpsX(String name, TestData.OfRef data) { > exerciseOps(data, s -> s.flatMap(mfLt)); > exerciseOps(data, s -> s.flatMap(integerRangeMapper)); > exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); > } > > I prefer the latter approach (applied to ref and primitive data sets). It?s more work, but i think the right direction. > > Paul. > >> On 26 Jan 2016, at 08:08, Hamlin Li wrote: >> >> Hi everyone, >> >> Would you please help to review the fix for bug https://bugs.openjdk.java.net/browse/JDK-8076458, java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout. >> webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.00/ >> >> Thank you >> -Hamlin From shilpi.rastogi at oracle.com Fri Jan 29 12:43:24 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Fri, 29 Jan 2016 18:13:24 +0530 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <56A21316.1070007@oracle.com> References: <56A21316.1070007@oracle.com> Message-ID: <56AB5E6C.1010201@oracle.com> Hi All, Please review the updated patch- http://cr.openjdk.java.net/~srastogi/8138578/webrev.01/ I verified Lookup.unreflectSpecial() also throws java.lang.IllegalAccessException so updated the JavaDoc. Thanks, Shilpi > > -------- Original Message -------- > Subject: Re: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc > fails to consider static methods > Date: Fri, 22 Jan 2016 10:30:29 +0100 > From: Paul Sandoz > CC: core-libs-dev at openjdk.java.net > > > > Hi Shilpi, > > Can you also double check Lookup.unreflectSpecial, it?s documentation might require updating too. > > Separating the ?or? statements with a comma would help readability in the JavaDoc (same applies to the the patch for findVirtual). > > Paul. > > > On 22 Jan 2016, at 10:22, shilpi rastogi wrote: > > > > > > Gentle Reminder! > > > > -------- Original Message -------- > > Subject: [9u-dev] 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods > > Date: Wed, 20 Jan 2016 18:12:30 +0530 > > From: shilpi rastogi > > Organization: Oracle Corporation > > To: core-libs-dev at openjdk.java.net > > > > > > > > Hi All, > > > > Please review my fix related to java doc. > > > > MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods- > > > > MethodHandles.Lookup.findSpecial() throws IllegalAccessException if the target method is static. > > > > Bug link-https://bugs.openjdk.java.net/browse/JDK-8138578 > > Webrev link-http://cr.openjdk.java.net/~srastogi/8138578/webrev.00/ > > > > Thnaks, > > Shilpi > > > > > > > From paul.sandoz at oracle.com Fri Jan 29 12:53:22 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jan 2016 13:53:22 +0100 Subject: RFR: 8076458: java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout In-Reply-To: <56AB5E57.50906@oracle.com> References: <56A71B6A.9090605@oracle.com> <8ED500E5-D39D-468D-A08D-2EE9C09B8029@oracle.com> <56AB5E57.50906@oracle.com> Message-ID: <420E39BB-6CB5-41BA-9BF1-5BA2E3EB5357@oracle.com> > On 29 Jan 2016, at 13:43, Hamlin Li wrote: > > Hi Paul, > > Sorry for delayed response, have been occupied by other higher priority task. > Thanks for your review, I agree with you that your second approach is better. > New webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.01/ > The changes to the data providers look ok. Would you mind splitting out the tests between StreamTestData and StreamTestData.small as outlined in 2) below. That way for the non-eplosive stuff we can still crunch on larger data without much of a slow down. > > Below are times cost for different ops: > total :169.996 > testOps only :108.988 > testIntOps only :23.865 > testLongOps only :22.326 > testDoubleOps only :16.944 > so, I build small data providers for each of them. > Ok, and i suspect/hope it drops by at least an order of magnitude with your changes applied. Out of curiosity i wonder what the times would be if using parallel GC rather than G1. Paul. > Thank you > -Hamlin > > On 2016/1/26 21:18, Paul Sandoz wrote: >> Hi Hamlin, >> >> Conservatively I would prefer not to remove data sets if at all possible. It will affect all tests, and leaf tasks for parallel streams should have enough data to crunch on. >> >> I suspect the problem of the flatMap test is not necessarily due to the source sizes being of 1000 elements but that there are tests that substitute an element whose value is m for n elements from 0..m, which can explode things and generate lots of garbage. >> >> Have you tried executing those kinds tests when the data size is < 1000? >> >> My bet is the FlatMapOpTest will run significantly faster and you will not need to split it out. >> >> There are two ways we could consider doing this: >> >> 1) Check the size in the test method: >> >> if (data.size() < 1000) { >> exerciseOps(data, s -> s.flatMap(mfLt)); >> exerciseOps(data, s -> s.flatMap(integerRangeMapper)); >> exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); >> } >> >> 2) Include a new data provider for smaller data sets >> >> @Test(dataProvider = "StreamTestData", dataProviderClass = StreamTestDataProvider.class) >> public void testOps(String name, TestData.OfRef data) { >> Collection result = exerciseOps(data, s -> s.flatMap(mfId)); >> assertEquals(data.size(), result.size()); >> >> result = exerciseOps(data, s -> s.flatMap(mfNull)); >> assertEquals(0, result.size()); >> >> result = exerciseOps(data, s-> s.flatMap(e -> Stream.empty())); >> assertEquals(0, result.size()); >> } >> >> @Test(dataProvider = "StreamTestData.small", dataProviderClass = StreamTestDataProvider.class) >> public void testOpsX(String name, TestData.OfRef data) { >> exerciseOps(data, s -> s.flatMap(mfLt)); >> exerciseOps(data, s -> s.flatMap(integerRangeMapper)); >> exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); >> } >> >> I prefer the latter approach (applied to ref and primitive data sets). It?s more work, but i think the right direction. >> >> Paul. From erik.joelsson at oracle.com Fri Jan 29 13:32:06 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Fri, 29 Jan 2016 14:32:06 +0100 Subject: RFR (L) JEP 280: Indify String Concatenation (integration) In-Reply-To: <56A8CC64.3080500@oracle.com> References: <56A8CC64.3080500@oracle.com> Message-ID: <56AB69D6.7060801@oracle.com> On 2016-01-27 14:55, Aleksey Shipilev wrote: > c) (XS) Build changes that force emitting the "legacy" inline > StringBuilder concat in a few cases (e.g. when pre-JDK 9 bytecode is > expected): > http://cr.openjdk.java.net/~shade/8085796/webrev.root.00/ In what context do we need pre-JDK 9 bytecode for java.base and jdk.compiler? /Erik From Alan.Bateman at oracle.com Fri Jan 29 13:41:12 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 29 Jan 2016 13:41:12 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> References: <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> Message-ID: <56AB6BF8.5010302@oracle.com> On 27/01/2016 23:37, Steve Drach wrote: > >> I'm still wondering about the phrase "root entry" as it continues to >> give the impression (to me anyway) that it's a resource in the root >> directory. I think "root" works in the JEP because it deals with >> simple resources like A.class and B.class that are in the root >> directory but it's confusing when there resources with a slash in the >> name. Add to this is the META-INF/versions/ directories which are >> roots for the version specific resources. I think part of the >> confusion is that the first mention of "root entry" is in the second >> paragraph where it has "overrides the unversioned root entry" without >> defining what it means. In summary, I'm wondering whether you would >> be up for change the terminology so that "root entry" isn't in the >> javadoc? > > I?ve released a new webrev, > http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html > that > addresses the above issue. > Thank you, the javadoc is much clearer and readable now. The first mention of multi-release JARs is in the first paragraph where it has "for processing multi-release jar files". I would be tempted to drop this from the first paragraph because the second paragraph introduces the concept. In the second paragraph is has "The versioned entries are partitioned by the major version of Java platform releases, starting with release 9" and then it goes on to explain how versioned entries are partitioned. This has potential to confuse the reader as it gives an initial impression that the oldest version entry is 9 but then the says 8 < n. I realize the text is trying to say that Java SE 9 is the first release to support this but it could be confused. I would be tempted to just drop the mention of release 9 in this paragraph. This may have come up before but JarFile has two sets of constructors, one takes the file path as a String, the other as a File. I just wondering about introduce a second constructor so that they match. One other thing that I've been wondering about is the stream() and entries() methods. Has there been any discussion about these doing filter? Maybe it is too expensive and best left to the user of the API? Part of the context for asking is modular JARs of course. -Alan From amaembo at gmail.com Fri Jan 29 13:47:30 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 29 Jan 2016 19:47:30 +0600 Subject: RFR-8148250: Stream.limit parallel ordered performance In-Reply-To: <22597C98-6A73-4DA0-BFA5-CD3C0C14C30F@oracle.com> References: <919184407.20160124173651@gmail.com> <1059144622.20160126215143@gmail.com> <22597C98-6A73-4DA0-BFA5-CD3C0C14C30F@oracle.com> Message-ID: <108664369.20160129194730@gmail.com> Hello! Here's webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8148250/r1/ With best regards, Tagir Valeev. >> On 26 Jan 2016, at 16:51, Tagir F. Valeev wrote: >> >> Hello! >> >> Thank you for review! Here's the issue: >> https://bugs.openjdk.java.net/browse/JDK-8148250 >> Will post complete webrev later. >> >> PS> Note that it is still the case that in almost all scenarios this >> PS> is likely to be a bad form of parallel stream. >> >> Well not always it's possible to estimate in advance the size of the >> stream. Consider that we have user-specified filter upstream which >> searches over the big collection (we want to return first 10 search >> results, order is important): >> >> data.parallelStream() >> .filter(element -> userQuery.test(element)) >> .limit(10).collect(toList()); >> >> If user query produces 10-15 results, using parallel stream is very >> reasonable, but if it produces millions of results it should not >> regress very much (at least should not become much slower than >> sequential version which is what we see currently). PS> I have my doubts that the cost of splitting and filtering a small PS> number of elements concurrently will pay off in the majority of PS> scenarios, hence the ?almost all?. PS> It could work in cases where there is lots of data to be filtered PS> and only a few items are reported that are proportionally spread PS> out, or over small data and the filter operation is costly. PS> In any case it?s good to avoid the OOME, i am very glad you found a simple way to resolve that. >> >> PS> I think the comment you refer to still applies but now for larger n, so we should refine it. >> >> Should we replace "regardless of the value of n" with "when >> n*parallelismLevel is sufficiently large?? PS> Yes, when N * P is sufficiently large e.g to pluck a number out of the air > 2^32, say PS> Paul. From paul.sandoz at oracle.com Fri Jan 29 13:56:00 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jan 2016 14:56:00 +0100 Subject: RFR-8148250: Stream.limit parallel ordered performance In-Reply-To: <108664369.20160129194730@gmail.com> References: <919184407.20160124173651@gmail.com> <1059144622.20160126215143@gmail.com> <22597C98-6A73-4DA0-BFA5-CD3C0C14C30F@oracle.com> <108664369.20160129194730@gmail.com> Message-ID: <41A35074-C01D-4ABA-88B8-E9C362839FEA@oracle.com> > On 29 Jan 2016, at 14:47, Tagir F. Valeev wrote: > > Hello! > > Here's webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8148250/r1/ > Thanks, reviewed and in my queue to push. Paul. From amaembo at gmail.com Fri Jan 29 14:20:04 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 29 Jan 2016 20:20:04 +0600 Subject: RFR 8148115: Stream.findFirst for unordered source optimization In-Reply-To: References: <1152739344.20160122121906@gmail.com> <30A68664-D525-41BF-9688-FAB7E1F6944A@oracle.com> <916767911.20160123184332@gmail.com> Message-ID: <1033010705.20160129202004@gmail.com> Here's updated webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8148115/r2/ PS> 293 if (!this.mustFindFirst) { PS> 310 if (this.mustFindFirst) { PS> Small thing, no need for ?this?. Done. PS> You can now use Optional.ifPresentOrElse, which was added in 9: PS> ifPresentOrElse( PS> t -> { ?}, PS> () -> assertFalse(it.hasNext()): Done. I also added printing of the searched value if it's not found: assertTrue(contained, "Not found: "+r); Hopefully additional toString() and string concatenation does not eat much time. Alternatively it could be rewritten as if(!contained) fail("Not found: "+r); PS> For primitive streams, you are correct there are no unordered PS> providers of data. in certain cases unordered is induced e.g. forEach testing. PS> A focused fix would be to exercise with say PS> s.filter().unordered(), a more general fix would be as you say to PS> add another provider for int, long and double (backed by say PS> HashSet, or more simply/efficiently using unordered()). PS> We have to carefully review that because that may increase test PS> times. We don?t have to apply to all data sets, plus we could also PS> refine the ref data sets too in that regard, thus keep the overall actions similar or even smaller. PS> Shall we separate that in a new issue? Yes, I guess it would be better to log a new issue. Though I don't know how to formulate it correctly. "Add tests for unordered primitive sources for Stream operations"? "Refine test data sets for Stream API"? Probably it would be better if you open an issue by yourself :-) With best regards, Tagir Valeev. PS> ? PS> I will take this and other patches (close/limit etc) you propose in bulk. PS> Thanks, PS> Paul. >> On 23 Jan 2016, at 13:43, Tagir F. Valeev wrote: >> >> Hello, Paul! >> >> Thank you for review. I implemented it according to your suggestions, >> here's issue: >> https://bugs.openjdk.java.net/browse/JDK-8148115 >> And webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8148115/r1/ >> >> This patch required some changes in existing tests as FindFirstOpTest >> asserted that findFirst() result must be equal to .iterator().next() >> even if the source is HashSet. I extracted the relevant part of >> FindAnyOpTest into assertContains(Optional, Iterator) method and >> placed it into LambdaTestHelpers. >> >> The corresponding changes in primitive tests are unnecessary, because >> it seems that primitive data providers never produce unordered >> streams. I can try to add unordered primitive sources and correct >> these tests as well if it's necessary. >> >> With best regards, >> Tagir Valeev. >> >> >> PS> Hi Tagir, >> >> PS> Thanks for looking at this. >> >> PS> I would prefer if a boolean was passed into the task constructor e.g.: >> >> PS> @Override >> PS> public O evaluateParallel(PipelineHelper helper, >> PS> Spliterator spliterator) { >> PS> // This takes into account the upstream ops flags and the terminal >> PS> // op flags and therefore takes into account findFirst or findAny >> PS> boolean mustFindFirst = >> PS> StreamOpFlag.ORDERED.isKnown(helper.getStreamAndOpFlags()); >> PS> return new FindTask<>(this, mustFindFirst, helper, spliterator).invoke(); >> PS> } >> >> PS> so we keep the flag evaluation logic within the evaluate method. >> >> PS> Then change FindOp.mustFindFirst to be an int of the opFlags calculated in the constructor: >> >> PS> this.opFlags = StreamOpFlag.IS_SHORT_CIRCUIT | (mustFindFirst ? 0 : StreamOpFlag.NOT_ORDERED); >> >> PS> Paul. >> >>>> On 22 Jan 2016, at 07:19, Tagir F. Valeev wrote: >>>> >>>> Hello! >>>> >>>> Seems that currently Stream.findFirst is not optimized for unordered >>>> source. I think it should work as findAny in this case. Here's a small >>>> patch which fixes this: >>>> >>>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/find_patch.txt >>>> >>>> Simple JMH test: >>>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/FindTest.java >>>> >>>> Original: >>>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_orig.txt >>>> # JMH 1.11.2 (released 85 days ago) >>>> # VM version: JDK 9-ea, VM 9-ea+99-2015-12-23-183325.javare.4146.nc >>>> Benchmark Mode Cnt Score Error Units >>>> FindTest.findAny avgt 30 12439,631 ? 1787,866 us/op >>>> FindTest.findAnyUnordered avgt 30 12923,080 ? 1072,537 us/op >>>> FindTest.findFirst avgt 30 48047,467 ? 2713,489 us/op >>>> FindTest.findFirstUnordered avgt 30 52648,893 ? 3934,682 us/op >>>> >>>> Patched: >>>> http://cr.openjdk.java.net/~tvaleev/patches/findFirst/jmh_out_patched.txt >>>> Benchmark Mode Cnt Score Error Units >>>> FindTest.findAny avgt 30 11312,238 ? 386,627 us/op >>>> FindTest.findAnyUnordered avgt 30 12136,953 ? 1536,817 us/op >>>> FindTest.findFirst avgt 30 47517,776 ? 2844,607 us/op >>>> FindTest.findFirstUnordered avgt 30 13147,492 ? 1140,592 us/op >>>> >>>> If you think it's a reasonable thing to patch, I can log an issue, >>>> generate webrev and check whether jtreg tests pass. >>>> >>>> With best regards, >>>> Tagir Valeev. >>>> >> From amaembo at gmail.com Fri Jan 29 14:37:59 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Fri, 29 Jan 2016 20:37:59 +0600 Subject: Stream.foldLeft, one more time (8133680) Message-ID: <1957867051.20160129203759@gmail.com> Hello! What do you think, is it still reasonable to add foldLeft operation to JDK-9 Stream API (foldLeft only, no foldRight)? I propose the following signatures: interface Stream { public U foldLeft(U seed, BiFunction accumulator); public Optional foldLeft(BinaryOperator accumulator); } interface IntStream { public int foldLeft(int seed, IntBinaryOperator accumulator); public OptionalInt foldLeft(IntBinaryOperator accumulator); } interface LongStream { public OptionalLong foldLeft(LongBinaryOperator accumulator); public long foldLeft(long seed, LongBinaryOperator accumulator); } interface DoubleStream { public OptionalDouble foldLeft(DoubleBinaryOperator accumulator); public double foldLeft(double seed, DoubleBinaryOperator accumulator); } They are similar to the corresponding reduce operations, but it's not required for accumulator to be associative and it's not required for seed to have identity properties. It should be clearly stated in JavaDoc that for associative accumulator reduce should be used instead. All methods can be easily implemented as default methods in the interfaces. For example, DoubleStream::foldLeft: public double foldLeft(double seed, DoubleBinaryOperator accumulator) { double[] box = new double[] { seed }; forEachOrdered(t -> box[0] = accumulator.applyAsDouble(box[0], t)); return box[0]; } I can take this issue (if nobody works on it already), but in order not to do the useless work I want to be sure that there are no strong objections against such feature. With best regards, Tagir Valeev. From Roger.Riggs at Oracle.com Fri Jan 29 14:52:19 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Jan 2016 09:52:19 -0500 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: <56A8F19C.8090509@oracle.com> References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> Message-ID: <56AB7CA3.7050801@Oracle.com> Hi Nadeesh, Looks fine, Thanks, Roger On 1/27/2016 11:34 AM, nadeesh tv wrote: > Hi all, > > Thanks for the suggestions. > Please see the updated webrev > http://cr.openjdk.java.net/~ntv/8141452/webrev.01/ > > Regards, > Nadeesh TV > > On 1/25/2016 10:24 PM, Roger Riggs wrote: >> Hi Stephen, Nadeesh, >> >> TimeUnit.toChronoUnit is a static method. It seems redundant to have >> to pass an instance to a static method of its type. >> cu = TimeUnit.toChronoUnit(TimeUnit.SECONDS); >> >> Instead of: >> TimeUnit tu = TimeUnit.SECONDS; >> ChronoUnit cu = tu.toChronoUnit(); >> >> >> Minor edits please: >> >> in @param and @return use the type name when referring to the type. >> For example, TimeUnit vs timeUnit (the parameter). >> >> in @throws, use the parameter name instead of "the unit"; >> For example, >> + * @throws IllegalArgumentException if timeUnit cannot be converted >> Thanks, Roger >> >> On 1/25/2016 11:06 AM, nadeesh tv wrote: >>> Hi all, >>> >>> Please see the updated webrev >>> http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> >>> >>> On 1/25/2016 9:01 PM, Stephen Colebourne wrote: >>>> Typo "TimeUnitequivalent" >>>> Otherwise looks good. >>>> thanks >>>> Stephen >>>> >>>> >>>> >>>> On 25 January 2016 at 15:25, nadeesh tv wrote: >>>> >>>>> Hi all, >>>>> >>>>> Please review a fix for conversion between Chronounit and Timeunit >>>>> >>>>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >>>>> >>>>> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>> >>>>> -- >>>>> Thanks and Regards, >>>>> Nadeesh TV >>>>> >>>>> >>> >> > > -- > Thanks and Regards, > Nadeesh TV > From chris.hegarty at oracle.com Fri Jan 29 15:12:12 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 29 Jan 2016 15:12:12 +0000 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: <56AB7CA3.7050801@Oracle.com> References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> Message-ID: <56AB814C.7070409@oracle.com> On 29/01/16 14:52, Roger Riggs wrote: > Hi Nadeesh, > > Looks fine, > > Thanks, Roger > > > On 1/27/2016 11:34 AM, nadeesh tv wrote: >> Hi all, >> >> Thanks for the suggestions. >> Please see the updated webrev >> http://cr.openjdk.java.net/~ntv/8141452/webrev.01/ +1 This looks fine. Martin, Doug, I assume you are ok to accept this small change in java.util.concurrent.TimeUnit. -Chris. >> Regards, >> Nadeesh TV >> >> On 1/25/2016 10:24 PM, Roger Riggs wrote: >>> Hi Stephen, Nadeesh, >>> >>> TimeUnit.toChronoUnit is a static method. It seems redundant to have >>> to pass an instance to a static method of its type. >>> cu = TimeUnit.toChronoUnit(TimeUnit.SECONDS); >>> >>> Instead of: >>> TimeUnit tu = TimeUnit.SECONDS; >>> ChronoUnit cu = tu.toChronoUnit(); >>> >>> >>> Minor edits please: >>> >>> in @param and @return use the type name when referring to the type. >>> For example, TimeUnit vs timeUnit (the parameter). >>> >>> in @throws, use the parameter name instead of "the unit"; >>> For example, >>> + * @throws IllegalArgumentException if timeUnit cannot be converted >>> Thanks, Roger >>> >>> On 1/25/2016 11:06 AM, nadeesh tv wrote: >>>> Hi all, >>>> >>>> Please see the updated webrev >>>> http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>> >>>> -- >>>> Thanks and Regards, >>>> Nadeesh TV >>>> >>>> >>>> On 1/25/2016 9:01 PM, Stephen Colebourne wrote: >>>>> Typo "TimeUnitequivalent" >>>>> Otherwise looks good. >>>>> thanks >>>>> Stephen >>>>> >>>>> >>>>> >>>>> On 25 January 2016 at 15:25, nadeesh tv wrote: >>>>> >>>>>> Hi all, >>>>>> >>>>>> Please review a fix for conversion between Chronounit and Timeunit >>>>>> >>>>>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >>>>>> >>>>>> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>>> >>>>>> -- >>>>>> Thanks and Regards, >>>>>> Nadeesh TV >>>>>> >>>>>> >>>> >>> >> >> -- >> Thanks and Regards, >> Nadeesh TV >> > From paul.sandoz at oracle.com Fri Jan 29 15:45:45 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jan 2016 16:45:45 +0100 Subject: RFR 8148115: Stream.findFirst for unordered source optimization In-Reply-To: <1033010705.20160129202004@gmail.com> References: <1152739344.20160122121906@gmail.com> <30A68664-D525-41BF-9688-FAB7E1F6944A@oracle.com> <916767911.20160123184332@gmail.com> <1033010705.20160129202004@gmail.com> Message-ID: > On 29 Jan 2016, at 15:20, Tagir F. Valeev wrote: > > Here's updated webrev: > > http://cr.openjdk.java.net/~tvaleev/webrev/8148115/r2/ > Thanks, reviewed and in my queue. > PS> 293 if (!this.mustFindFirst) { > PS> 310 if (this.mustFindFirst) { > PS> Small thing, no need for ?this?. > > Done. > > > PS> You can now use Optional.ifPresentOrElse, which was added in 9: > > PS> ifPresentOrElse( > PS> t -> { ?}, > PS> () -> assertFalse(it.hasNext()): > > Done. I also added printing of the searched value if it's not found: > > assertTrue(contained, "Not found: "+r); > > Hopefully additional toString() and string concatenation does not eat > much time. Alternatively it could be rewritten as > > if(!contained) > fail("Not found: "+r); > > PS> For primitive streams, you are correct there are no unordered > PS> providers of data. in certain cases unordered is induced e.g. forEach testing. > > PS> A focused fix would be to exercise with say > PS> s.filter().unordered(), a more general fix would be as you say to > PS> add another provider for int, long and double (backed by say > PS> HashSet, or more simply/efficiently using unordered()). > > PS> We have to carefully review that because that may increase test > PS> times. We don?t have to apply to all data sets, plus we could also > PS> refine the ref data sets too in that regard, thus keep the overall actions similar or even smaller. > > PS> Shall we separate that in a new issue? > > Yes, I guess it would be better to log a new issue. Though I don't > know how to formulate it correctly. "Add tests for unordered primitive > sources for Stream operations"? "Refine test data sets for Stream > API"? Probably it would be better if you open an issue by yourself :-) > https://bugs.openjdk.java.net/browse/JDK-8148615 Thanks, Paul. From rachel.protacio at oracle.com Fri Jan 29 16:12:48 2016 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Fri, 29 Jan 2016 11:12:48 -0500 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56AA5FD6.4020400@oracle.com> References: <56A94C8C.9050306@oracle.com> <56AA5FD6.4020400@oracle.com> Message-ID: <56AB8F80.3020402@oracle.com> Thanks for the review, Dan. We'll see how the 9 v. 9_0 discussion plays out. Rachel On 1/28/2016 1:37 PM, Daniel D. Daugherty wrote: > On 1/27/16 4:02 PM, Rachel Protacio wrote: >> Hello! >> >> Small but important change for review: updating the JNI_VERSION and >> in so doing, changing the format from JNI_VERSION_1_x to >> JNI_VERSION_x_y (see code/bug for details). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8145098 >> >> hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot/ > > src/share/vm/prims/jni.cpp > No comments. > > src/share/vm/prims/jni.h > No comments. > > src/share/vm/runtime/thread.cpp > No comments. > > test/native_sanity/JniVersion.java > No comments. > > >> jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk/ > > src/java.base/share/native/include/jni.h > No comments. > > Meta-comment not specific to this bug. It sure would be nice if > finally had "one source of truth" for "jni.h" (and friends). > Perhaps in the top-level repo... :-) > > > Thumbs up! > > On Alan's comments about JNI_VERSION_9_0 versus JNI_VERSION_9, my > personal preference is for trailing zero version. There was recently > a comment on hotspot-dev at o.j.n and verona-dev at o.j.n about the missing > trailing zeros. Subject line is: Version special case '9' > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-January/021525.html > > > Dan > > >> >> I grep'ed through the code to find references to the current JNI >> version and believe I have caught all the ones that needed changing, >> plus the fact that all these tests pass: >> >> * local hotspot jtreg tests >> * my own sample JNI test to print and visually inspect the version >> (essentially what is performed by the updated >> hotspot/test/native_sanity/JniVersion.java test) >> * jck vm tests >> * local rbt colcated and noncolocated tests, especially for the >> purpose of hitting tonga/src/nsk/share/jvmti tests where >> JNI_GetVersion() is used >> >> Thank you, >> Rachel > From ivan.gerasimov at oracle.com Fri Jan 29 16:14:24 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Fri, 29 Jan 2016 19:14:24 +0300 Subject: RFR: 8079136: Accessing a nested sublist leads to StackOverflowError In-Reply-To: <85988769.20160129101608@gmail.com> References: <55486EA5.8000009@oracle.com> <56A9D4D7.5040506@oracle.com> <85988769.20160129101608@gmail.com> Message-ID: <56AB8FE0.5080301@oracle.com> Thank you Tagir for the review! On 29.01.2016 7:16, Tagir F. Valeev wrote: > Hello! > > AbstractList::subListRangeCheck could be replaced with new > Objects::checkFromToIndex for consistency with other methods. Also Yes, that would be good. However, for AbstractList.subList() we have this in spec: * @throws IllegalArgumentException if the endpoint indices are out of order * {@code (fromIndex > toIndex)} but the 2-args checkFromToIndex() will throw IndexOutOfBoundsException instead. If we use 3-args checkFromToIndex, it wouldn't save us anything in the terms of lines of code. It's misfortune that the spec of subList() doesn't match the spec of similar methods, like CharSequence.subSequence() or String.substring(). It even contradicts the spec of List.subList(), which declares only IOOB to be thrown! > AbstractList::rangeCheck could be replaced with Obejcts::checkIndex. > The same for ArrayList class. Right. Here's the updated webrev: http://cr.openjdk.java.net/~igerasim/8079136/06/webrev/ Sincerely yours, Ivan > Otherwise looks good to me. > > With best regards, > Tagir Valeev. > > IG> Hello everyone! > > IG> I'd like to respin the discussion. > IG> The previous attempt can be found here: > IG> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033159.html > > IG> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8079136 > IG> WEBREV: http://cr.openjdk.java.net/~igerasim/8079136/05/webrev/ > > IG> Here's the summary of the proposed changes: > IG> 1) > IG> Sublist of an AbstractList (AbstractList.SubList class) now maintains a > IG> link to the root AbstractList, and not only to the immediate parent list. > IG> This allows to perform modifications on the chain of sub-lists in a loop > IG> instead of using recursion (which, in particular, helps avoid the > IG> stack-overflow problem). > IG> The sublist is still backed by its parent list, in sense that all the > IG> modifications are correctly reflected in the backing list (as well as in > IG> all the grand parents the sublist, if any), so the fix does not violate > IG> the existing specification. > > IG> 2) > IG> It is proposed to update the spec of AbstractList.subList() in the > IG> @implSpec section by removing the words about private fields. > IG> With the fix, some of those private fields are removed. > > IG> 3) > IG> A similar change is proposed for the ArrayList.SubList class. > > IG> 4) > IG> Two regression tests are added: > IG> NestedSubList.java - demonstrates a stack-overflow problem when dealing > IG> with a long chain of sublists, > IG> SubList.java - tests basic functionality of sub-lists, which helps us > IG> make sure nothing is broken with the proposed change. > > IG> If people agree that the fix is good, I'll file a CCC request for > IG> changing the spec of AbstractList.subList(). > > IG> Sincerely yours, > IG> Ivan > > From huaming.li at oracle.com Fri Jan 29 16:43:28 2016 From: huaming.li at oracle.com (Hamlin Li) Date: Sat, 30 Jan 2016 00:43:28 +0800 Subject: RFR: 8076458: java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout In-Reply-To: <420E39BB-6CB5-41BA-9BF1-5BA2E3EB5357@oracle.com> References: <56A71B6A.9090605@oracle.com> <8ED500E5-D39D-468D-A08D-2EE9C09B8029@oracle.com> <56AB5E57.50906@oracle.com> <420E39BB-6CB5-41BA-9BF1-5BA2E3EB5357@oracle.com> Message-ID: <56AB96B0.60601@oracle.com> On 2016/1/29 20:53, Paul Sandoz wrote: >> On 29 Jan 2016, at 13:43, Hamlin Li wrote: >> >> Hi Paul, >> >> Sorry for delayed response, have been occupied by other higher priority task. >> Thanks for your review, I agree with you that your second approach is better. >> New webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.01/ >> > The changes to the data providers look ok. > > Would you mind splitting out the tests between StreamTestData and StreamTestData.small as outlined in 2) below. That way for the non-eplosive stuff we can still crunch on larger data without much of a slow down. Hi Pual, Yes, you're right, it does not slow down too much, it cost 15.553 seconds after the first revision(webrev.01), and it cost 16.064 after the second revision(webrev.02). Please check the webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.02/ >> Below are times cost for different ops: >> total :169.996 >> testOps only :108.988 >> testIntOps only :23.865 >> testLongOps only :22.326 >> testDoubleOps only :16.944 >> so, I build small data providers for each of them. >> > Ok, and i suspect/hope it drops by at least an order of magnitude with your changes applied. Yes, it cost 15.553 seconds after the first revision(webrev.01). > Out of curiosity i wonder what the times would be if using parallel GC rather than G1. With different GC options after second revision(webrev.02): -UseParallelGC: elapsed time (seconds): 16.047 +UseParallelGC: elapsed time (seconds): 13.263 -UseG1GC: elapsed time (seconds): 16.612 +UseG1GC: elapsed time (seconds): 16.998 -UseParallelOldGC: elapsed time (seconds): 16.039 +UseParallelOldGC: elapsed time (seconds): 14.297 Thank you -Hamlin > > Paul. > >> Thank you >> -Hamlin >> >> On 2016/1/26 21:18, Paul Sandoz wrote: >>> Hi Hamlin, >>> >>> Conservatively I would prefer not to remove data sets if at all possible. It will affect all tests, and leaf tasks for parallel streams should have enough data to crunch on. >>> >>> I suspect the problem of the flatMap test is not necessarily due to the source sizes being of 1000 elements but that there are tests that substitute an element whose value is m for n elements from 0..m, which can explode things and generate lots of garbage. >>> >>> Have you tried executing those kinds tests when the data size is < 1000? >>> >>> My bet is the FlatMapOpTest will run significantly faster and you will not need to split it out. >>> >>> There are two ways we could consider doing this: >>> >>> 1) Check the size in the test method: >>> >>> if (data.size() < 1000) { >>> exerciseOps(data, s -> s.flatMap(mfLt)); >>> exerciseOps(data, s -> s.flatMap(integerRangeMapper)); >>> exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); >>> } >>> >>> 2) Include a new data provider for smaller data sets >>> >>> @Test(dataProvider = "StreamTestData", dataProviderClass = StreamTestDataProvider.class) >>> public void testOps(String name, TestData.OfRef data) { >>> Collection result = exerciseOps(data, s -> s.flatMap(mfId)); >>> assertEquals(data.size(), result.size()); >>> >>> result = exerciseOps(data, s -> s.flatMap(mfNull)); >>> assertEquals(0, result.size()); >>> >>> result = exerciseOps(data, s-> s.flatMap(e -> Stream.empty())); >>> assertEquals(0, result.size()); >>> } >>> >>> @Test(dataProvider = "StreamTestData.small", dataProviderClass = StreamTestDataProvider.class) >>> public void testOpsX(String name, TestData.OfRef data) { >>> exerciseOps(data, s -> s.flatMap(mfLt)); >>> exerciseOps(data, s -> s.flatMap(integerRangeMapper)); >>> exerciseOps(data, s -> s.flatMap((Integer e) -> IntStream.range(0, e).boxed().limit(10))); >>> } >>> >>> I prefer the latter approach (applied to ref and primitive data sets). It?s more work, but i think the right direction. >>> >>> Paul. From martinrb at google.com Fri Jan 29 16:46:38 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jan 2016 08:46:38 -0800 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: <56AB814C.7070409@oracle.com> References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> Message-ID: I missed that this was modifying jsr166 files - looking now... On Fri, Jan 29, 2016 at 7:12 AM, Chris Hegarty wrote: > On 29/01/16 14:52, Roger Riggs wrote: >> >> Hi Nadeesh, >> >> Looks fine, >> >> Thanks, Roger >> >> >> On 1/27/2016 11:34 AM, nadeesh tv wrote: >>> >>> Hi all, >>> >>> Thanks for the suggestions. >>> Please see the updated webrev >>> http://cr.openjdk.java.net/~ntv/8141452/webrev.01/ > > > +1 This looks fine. > > Martin, Doug, > > I assume you are ok to accept this small change in > java.util.concurrent.TimeUnit. > > -Chris. > > >>> Regards, >>> Nadeesh TV >>> >>> On 1/25/2016 10:24 PM, Roger Riggs wrote: >>>> >>>> Hi Stephen, Nadeesh, >>>> >>>> TimeUnit.toChronoUnit is a static method. It seems redundant to have >>>> to pass an instance to a static method of its type. >>>> cu = TimeUnit.toChronoUnit(TimeUnit.SECONDS); >>>> >>>> Instead of: >>>> TimeUnit tu = TimeUnit.SECONDS; >>>> ChronoUnit cu = tu.toChronoUnit(); >>>> >>>> >>>> Minor edits please: >>>> >>>> in @param and @return use the type name when referring to the type. >>>> For example, TimeUnit vs timeUnit (the parameter). >>>> >>>> in @throws, use the parameter name instead of "the unit"; >>>> For example, >>>> + * @throws IllegalArgumentException if timeUnit cannot be converted >>>> Thanks, Roger >>>> >>>> On 1/25/2016 11:06 AM, nadeesh tv wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Please see the updated webrev >>>>> http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>> >>>>> -- >>>>> Thanks and Regards, >>>>> Nadeesh TV >>>>> >>>>> >>>>> On 1/25/2016 9:01 PM, Stephen Colebourne wrote: >>>>>> >>>>>> Typo "TimeUnitequivalent" >>>>>> Otherwise looks good. >>>>>> thanks >>>>>> Stephen >>>>>> >>>>>> >>>>>> >>>>>> On 25 January 2016 at 15:25, nadeesh tv wrote: >>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Please review a fix for conversion between Chronounit and Timeunit >>>>>>> >>>>>>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >>>>>>> >>>>>>> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>>>> >>>>>>> -- >>>>>>> Thanks and Regards, >>>>>>> Nadeesh TV >>>>>>> >>>>>>> >>>>> >>>> >>> >>> -- >>> Thanks and Regards, >>> Nadeesh TV >>> >> > From paul.sandoz at oracle.com Fri Jan 29 17:21:40 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jan 2016 18:21:40 +0100 Subject: RFR: 8076458: java/util/stream/test/org/openjdk/tests/java/util/stream/FlatMapOpTest.java timeout In-Reply-To: <56AB96B0.60601@oracle.com> References: <56A71B6A.9090605@oracle.com> <8ED500E5-D39D-468D-A08D-2EE9C09B8029@oracle.com> <56AB5E57.50906@oracle.com> <420E39BB-6CB5-41BA-9BF1-5BA2E3EB5357@oracle.com> <56AB96B0.60601@oracle.com> Message-ID: > On 29 Jan 2016, at 17:43, Hamlin Li wrote: > > > > On 2016/1/29 20:53, Paul Sandoz wrote: >>> On 29 Jan 2016, at 13:43, Hamlin Li wrote: >>> >>> Hi Paul, >>> >>> Sorry for delayed response, have been occupied by other higher priority task. >>> Thanks for your review, I agree with you that your second approach is better. >>> New webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.01/ >>> >> The changes to the data providers look ok. >> >> Would you mind splitting out the tests between StreamTestData and StreamTestData.small as outlined in 2) below. That way for the non-eplosive stuff we can still crunch on larger data without much of a slow down. > Hi Pual, > > Yes, you're right, it does not slow down too much, it cost 15.553 seconds after the first revision(webrev.01), and it cost 16.064 after the second revision(webrev.02). > Please check the webrev: http://cr.openjdk.java.net/~mli/8076458/webrev.02/ +1, reviewed. Do you need me to push it? >>> Below are times cost for different ops: >>> total :169.996 >>> testOps only :108.988 >>> testIntOps only :23.865 >>> testLongOps only :22.326 >>> testDoubleOps only :16.944 >>> so, I build small data providers for each of them. >>> >> Ok, and i suspect/hope it drops by at least an order of magnitude with your changes applied. > Yes, it cost 15.553 seconds after the first revision(webrev.01). Great. >> Out of curiosity i wonder what the times would be if using parallel GC rather than G1. > With different GC options after second revision(webrev.02): > -UseParallelGC: elapsed time (seconds): 16.047 > +UseParallelGC: elapsed time (seconds): 13.263 > -UseG1GC: elapsed time (seconds): 16.612 > +UseG1GC: elapsed time (seconds): 16.998 > -UseParallelOldGC: elapsed time (seconds): 16.039 > +UseParallelOldGC: elapsed time (seconds): 14.297 > Ok, so i suspect in the case of when your patch is not applied the difference is greater in absolute terms and G1 in a sense might be causing such memory intensive tests to slow down. Paul. From martinrb at google.com Fri Jan 29 17:24:14 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jan 2016 09:24:14 -0800 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> Message-ID: I propose that we the jsr166 maintainers take over this change (sorry for butting in!), pushing it into openjdk from jsr166 CVS. The tests as they are written today won't work because TimeUnit/Basic.java is not a testng test. But I don't think we should fix that - instead, tests for these methods should be added to our tck tests (I can do that). Coincidentally, I am close to committing our tck tests to openjdk. As for the new API - the simple changes in the webrev are perfectly fine, but I expected to see some additional conversions. TimeUnit is for expressing elapsed time durations (even though j.u.c. doesn't have a separate class for that), so I vaguely expect conversions to/from Duration. But we won't do anything unless date/time experts (Stephen?) think it's a good idea. On Fri, Jan 29, 2016 at 8:46 AM, Martin Buchholz wrote: > I missed that this was modifying jsr166 files - looking now... > > On Fri, Jan 29, 2016 at 7:12 AM, Chris Hegarty wrote: >> On 29/01/16 14:52, Roger Riggs wrote: >>> >>> Hi Nadeesh, >>> >>> Looks fine, >>> >>> Thanks, Roger >>> >>> >>> On 1/27/2016 11:34 AM, nadeesh tv wrote: >>>> >>>> Hi all, >>>> >>>> Thanks for the suggestions. >>>> Please see the updated webrev >>>> http://cr.openjdk.java.net/~ntv/8141452/webrev.01/ >> >> >> +1 This looks fine. >> >> Martin, Doug, >> >> I assume you are ok to accept this small change in >> java.util.concurrent.TimeUnit. >> >> -Chris. >> >> >>>> Regards, >>>> Nadeesh TV >>>> >>>> On 1/25/2016 10:24 PM, Roger Riggs wrote: >>>>> >>>>> Hi Stephen, Nadeesh, >>>>> >>>>> TimeUnit.toChronoUnit is a static method. It seems redundant to have >>>>> to pass an instance to a static method of its type. >>>>> cu = TimeUnit.toChronoUnit(TimeUnit.SECONDS); >>>>> >>>>> Instead of: >>>>> TimeUnit tu = TimeUnit.SECONDS; >>>>> ChronoUnit cu = tu.toChronoUnit(); >>>>> >>>>> >>>>> Minor edits please: >>>>> >>>>> in @param and @return use the type name when referring to the type. >>>>> For example, TimeUnit vs timeUnit (the parameter). >>>>> >>>>> in @throws, use the parameter name instead of "the unit"; >>>>> For example, >>>>> + * @throws IllegalArgumentException if timeUnit cannot be converted >>>>> Thanks, Roger >>>>> >>>>> On 1/25/2016 11:06 AM, nadeesh tv wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> Please see the updated webrev >>>>>> http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>>> >>>>>> -- >>>>>> Thanks and Regards, >>>>>> Nadeesh TV >>>>>> >>>>>> >>>>>> On 1/25/2016 9:01 PM, Stephen Colebourne wrote: >>>>>>> >>>>>>> Typo "TimeUnitequivalent" >>>>>>> Otherwise looks good. >>>>>>> thanks >>>>>>> Stephen >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 25 January 2016 at 15:25, nadeesh tv wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> Please review a fix for conversion between Chronounit and Timeunit >>>>>>>> >>>>>>>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >>>>>>>> >>>>>>>> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>>>>> >>>>>>>> -- >>>>>>>> Thanks and Regards, >>>>>>>> Nadeesh TV >>>>>>>> >>>>>>>> >>>>>> >>>>> >>>> >>>> -- >>>> Thanks and Regards, >>>> Nadeesh TV >>>> >>> >> From steve.drach at oracle.com Fri Jan 29 17:27:43 2016 From: steve.drach at oracle.com (Steve Drach) Date: Fri, 29 Jan 2016 09:27:43 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AB6BF8.5010302@oracle.com> References: <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! F8.5010302@oracle.com> Message-ID: >> I?ve released a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.04/index.html that addresses the above issue. >> > Thank you, the javadoc is much clearer and readable now. > > The first mention of multi-release JARs is in the first paragraph where it has "for processing multi-release jar files". I would be tempted to drop this from the first paragraph because the second paragraph introduces the concept. It makes sense to leave it in there because of the context. It?s one of 2 things that differentiate JarFile from ZipFile . > > In the second paragraph is has "The versioned entries are partitioned by the major version of Java platform releases, starting with release 9" and then it goes on to explain how versioned entries are partitioned. This has potential to confuse the reader as it gives an initial impression that the oldest version entry is 9 but then the says 8 < n. I realize the text is trying to say that Java SE 9 is the first release to support this but it could be confused. I would be tempted to just drop the mention of release 9 in this paragraph. I?ll do that. > > This may have come up before but JarFile has two sets of constructors, one takes the file path as a String, the other as a File. I just wondering about introduce a second constructor so that they match. We felt that one constructor was sufficient on the theory that it?s use will be infrequent, at least initially. And it?s easy to map a String to a File. > > One other thing that I've been wondering about is the stream() and entries() methods. Has there been any discussion about these doing filter? Not that I?m aware of. > Maybe it is too expensive and best left to the user of the API? Part of the context for asking is modular JARs of course. Perhaps you can elaborate. From scolebourne at joda.org Fri Jan 29 17:37:40 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 29 Jan 2016 17:37:40 +0000 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> Message-ID: In the ideal fix, all methods that take the combination (long, TimeUnit) would be supplemented by an override that takes (Duration). eg. Future.get(long, TimeUnit) has an additional get(Duration). However, this is a lot of work, might be unpopular and would be slower for j.u.concurrent use. The proposal is minimal, allowing the conversion to occur more easily. The Duration.of(long, TemporalUnit) could then be used to create a Duration. I'll note again that the alternative to this would be to make TimeUnit implement TemporalUnit. Whether that is less confusing is an open question. The specific question asked is whether there should be a TimeUnit::toDuration(long) method. I'm not overly fussed, as I think there are probably enough alternatives. Stephen On 29 January 2016 at 17:24, Martin Buchholz wrote: > I propose that we the jsr166 maintainers take over this change (sorry > for butting in!), pushing it into openjdk from jsr166 CVS. > The tests as they are written today won't work because > TimeUnit/Basic.java is not a testng test. > But I don't think we should fix that - instead, tests for these > methods should be added to our tck tests (I can do that). > Coincidentally, I am close to committing our tck tests to openjdk. > > As for the new API - the simple changes in the webrev are perfectly > fine, but I expected to see some additional conversions. TimeUnit is > for expressing elapsed time durations (even though j.u.c. doesn't have > a separate class for that), so I vaguely expect conversions to/from > Duration. But we won't do anything unless date/time experts > (Stephen?) think it's a good idea. > > > On Fri, Jan 29, 2016 at 8:46 AM, Martin Buchholz wrote: >> I missed that this was modifying jsr166 files - looking now... >> >> On Fri, Jan 29, 2016 at 7:12 AM, Chris Hegarty wrote: >>> On 29/01/16 14:52, Roger Riggs wrote: >>>> >>>> Hi Nadeesh, >>>> >>>> Looks fine, >>>> >>>> Thanks, Roger >>>> >>>> >>>> On 1/27/2016 11:34 AM, nadeesh tv wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Thanks for the suggestions. >>>>> Please see the updated webrev >>>>> http://cr.openjdk.java.net/~ntv/8141452/webrev.01/ >>> >>> >>> +1 This looks fine. >>> >>> Martin, Doug, >>> >>> I assume you are ok to accept this small change in >>> java.util.concurrent.TimeUnit. >>> >>> -Chris. >>> >>> >>>>> Regards, >>>>> Nadeesh TV >>>>> >>>>> On 1/25/2016 10:24 PM, Roger Riggs wrote: >>>>>> >>>>>> Hi Stephen, Nadeesh, >>>>>> >>>>>> TimeUnit.toChronoUnit is a static method. It seems redundant to have >>>>>> to pass an instance to a static method of its type. >>>>>> cu = TimeUnit.toChronoUnit(TimeUnit.SECONDS); >>>>>> >>>>>> Instead of: >>>>>> TimeUnit tu = TimeUnit.SECONDS; >>>>>> ChronoUnit cu = tu.toChronoUnit(); >>>>>> >>>>>> >>>>>> Minor edits please: >>>>>> >>>>>> in @param and @return use the type name when referring to the type. >>>>>> For example, TimeUnit vs timeUnit (the parameter). >>>>>> >>>>>> in @throws, use the parameter name instead of "the unit"; >>>>>> For example, >>>>>> + * @throws IllegalArgumentException if timeUnit cannot be converted >>>>>> Thanks, Roger >>>>>> >>>>>> On 1/25/2016 11:06 AM, nadeesh tv wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Please see the updated webrev >>>>>>> http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>>>> >>>>>>> -- >>>>>>> Thanks and Regards, >>>>>>> Nadeesh TV >>>>>>> >>>>>>> >>>>>>> On 1/25/2016 9:01 PM, Stephen Colebourne wrote: >>>>>>>> >>>>>>>> Typo "TimeUnitequivalent" >>>>>>>> Otherwise looks good. >>>>>>>> thanks >>>>>>>> Stephen >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On 25 January 2016 at 15:25, nadeesh tv wrote: >>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> Please review a fix for conversion between Chronounit and Timeunit >>>>>>>>> >>>>>>>>> Bug ID : https://bugs.openjdk.java.net/browse/JDK-8141452 >>>>>>>>> >>>>>>>>> webrev: http://cr.openjdk.java.net/~ntv/8141452/webrev.00/ >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Thanks and Regards, >>>>>>>>> Nadeesh TV >>>>>>>>> >>>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> -- >>>>> Thanks and Regards, >>>>> Nadeesh TV >>>>> >>>> >>> From paul.sandoz at oracle.com Fri Jan 29 17:39:48 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 29 Jan 2016 18:39:48 +0100 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> Message-ID: <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> > On 29 Jan 2016, at 18:27, Steve Drach wrote: >> >> This may have come up before but JarFile has two sets of constructors, one takes the file path as a String, the other as a File. I just wondering about introduce a second constructor so that they match. > > We felt that one constructor was sufficient on the theory that it?s use will be infrequent, at least initially. And it?s easy to map a String to a File. > Right, my preference is to stick to one for the moment and add another later if really needed. >> >> One other thing that I've been wondering about is the stream() and entries() methods. Has there been any discussion about these doing filter? > > Not that I?m aware of. > >> Maybe it is too expensive and best left to the user of the API? Part of the context for asking is modular JARs of course. > > Perhaps you can elaborate. Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. My assumption was the traversal should by default be consistent with a calls to getEntry, thus: jarFile.stream().forEach(e -> { JarEntry je = jarFile.getJarEntry(e.getName()); assert e.equals(je); }); There might need to be another stream method that returns all entries. Paul. From miroslav.kos at oracle.com Fri Jan 29 17:40:49 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Fri, 29 Jan 2016 18:40:49 +0100 Subject: RFR [9]: 8138699 Iproving JAX-B javadoc Message-ID: <56ABA421.5050605@oracle.com> Hi everybody, please review following patch - those are changes to javadoc, previously consulted with sqe. The javadoc now is not always clear, which makes difficult to develop proper tests. JIRA: https://bugs.openjdk.java.net/browse/JDK-8138699 webrev: http://cr.openjdk.java.net/~mkos/8138699/jaxws.01/ Thanks Miran From ramanand.patil at oracle.com Fri Jan 29 17:56:25 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Fri, 29 Jan 2016 09:56:25 -0800 (PST) Subject: RFR: 8148570: TzdbZoneRulesCompiler.java throws Null Pointer Exception While Compiling and building TZDB data file Message-ID: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> HI all, Please review this trivial fix for Bug - https://bugs.openjdk.java.net/browse/JDK-8148570 Bug Description - While compiling and building TZDB data file(tzdata2016a) when transition rule(for Iran) doesn't have the day-of-week data a null pointer exception is thrown. Webrev - http://cr.openjdk.java.net/~rpatil/8148570/webrev.00/ Fix - While getting value from DayOfWeek, -1 is returned if the DayOfWeek is null. The reason to return "-1" being, the same value is checked later while getting day-of-week byte (dowbyte) at line no. 251, ZoneRules.java. Regards, Ramanand. From Roger.Riggs at Oracle.com Fri Jan 29 18:12:54 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Jan 2016 13:12:54 -0500 Subject: RFR: 8148570: TzdbZoneRulesCompiler.java throws Null Pointer Exception While Compiling and building TZDB data file In-Reply-To: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> References: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> Message-ID: <56ABABA6.2040108@Oracle.com> Hi Ramanand, Is there a specific regression test that can be written for the case? Otherwise, looks fine. Roger On 1/29/2016 12:56 PM, Ramanand Patil wrote: > HI all, > > Please review this trivial fix for Bug - https://bugs.openjdk.java.net/browse/JDK-8148570 > > Bug Description - While compiling and building TZDB data file(tzdata2016a) when transition rule(for Iran) doesn't have the day-of-week data a null pointer exception is thrown. > > Webrev - http://cr.openjdk.java.net/~rpatil/8148570/webrev.00/ > > Fix - While getting value from DayOfWeek, -1 is returned if the DayOfWeek is null. The reason to return "-1" being, the same value is checked later while getting day-of-week byte (dowbyte) at line no. 251, ZoneRules.java. > > > > Regards, > > Ramanand. > > > > From Roger.Riggs at Oracle.com Fri Jan 29 18:47:09 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Jan 2016 13:47:09 -0500 Subject: RFR 9: 8146773 java/lang/ref/CleanerTest.java CleanerTest.testRefSubtypes() fails Message-ID: <56ABB3AD.2030401@Oracle.com> Please review a fix for two test issues. When run with -Xcomp and other switches that change GC behavior; the CleanerTest was not giving enough to time to see the result of cleaning. Also, added an adjustment of the number of cycles by the timeoutFactor. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-cleanertest-8146773/ 8146773: java/lang/ref/CleanerTest.java CleanerTest.testRefSubtypes() fails 8148352: CleanerTest fails: Cleanable should have been freed Thanks, Roger From joe.darcy at oracle.com Fri Jan 29 18:53:57 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 29 Jan 2016 10:53:57 -0800 Subject: JDK 9 RFR of JDK-8148627: Problem list TestMaxCachedBufferSize.java Message-ID: <56ABB545.7080600@oracle.com> Hello, Until JDK-8148540 is addressed, the test sun/nio/ch/TestMaxCachedBufferSize.java should be problem listed on the platforms is fails on. Please review the patch below. Thanks, -Joe diff -r eecb3e75b0d8 test/ProblemList.txt --- a/test/ProblemList.txt Thu Jan 28 18:08:53 2016 -0800 +++ b/test/ProblemList.txt Fri Jan 29 10:51:25 2016 -0800 @@ -197,6 +197,9 @@ java/nio/file/WatchService/Basic.java solaris-all java/nio/file/WatchService/LotsOfEvents.java solaris-all +# 8148540 +sun/nio/ch/TestMaxCachedBufferSize.java solaris-i586,windows-i586 + ############################################################################ # jdk_rmi diff -r eecb3e75b0d8 test/sun/nio/ch/TestMaxCachedBufferSize.java --- a/test/sun/nio/ch/TestMaxCachedBufferSize.java Thu Jan 28 18:08:53 2016 -0800 +++ b/test/sun/nio/ch/TestMaxCachedBufferSize.java Fri Jan 29 10:51:25 2016 -0800 @@ -50,6 +50,7 @@ * @run main/othervm -Djdk.nio.maxCachedBufferSize=10000000 TestMaxCachedBufferSize * * @summary Test the implementation of the jdk.nio.maxCachedBufferSize property. + * @key intermittent */ public class TestMaxCachedBufferSize { private static final int DEFAULT_ITERS = 10 * 1000; From ramanand.patil at oracle.com Fri Jan 29 19:01:02 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Fri, 29 Jan 2016 11:01:02 -0800 (PST) Subject: RFR: 8148570: TzdbZoneRulesCompiler.java throws Null Pointer Exception While Compiling and building TZDB data file In-Reply-To: <56ABABA6.2040108@Oracle.com> References: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> <56ABABA6.2040108@Oracle.com> Message-ID: <657b0821-2fd0-425a-b401-83b1c2e0eb91@default> Hi Roger, Writing the regression test will be little tricky, because the transition rules are read from the IANA provided tzdata files and during JDK building the tzdb.dat file is created. This issue was actually found after JDK9 build failure with integrated tzdata2016a files. Being a very obvious trivial bug fix, I feel this should be Ok without test. I have updated the label to include 'noreg-trivial'. Regards, Ramanand. -----Original Message----- From: Roger Riggs Sent: Friday, January 29, 2016 11:43 PM To: core-libs-dev at openjdk.java.net Subject: Re: RFR: 8148570: TzdbZoneRulesCompiler.java throws Null Pointer Exception While Compiling and building TZDB data file Hi Ramanand, Is there a specific regression test that can be written for the case? Otherwise, looks fine. Roger On 1/29/2016 12:56 PM, Ramanand Patil wrote: > HI all, > > Please review this trivial fix for Bug - https://bugs.openjdk.java.net/browse/JDK-8148570 > > Bug Description - While compiling and building TZDB data file(tzdata2016a) when transition rule(for Iran) doesn't have the day-of-week data a null pointer exception is thrown. > > Webrev - http://cr.openjdk.java.net/~rpatil/8148570/webrev.00/ > > Fix - While getting value from DayOfWeek, -1 is returned if the DayOfWeek is null. The reason to return "-1" being, the same value is checked later while getting day-of-week byte (dowbyte) at line no. 251, ZoneRules.java. > > > > Regards, > > Ramanand. > > > > From martinrb at google.com Fri Jan 29 19:12:26 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jan 2016 11:12:26 -0800 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> Message-ID: Here is a proposed alternative patch (against jsr166 CVS). Code is reworded and reformatted. Tests are junit-ified. Round-trip tests are added. toChronoUnit no longer throws IAE, because it cannot (and we commit to having ChronoUnit be a superset of TimeUnit, in perpetuity; toChronoUnit will forever be an injective function). Index: src/main/java/util/concurrent/TimeUnit.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/TimeUnit.java,v retrieving revision 1.47 diff -u -r1.47 TimeUnit.java --- src/main/java/util/concurrent/TimeUnit.java 20 Sep 2015 17:03:23 -0000 1.47 +++ src/main/java/util/concurrent/TimeUnit.java 29 Jan 2016 19:05:01 -0000 @@ -6,6 +6,9 @@ package java.util.concurrent; +import java.time.temporal.ChronoUnit; +import java.util.Objects; + /** * A {@code TimeUnit} represents time durations at a given unit of * granularity and provides utility methods to convert across units, @@ -361,4 +364,48 @@ } } + /** + * Converts this {@code TimeUnit} to the equivalent {@code ChronoUnit}. + * + * @return the converted equivalent ChronoUnit + * @since 9 + */ + public ChronoUnit toChronoUnit() { + switch (this) { + case NANOSECONDS: return ChronoUnit.NANOS; + case MICROSECONDS: return ChronoUnit.MICROS; + case MILLISECONDS: return ChronoUnit.MILLIS; + case SECONDS: return ChronoUnit.SECONDS; + case MINUTES: return ChronoUnit.MINUTES; + case HOURS: return ChronoUnit.HOURS; + case DAYS: return ChronoUnit.DAYS; + default: throw new AssertionError(); + } + } + + /** + * Converts a {@code ChronoUnit} to the equivalent {@code TimeUnit}. + * + * @param chronoUnit the ChronoUnit to convert + * @return the converted equivalent TimeUnit + * @throws IllegalArgumentException if {@code chronoUnit} has no + * equivalent TimeUnit + * @throws NullPointerException if {@code chronoUnit} is null + * @since 9 + */ + public static TimeUnit of(ChronoUnit chronoUnit) { + switch (Objects.requireNonNull(chronoUnit, "chronoUnit")) { + case NANOS: return TimeUnit.NANOSECONDS; + case MICROS: return TimeUnit.MICROSECONDS; + case MILLIS: return TimeUnit.MILLISECONDS; + case SECONDS: return TimeUnit.SECONDS; + case MINUTES: return TimeUnit.MINUTES; + case HOURS: return TimeUnit.HOURS; + case DAYS: return TimeUnit.DAYS; + default: + throw new IllegalArgumentException( + "No TimeUnit equivalent for " + chronoUnit); + } + } + } Index: src/test/tck/TimeUnitTest.java =================================================================== RCS file: /export/home/jsr166/jsr166/jsr166/src/test/tck/TimeUnitTest.java,v retrieving revision 1.25 diff -u -r1.25 TimeUnitTest.java --- src/test/tck/TimeUnitTest.java 25 Apr 2015 04:55:31 -0000 1.25 +++ src/test/tck/TimeUnitTest.java 29 Jan 2016 19:05:02 -0000 @@ -14,6 +14,7 @@ import static java.util.concurrent.TimeUnit.NANOSECONDS; import static java.util.concurrent.TimeUnit.SECONDS; +import java.time.temporal.ChronoUnit; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit; @@ -433,4 +434,49 @@ assertSame(x, serialClone(x)); } + /** + * tests for toChronoUnit. + */ + public void testToChronoUnit() throws Exception { + assertSame(ChronoUnit.NANOS, NANOSECONDS.toChronoUnit()); + assertSame(ChronoUnit.MICROS, MICROSECONDS.toChronoUnit()); + assertSame(ChronoUnit.MILLIS, MILLISECONDS.toChronoUnit()); + assertSame(ChronoUnit.SECONDS, SECONDS.toChronoUnit()); + assertSame(ChronoUnit.MINUTES, MINUTES.toChronoUnit()); + assertSame(ChronoUnit.HOURS, HOURS.toChronoUnit()); + assertSame(ChronoUnit.DAYS, DAYS.toChronoUnit()); + + // Every TimeUnit has a defined ChronoUnit equivalent + for (TimeUnit x : TimeUnit.values()) + assertSame(x, TimeUnit.of(x.toChronoUnit())); + } + + /** + * tests for TimeUnit.of(ChronoUnit). + */ + public void testTimeUnitOf() throws Exception { + assertSame(NANOSECONDS, TimeUnit.of(ChronoUnit.NANOS)); + assertSame(MICROSECONDS, TimeUnit.of(ChronoUnit.MICROS)); + assertSame(MILLISECONDS, TimeUnit.of(ChronoUnit.MILLIS)); + assertSame(SECONDS, TimeUnit.of(ChronoUnit.SECONDS)); + assertSame(MINUTES, TimeUnit.of(ChronoUnit.MINUTES)); + assertSame(HOURS, TimeUnit.of(ChronoUnit.HOURS)); + assertSame(DAYS, TimeUnit.of(ChronoUnit.DAYS)); + + assertThrows(NullPointerException.class, + () -> TimeUnit.of((ChronoUnit)null)); + + // ChronoUnits either round trip to their TimeUnit + // equivalents, or throw IllegalMonitorStateException. + for (ChronoUnit cu : ChronoUnit.values()) { + final TimeUnit tu; + try { + tu = TimeUnit.of(cu); + } catch (IllegalArgumentException acceptable) { + continue; + } + assertSame(cu, tu.toChronoUnit()); + } + } + } From Roger.Riggs at Oracle.com Fri Jan 29 19:17:26 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Jan 2016 14:17:26 -0500 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> Message-ID: <56ABBAC6.8020507@Oracle.com> Hi Martin, Where did IllegalMonitorStateException creep in from? Should I update the CCC I started or will you start fresh? Otherwise, ok. Roger On 1/29/2016 2:12 PM, Martin Buchholz wrote: > Here is a proposed alternative patch (against jsr166 CVS). > Code is reworded and reformatted. > Tests are junit-ified. > Round-trip tests are added. > toChronoUnit no longer throws IAE, because it cannot (and we commit to > having ChronoUnit be a superset of TimeUnit, in perpetuity; > toChronoUnit will forever be an injective function). > > Index: src/main/java/util/concurrent/TimeUnit.java > =================================================================== > RCS file: /export/home/jsr166/jsr166/jsr166/src/main/java/util/concurrent/TimeUnit.java,v > retrieving revision 1.47 > diff -u -r1.47 TimeUnit.java > --- src/main/java/util/concurrent/TimeUnit.java 20 Sep 2015 17:03:23 -0000 1.47 > +++ src/main/java/util/concurrent/TimeUnit.java 29 Jan 2016 19:05:01 -0000 > @@ -6,6 +6,9 @@ > > package java.util.concurrent; > > +import java.time.temporal.ChronoUnit; > +import java.util.Objects; > + > /** > * A {@code TimeUnit} represents time durations at a given unit of > * granularity and provides utility methods to convert across units, > @@ -361,4 +364,48 @@ > } > } > > + /** > + * Converts this {@code TimeUnit} to the equivalent {@code ChronoUnit}. > + * > + * @return the converted equivalent ChronoUnit > + * @since 9 > + */ > + public ChronoUnit toChronoUnit() { > + switch (this) { > + case NANOSECONDS: return ChronoUnit.NANOS; > + case MICROSECONDS: return ChronoUnit.MICROS; > + case MILLISECONDS: return ChronoUnit.MILLIS; > + case SECONDS: return ChronoUnit.SECONDS; > + case MINUTES: return ChronoUnit.MINUTES; > + case HOURS: return ChronoUnit.HOURS; > + case DAYS: return ChronoUnit.DAYS; > + default: throw new AssertionError(); > + } > + } > + > + /** > + * Converts a {@code ChronoUnit} to the equivalent {@code TimeUnit}. > + * > + * @param chronoUnit the ChronoUnit to convert > + * @return the converted equivalent TimeUnit > + * @throws IllegalArgumentException if {@code chronoUnit} has no > + * equivalent TimeUnit > + * @throws NullPointerException if {@code chronoUnit} is null > + * @since 9 > + */ > + public static TimeUnit of(ChronoUnit chronoUnit) { > + switch (Objects.requireNonNull(chronoUnit, "chronoUnit")) { > + case NANOS: return TimeUnit.NANOSECONDS; > + case MICROS: return TimeUnit.MICROSECONDS; > + case MILLIS: return TimeUnit.MILLISECONDS; > + case SECONDS: return TimeUnit.SECONDS; > + case MINUTES: return TimeUnit.MINUTES; > + case HOURS: return TimeUnit.HOURS; > + case DAYS: return TimeUnit.DAYS; > + default: > + throw new IllegalArgumentException( > + "No TimeUnit equivalent for " + chronoUnit); > + } > + } > + > } > Index: src/test/tck/TimeUnitTest.java > =================================================================== > RCS file: /export/home/jsr166/jsr166/jsr166/src/test/tck/TimeUnitTest.java,v > retrieving revision 1.25 > diff -u -r1.25 TimeUnitTest.java > --- src/test/tck/TimeUnitTest.java 25 Apr 2015 04:55:31 -0000 1.25 > +++ src/test/tck/TimeUnitTest.java 29 Jan 2016 19:05:02 -0000 > @@ -14,6 +14,7 @@ > import static java.util.concurrent.TimeUnit.NANOSECONDS; > import static java.util.concurrent.TimeUnit.SECONDS; > > +import java.time.temporal.ChronoUnit; > import java.util.concurrent.CountDownLatch; > import java.util.concurrent.TimeUnit; > > @@ -433,4 +434,49 @@ > assertSame(x, serialClone(x)); > } > > + /** > + * tests for toChronoUnit. > + */ > + public void testToChronoUnit() throws Exception { > + assertSame(ChronoUnit.NANOS, NANOSECONDS.toChronoUnit()); > + assertSame(ChronoUnit.MICROS, MICROSECONDS.toChronoUnit()); > + assertSame(ChronoUnit.MILLIS, MILLISECONDS.toChronoUnit()); > + assertSame(ChronoUnit.SECONDS, SECONDS.toChronoUnit()); > + assertSame(ChronoUnit.MINUTES, MINUTES.toChronoUnit()); > + assertSame(ChronoUnit.HOURS, HOURS.toChronoUnit()); > + assertSame(ChronoUnit.DAYS, DAYS.toChronoUnit()); > + > + // Every TimeUnit has a defined ChronoUnit equivalent > + for (TimeUnit x : TimeUnit.values()) > + assertSame(x, TimeUnit.of(x.toChronoUnit())); > + } > + > + /** > + * tests for TimeUnit.of(ChronoUnit). > + */ > + public void testTimeUnitOf() throws Exception { > + assertSame(NANOSECONDS, TimeUnit.of(ChronoUnit.NANOS)); > + assertSame(MICROSECONDS, TimeUnit.of(ChronoUnit.MICROS)); > + assertSame(MILLISECONDS, TimeUnit.of(ChronoUnit.MILLIS)); > + assertSame(SECONDS, TimeUnit.of(ChronoUnit.SECONDS)); > + assertSame(MINUTES, TimeUnit.of(ChronoUnit.MINUTES)); > + assertSame(HOURS, TimeUnit.of(ChronoUnit.HOURS)); > + assertSame(DAYS, TimeUnit.of(ChronoUnit.DAYS)); > + > + assertThrows(NullPointerException.class, > + () -> TimeUnit.of((ChronoUnit)null)); > + > + // ChronoUnits either round trip to their TimeUnit > + // equivalents, or throw IllegalMonitorStateException. > + for (ChronoUnit cu : ChronoUnit.values()) { > + final TimeUnit tu; > + try { > + tu = TimeUnit.of(cu); > + } catch (IllegalArgumentException acceptable) { > + continue; > + } > + assertSame(cu, tu.toChronoUnit()); > + } > + } > + > } From martinrb at google.com Fri Jan 29 19:28:20 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jan 2016 11:28:20 -0800 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: <56ABBAC6.8020507@Oracle.com> References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> <56ABBAC6.8020507@Oracle.com> Message-ID: On Fri, Jan 29, 2016 at 11:17 AM, Roger Riggs wrote: > Hi Martin, > > Where did IllegalMonitorStateException creep in from? Oops. My Emacs keeps suggesting the wrong completion. Fixed. > Should I update the CCC I started or will you start fresh? If you could update the CCC, that would be awesome! The only substantive change was removing a @throws IAE and adding @throws NPE. From Roger.Riggs at Oracle.com Fri Jan 29 19:52:08 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Jan 2016 14:52:08 -0500 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> <56ABBAC6.8020507@Oracle.com> Message-ID: <56ABC2E8.2030209@Oracle.com> Hi Martin, On 1/29/2016 2:28 PM, Martin Buchholz wrote: > On Fri, Jan 29, 2016 at 11:17 AM, Roger Riggs wrote: >> Hi Martin, >> >> Where did IllegalMonitorStateException creep in from? > Oops. My Emacs keeps suggesting the wrong completion. Fixed. > >> Should I update the CCC I started or will you start fresh? > If you could update the CCC, that would be awesome! > The only substantive change was removing a @throws IAE and adding @throws NPE. ok Roger From steve.drach at oracle.com Fri Jan 29 19:58:19 2016 From: steve.drach at oracle.com (Steve Drach) Date: Fri, 29 Jan 2016 11:58:19 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> References: <561F16C1.6090702@oracle.com> <56207360.6040502@oracle.com> <18A22B13-E31D-45D6-A350-40C35BDDEE73@oracle.com> <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> Message-ID: >>> One other thing that I've been wondering about is the stream() and entries() methods. Has there been any discussion about these doing filter? >> >> Not that I?m aware of. >> >>> Maybe it is too expensive and best left to the user of the API? Part of the context for asking is modular JARs of course. >> >> Perhaps you can elaborate. > > Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. > > My assumption was the traversal should by default be consistent with a calls to getEntry, thus: > > jarFile.stream().forEach(e -> { > JarEntry je = jarFile.getJarEntry(e.getName()); > assert e.equals(je); > }); > > There might need to be another stream method that returns all entries. JarFile.entries() and JarFile.stream() return all the entries; they are not filtered. Your example would work if the JarFile is constructed without a Release argument or with the Release.BASE argument. The assertion would fail if it?s constructed with any other Release argument. Adding a filter and a map to the stream pipeline would make your example succeed with all values of Release. From Alan.Bateman at oracle.com Fri Jan 29 19:59:12 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 29 Jan 2016 19:59:12 +0000 Subject: JDK 9 RFR of JDK-8148627: Problem list TestMaxCachedBufferSize.java In-Reply-To: <56ABB545.7080600@oracle.com> References: <56ABB545.7080600@oracle.com> Message-ID: <56ABC490.4060609@oracle.com> On 29/01/2016 18:53, joe darcy wrote: > Hello, > > Until JDK-8148540 is addressed, the test > > sun/nio/ch/TestMaxCachedBufferSize.java > > should be problem listed on the platforms is fails on. Please review > the patch below. > I think this test will be tricky to be reliable on 32-bit so the simplest is to just restrict it to 64-bit with: @requires (sun.arch.data.model == "64") -Alan. From joe.darcy at oracle.com Fri Jan 29 20:13:33 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 29 Jan 2016 12:13:33 -0800 Subject: JDK 9 RFR of JDK-8148627: Problem list TestMaxCachedBufferSize.java In-Reply-To: <56ABC490.4060609@oracle.com> References: <56ABB545.7080600@oracle.com> <56ABC490.4060609@oracle.com> Message-ID: <56ABC7ED.60103@oracle.com> Hi Alan, Limiting the test to 64-bit platforms is fine with me; I'll push the change as you've suggested later today. Thanks, -Joe On 1/29/2016 11:59 AM, Alan Bateman wrote: > > > On 29/01/2016 18:53, joe darcy wrote: >> Hello, >> >> Until JDK-8148540 is addressed, the test >> >> sun/nio/ch/TestMaxCachedBufferSize.java >> >> should be problem listed on the platforms is fails on. Please review >> the patch below. >> > I think this test will be tricky to be reliable on 32-bit so the > simplest is to just restrict it to 64-bit with: > > @requires (sun.arch.data.model == "64") > > -Alan. > From martinrb at google.com Fri Jan 29 20:53:12 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jan 2016 12:53:12 -0800 Subject: RFR:JDK-8141452:Convert between TimeUnit and ChronoUnit In-Reply-To: References: <56A63E50.5030007@oracle.com> <56A64823.20200@oracle.com> <56A65328.1080207@Oracle.com> <56A8F19C.8090509@oracle.com> <56AB7CA3.7050801@Oracle.com> <56AB814C.7070409@oracle.com> Message-ID: On Fri, Jan 29, 2016 at 9:37 AM, Stephen Colebourne wrote: > In the ideal fix, all methods that take the combination (long, > TimeUnit) would be supplemented by an override that takes (Duration). > eg. Future.get(long, TimeUnit) has an additional get(Duration). > However, this is a lot of work, might be unpopular and would be slower > for j.u.concurrent use. Of course, having (long, TimeUnit) argument pairs is a performance hack, but there's not much demand for an improved API, and I still hope that someday we'll have proper value types in Java, at which time adding support for value type durations will be more attractive. So let's do nothing else for now, unless there's a more urgent compelling reason? From martinrb at google.com Fri Jan 29 21:49:44 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jan 2016 13:49:44 -0800 Subject: Please approve fix for failing test JDK-8148638 Message-ID: Sorry, I made a mistake in 8146467 and checked in a broken test. Please approve my temporary fix https://bugs.openjdk.java.net/browse/JDK-8148638 http://cr.openjdk.java.net/~martin/webrevs/openjdk9/CompletableFuture-temporary-revert/ From Roger.Riggs at Oracle.com Fri Jan 29 21:56:08 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 29 Jan 2016 16:56:08 -0500 Subject: Please approve fix for failing test JDK-8148638 In-Reply-To: References: Message-ID: <56ABDFF8.8040700@Oracle.com> +1 On 1/29/2016 4:49 PM, Martin Buchholz wrote: > Sorry, I made a mistake in 8146467 and checked in a broken test. > Please approve my temporary fix > https://bugs.openjdk.java.net/browse/JDK-8148638 > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/CompletableFuture-temporary-revert/ From martinrb at google.com Fri Jan 29 22:04:51 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 29 Jan 2016 14:04:51 -0800 Subject: Please approve fix for failing test JDK-8148638 In-Reply-To: <56ABDFF8.8040700@Oracle.com> References: <56ABDFF8.8040700@Oracle.com> Message-ID: Thanks! Pushed. On Fri, Jan 29, 2016 at 1:56 PM, Roger Riggs wrote: > +1 > > > On 1/29/2016 4:49 PM, Martin Buchholz wrote: >> >> Sorry, I made a mistake in 8146467 and checked in a broken test. >> Please approve my temporary fix >> https://bugs.openjdk.java.net/browse/JDK-8148638 >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/CompletableFuture-temporary-revert/ > > From lance.andersen at oracle.com Fri Jan 29 22:06:13 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 29 Jan 2016 17:06:13 -0500 Subject: RFR [9]: 8138699 Iproving JAX-B javadoc In-Reply-To: <56ABA421.5050605@oracle.com> References: <56ABA421.5050605@oracle.com> Message-ID: <85738C93-80A6-455E-A4E1-57B04A6E24AC@oracle.com> Hi Miran, Overall, looks ok. Please change JavaSE to Java SE. Has a CCC been approved for the javadoc changes as it will need to Best Lance On Jan 29, 2016, at 12:40 PM, Miroslav Kos wrote: > Hi everybody, > please review following patch - those are changes to javadoc, previously consulted with sqe. The javadoc now is not always clear, which makes difficult to develop proper tests. > > JIRA: https://bugs.openjdk.java.net/browse/JDK-8138699 > webrev: http://cr.openjdk.java.net/~mkos/8138699/jaxws.01/ > > Thanks > Miran > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From cushon at google.com Sat Jan 30 03:45:32 2016 From: cushon at google.com (Liam Miller-Cushon) Date: Fri, 29 Jan 2016 19:45:32 -0800 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: Message-ID: On Thu, Jan 28, 2016 at 11:50 AM, Joel Borggr?n-Franck < joel.borggren.franck at gmail.com> wrote: > Thanks for the update. I think it makes sense to expand testing > slightly, testing all three parsing clauses that you fixed, and for > all three of them also checking that a "thing" after the broken item > is parsed correctly. > Sure, I'd be happy to add additional tests if it looks like the other questions can be addressed. > But, the reason we didn't fix this the last two times we looked at it > (that I am aware of) is the compatibility story. In order to fix his > you need to figure out two things: > > - Is this is a spec change, that is, can we get away with throwing an > AnnotationFormatError where we would now do? Should we clarify the > spec? > Can you clarify which parts of the spec might need to be updated? I can't find anything relevant in the jls or jvms. The javadoc for AnnotatedElement mentions some conditions under which TypeNotPresentException is thrown: "If an annotation returned by a method in this interface contains (directly or indirectly) a Class-valued member referring to a class that is not accessible in this VM, attempting to read the class by calling the relevant Class-returning method on the returned annotation will result in a TypeNotPresentException." So throwing TypeNotPresentException when an array-valued annotation indirectly references an inaccessible class sounds like the right behaviour, and is consistent with how the implementation currently handles similar cases. - Since this is a behaviorally incompatible change, how big is the > impact? This is of course a hard question to answer, but if one could > do a corpus analysis over a large code base and look for catches of > ArrayStoreExceptions when reflecting over annotations, that could be > useful. If it turns out that "a lot" of users have adopted to this > bug, perhaps it isn't worth fixing? On the other hand I can imagine > that this is so uncommon that no one catches either type of error. > I'm working on evaluating the impact. A review of our code base showed that handling ArrayStoreException is fairly uncommon. Of the instances I found, none of them were in code that was inspecting annotations. The next step is to start using the patch internally and see if anything breaks. I'll update the thread when I have results on that. From joe.darcy at oracle.com Sat Jan 30 05:01:45 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 29 Jan 2016 21:01:45 -0800 Subject: JDK 9 RFR of JDK-8148653: Problem list B5086147.java on windows Message-ID: <56AC43B9.8030701@oracle.com> The test java/net/URL/B5086147.java has been failing on windows since the changesets for JDK-8147462 and JDK-8145104 were pushed. The test should be problem listed until the root cause of JDK-8148626 is found and fixed. Please review the patch to do this below. Thanks, -Joe diff -r 3bce90b8839e test/ProblemList.txt --- a/test/ProblemList.txt Fri Jan 29 17:03:17 2016 -0800 +++ b/test/ProblemList.txt Fri Jan 29 21:01:06 2016 -0800 @@ -180,6 +180,9 @@ # 7143960 java/net/DatagramSocket/SendDatagramToBadAddress.java macosx-all +# 8148626 +java/net/URL/B5086147.java windows-all + ############################################################################ # jdk_nio From Alan.Bateman at oracle.com Sat Jan 30 08:00:06 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sat, 30 Jan 2016 08:00:06 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> References: <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> Message-ID: <56AC6D86.7050101@oracle.com> On 29/01/2016 17:39, Paul Sandoz wrote: > : > Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. > > My assumption was the traversal should by default be consistent with a calls to getEntry, thus: > > jarFile.stream().forEach(e -> { > JarEntry je = jarFile.getJarEntry(e.getName()); > assert e.equals(je); > }); > > There might need to be another stream method that returns all entries. > Right, I'm mostly just wondering if entries()/streams() should override the entries in the stream with versioned entries and filter out the META-INF/versions/ tree. If I've gone to trouble of specifying the a Release then it seems the right thing to do. On the other hand, it comes at a cost and there will be use-cases like "get the names of all entries" that would be more efficient to just build on the current entries()/stream(). I'm loath to suggest this might need a new method but it might be one of the options to consider here. Minimally there is a javadoc to specify on how these methods behave when the JAR is multi-release and opened by specifying a release. -Alan From claes.redestad at oracle.com Sat Jan 30 14:30:23 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 30 Jan 2016 15:30:23 +0100 Subject: RFR: 8148626: URI.toURL needs to use protocol Handler to parse file URIs Message-ID: <56ACC8FF.6050405@oracle.com> Hi, it turns out trying to optimize URI to URL conversion for file URLs don't work out very well due to special treatment of backward slashes. Instead of trying to deal with such file URLs, this patch partially backs out 8147462 and only leave the optimization in place for the jrt protocol. This still helps reducing footprint in jigsaw while keeping things performance neutral for other kinds of URLs. Bug: https://bugs.openjdk.java.net/browse/JDK-8148626 Webrev: http://cr.openjdk.java.net/~redestad/8148626/webrev.01/ /Claes From joe.darcy at oracle.com Sat Jan 30 18:35:24 2016 From: joe.darcy at oracle.com (joe darcy) Date: Sat, 30 Jan 2016 10:35:24 -0800 Subject: RFR: 8148626: URI.toURL needs to use protocol Handler to parse file URIs In-Reply-To: <56ACC8FF.6050405@oracle.com> References: <56ACC8FF.6050405@oracle.com> Message-ID: <56AD026C.7030703@oracle.com> Hello, The change looks okay in that the new code is limited to the jrt protocol. I assume the failing test java/net/URL/B5086147.java passes again with this change. If a fix for this issue is not pushed soon, I want to have the failing test problem listed (http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038459.html). Thanks, -Joe On 1/30/2016 6:30 AM, Claes Redestad wrote: > Hi, > > it turns out trying to optimize URI to URL conversion for file URLs > don't work out very well due to special treatment of backward slashes. > > Instead of trying to deal with such file URLs, this patch partially > backs out 8147462 and only leave the optimization in place for the jrt > protocol. This still helps > reducing footprint in jigsaw while keeping things performance neutral > for other kinds of URLs. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8148626 > Webrev: http://cr.openjdk.java.net/~redestad/8148626/webrev.01/ > > /Claes From claes.redestad at oracle.com Sun Jan 31 00:58:35 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 31 Jan 2016 01:58:35 +0100 Subject: RFR: 8148626: URI.toURL needs to use protocol Handler to parse file URIs In-Reply-To: <56AD026C.7030703@oracle.com> References: <56ACC8FF.6050405@oracle.com> <56AD026C.7030703@oracle.com> Message-ID: <56AD5C3B.9030803@oracle.com> Hi, On 2016-01-30 19:35, joe darcy wrote: > Hello, > > The change looks okay in that the new code is limited to the jrt > protocol. I assume the failing test > > java/net/URL/B5086147.java > > passes again with this change. I had to go through some hoops to find out how to ensure this test actually gets picked up by the remote-testing incantation I've been using so far, but have verified the test has been run and passes on the affected platforms. > > If a fix for this issue is not pushed soon, I want to have the failing > test problem listed > (http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038459.html). I'm ready push this, but might be good to get a second opinion from net-dev to ensure they aren't caught unaware of this follow-up fix. Thanks! /Claes > > Thanks, > > -Joe > > On 1/30/2016 6:30 AM, Claes Redestad wrote: >> Hi, >> >> it turns out trying to optimize URI to URL conversion for file URLs >> don't work out very well due to special treatment of backward slashes. >> >> Instead of trying to deal with such file URLs, this patch partially >> backs out 8147462 and only leave the optimization in place for the >> jrt protocol. This still helps >> reducing footprint in jigsaw while keeping things performance neutral >> for other kinds of URLs. >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8148626 >> Webrev: http://cr.openjdk.java.net/~redestad/8148626/webrev.01/ >> >> /Claes > From kim.barrett at oracle.com Sun Jan 31 04:37:21 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Sat, 30 Jan 2016 23:37:21 -0500 Subject: RFR: 8072777: java/lang/ref/ReferenceEnqueuePending.java: some references aren't queued Message-ID: <9C4B1F92-29F6-4977-BB48-342ADF886B56@oracle.com> Please review this fix of a bug in the ReferenceEnqueuePending test. The original code assumes the pending list processor will stay ahead of the test thread, so that the test's polling of the reference queue won't find it empty while there are still references on the internal pending list. That might not always happen though, possibly in part because the test raises the priority of its own thread to better compete with the pending list processor. The fix is to use queue.remove with a large timeout, rather than queue.poll, to pull references from the queue until we've obtained all we expect to get. Then switch to polling to verify nothing unexpected in the queue without making the test delay for an expected timeout. If all of the references are (eventually) queued as expected then the timeout will never come close to expiring. I also looked at uses of poll() by other tests in the same directory, and didn't find any that looked like they might have a similar problem of failing because the pending list processor had not yet gotten around to dealing with a reference. CR: https://bugs.openjdk.java.net/browse/JDK-8072777 Webrev: http://cr.openjdk.java.net/~kbarrett/8072777/webrev.00/ Testing: Local testing of before and after test code, with an occasional sleep inserted into the pending list processor's loop to simulate a scheduling delay. With that delay, the original test frequently fails, while the modified test consistently passes. From peter.levart at gmail.com Sun Jan 31 09:19:18 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 31 Jan 2016 10:19:18 +0100 Subject: RFR: 8072777: java/lang/ref/ReferenceEnqueuePending.java: some references aren't queued In-Reply-To: <9C4B1F92-29F6-4977-BB48-342ADF886B56@oracle.com> References: <9C4B1F92-29F6-4977-BB48-342ADF886B56@oracle.com> Message-ID: <56ADD196.9040809@gmail.com> Hi Kim, This change will make it practically impossible to miss the enqueued references. But this could be an opportunity to also clean-up the code a bit. The checkResult method has an 'obj' parameter whose purpose is not immediately obvious. It's only used in the following line: 136 // Use the last Reference object of those created above, so as to keep it "alive". 137 System.out.println("I must write " + obj + " to prevent compiler optimizations."); ...the comment is misleading. If we look at how checkResult is called, we can see that it is not the last Reference object that is passed to this parameter, but its referent (an Integer object). This is the referent of the Reference that is not supposed to be enqueued as checkResult is called with parameters (refQueue, obj, iterations-1), which means that it is expected to retrieve one less Reference object than the number of Reference objects created. I don't know why such complications are necessary. Is the test supposed to also verify the requirement that a Reference whose referent is kept strongly reachable will not be enqueued? If that is true, then there are two things that should be fixed to ensure this requirement is actually tested: - the last Reference object that contains a referent that is kept strongly reachable should also be kept strongly reachable while the checkResult method is dequeuing References or else it can be GCed before actually being discovered. - the checkReference method should also wait for some time for the last Reference to be enqueued. It is not necessary to wait 30 seconds. 2 seconds should be enough to not miss this last Reference if it was enqueued as a result of some bug. As to the implementation, it is not necessary to pass the last Reference or it's referent to the checkReference method to keep them strongly reachable. checkReference method does not have to deal with that. The last Reference and it's referent could be kept strongly reachable in the following way: 102 // Verify that all WeakReference objects (but last one) ended up queued. 103 checkResult(refQueue, iterations-1); // Keep the last Reference object and its referent alive until after the ckeck Reference.reachabilityFence(obj); Reference.reachabilityFence(weaky); 104 System.out.println("Test passed."); Regards, Peter On 01/31/2016 05:37 AM, Kim Barrett wrote: > Please review this fix of a bug in the ReferenceEnqueuePending test. > The original code assumes the pending list processor will stay ahead > of the test thread, so that the test's polling of the reference queue > won't find it empty while there are still references on the internal > pending list. That might not always happen though, possibly in part > because the test raises the priority of its own thread to better > compete with the pending list processor. > > The fix is to use queue.remove with a large timeout, rather than > queue.poll, to pull references from the queue until we've obtained all > we expect to get. Then switch to polling to verify nothing unexpected > in the queue without making the test delay for an expected timeout. If > all of the references are (eventually) queued as expected then the > timeout will never come close to expiring. > > I also looked at uses of poll() by other tests in the same directory, > and didn't find any that looked like they might have a similar problem > of failing because the pending list processor had not yet gotten > around to dealing with a reference. > > CR: > https://bugs.openjdk.java.net/browse/JDK-8072777 > > Webrev: > http://cr.openjdk.java.net/~kbarrett/8072777/webrev.00/ > > Testing: > Local testing of before and after test code, with an occasional sleep > inserted into the pending list processor's loop to simulate a > scheduling delay. With that delay, the original test frequently fails, > while the modified test consistently passes. > > From chris.hegarty at oracle.com Sun Jan 31 16:13:35 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Sun, 31 Jan 2016 16:13:35 +0000 Subject: RFR: 8148626: URI.toURL needs to use protocol Handler to parse file URIs In-Reply-To: <56AD5C3B.9030803@oracle.com> References: <56ACC8FF.6050405@oracle.com> <56AD026C.7030703@oracle.com> <56AD5C3B.9030803@oracle.com> Message-ID: <5483AAB2-ECC8-4A99-A8DC-05815EEFC5F0@oracle.com> > On 31 Jan 2016, at 00:58, Claes Redestad wrote: > > Hi, > > On 2016-01-30 19:35, joe darcy wrote: >> Hello, >> >> The change looks okay in that the new code is limited to the jrt protocol. I assume the failing test >> >> java/net/URL/B5086147.java >> >> passes again with this change. > > I had to go through some hoops to find out how to ensure this test actually gets picked up by the remote-testing incantation I've been using so far, but have verified the test has been run and passes on the affected platforms. > >> >> If a fix for this issue is not pushed soon, I want to have the failing test problem listed (http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038459.html). > > I'm ready push this, but might be good to get a second opinion from net-dev to ensure they aren't caught unaware of this follow-up fix. This looks ok Claes. -Chris. > Thanks! > > /Claes >> >> Thanks, >> >> -Joe >> >> On 1/30/2016 6:30 AM, Claes Redestad wrote: >>> Hi, >>> >>> it turns out trying to optimize URI to URL conversion for file URLs don't work out very well due to special treatment of backward slashes. >>> >>> Instead of trying to deal with such file URLs, this patch partially backs out 8147462 and only leave the optimization in place for the jrt protocol. This still helps >>> reducing footprint in jigsaw while keeping things performance neutral for other kinds of URLs. >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8148626 >>> Webrev: http://cr.openjdk.java.net/~redestad/8148626/webrev.01/ >>> >>> /Claes >> > From claes.redestad at oracle.com Sun Jan 31 21:37:19 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Sun, 31 Jan 2016 22:37:19 +0100 Subject: RFR: 8148626: URI.toURL needs to use protocol Handler to parse file URIs In-Reply-To: <5483AAB2-ECC8-4A99-A8DC-05815EEFC5F0@oracle.com> References: <56ACC8FF.6050405@oracle.com> <56AD026C.7030703@oracle.com> <56AD5C3B.9030803@oracle.com> <5483AAB2-ECC8-4A99-A8DC-05815EEFC5F0@oracle.com> Message-ID: <56AE7E8F.4030407@oracle.com> Joe, Chris, thanks for reviewing. Pushed. /Claes On 2016-01-31 17:13, Chris Hegarty wrote: >> On 31 Jan 2016, at 00:58, Claes Redestad wrote: >> >> Hi, >> >> On 2016-01-30 19:35, joe darcy wrote: >>> Hello, >>> >>> The change looks okay in that the new code is limited to the jrt protocol. I assume the failing test >>> >>> java/net/URL/B5086147.java >>> >>> passes again with this change. >> I had to go through some hoops to find out how to ensure this test actually gets picked up by the remote-testing incantation I've been using so far, but have verified the test has been run and passes on the affected platforms. >> >>> If a fix for this issue is not pushed soon, I want to have the failing test problem listed (http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038459.html). >> I'm ready push this, but might be good to get a second opinion from net-dev to ensure they aren't caught unaware of this follow-up fix. > This looks ok Claes. > > -Chris. > >> Thanks! >> >> /Claes >>> Thanks, >>> >>> -Joe >>> >>> On 1/30/2016 6:30 AM, Claes Redestad wrote: >>>> Hi, >>>> >>>> it turns out trying to optimize URI to URL conversion for file URLs don't work out very well due to special treatment of backward slashes. >>>> >>>> Instead of trying to deal with such file URLs, this patch partially backs out 8147462 and only leave the optimization in place for the jrt protocol. This still helps >>>> reducing footprint in jigsaw while keeping things performance neutral for other kinds of URLs. >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8148626 >>>> Webrev: http://cr.openjdk.java.net/~redestad/8148626/webrev.01/ >>>> >>>> /Claes From tprintezis at twitter.com Fri Jan 29 21:35:39 2016 From: tprintezis at twitter.com (Tony Printezis) Date: Fri, 29 Jan 2016 16:35:39 -0500 Subject: JDK 9 RFR of JDK-8148627: Problem list TestMaxCachedBufferSize.java In-Reply-To: <56ABC7ED.60103@oracle.com> References: <56ABB545.7080600@oracle.com> <56ABC490.4060609@oracle.com> <56ABC7ED.60103@oracle.com> Message-ID: Thanks Joe! On January 29, 2016 at 3:13:37 PM, joe darcy (joe.darcy at oracle.com) wrote: Hi Alan, Limiting the test to 64-bit platforms is fine with me; I'll push the change as you've suggested later today. Thanks, -Joe On 1/29/2016 11:59 AM, Alan Bateman wrote: > > > On 29/01/2016 18:53, joe darcy wrote: >> Hello, >> >> Until JDK-8148540 is addressed, the test >> >> sun/nio/ch/TestMaxCachedBufferSize.java >> >> should be problem listed on the platforms is fails on. Please review >> the patch below. >> > I think this test will be tricky to be reliable on 32-bit so the > simplest is to just restrict it to 64-bit with: > > @requires (sun.arch.data.model == "64") > > -Alan. > ----- Tony Printezis | JVM/GC Engineer / VM Team | Twitter @TonyPrintezis tprintezis at twitter.com