From nadeesh.tv at oracle.com Mon Feb 1 06:18:32 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Mon, 01 Feb 2016 11:48:32 +0530 Subject: RFR:JDK-8146747:java.time.Duration.toNanos() and toMillis() exception on negative durations Message-ID: <56AEF8B8.4070703@oracle.com> Hi all, Please review following Bug Id : https://bugs.openjdk.java.net/browse/JDK-8146747 Solution: Handled the negative duration separately webrev : http://cr.openjdk.java.net/~ntv/8146747/webrev.00/ -- Thanks and Regards, Nadeesh TV From aleksey.shipilev at oracle.com Mon Feb 1 09:21:00 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 1 Feb 2016 12:21:00 +0300 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs Message-ID: <56AF237C.4040004@oracle.com> Hi, Please review this tiny update, that puts @since 9 tags over new Indify String Concat JDK APIs: http://cr.openjdk.java.net/~shade/8148730/webrev.01/ Cheers, -Aleksey From kubota.yuji at gmail.com Mon Feb 1 09:23:21 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Mon, 1 Feb 2016 18:23:21 +0900 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> <568D6553.4090108@oracle.com> Message-ID: Hi Sherman and all, Could you please let know your thought and the past case about AES? Thanks, Yuji 2016-01-08 0:01 GMT+09:00 KUBOTA Yuji : > Hi Sherman, > > Thank you for sharing! > > 2016-01-07 4:04 GMT+09:00 Xueming Shen : >> The reason that I'm not convinced that we really need a public interface of >> ZipCryption here >> is I don't know how useful/helpful/likely it would be going forward that >> someone might really >> use this interface to implement other encryption(s), especially the pkware >> proprietary one, >> I doubt it might be not that straightforward. > > In this proposal, we aim to support "traditional" because most people need it > in secure environment. BTW, could you please share the reason why you did > not support WinZip AES? Do you have a plan to support in the future? > > If you can share the reason, we want to decide the way of implementation with > consideration for your information. I think we can implement by two > way as below. > > 1. Implementing by reference to > http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java > This is good simply API. If we need to implement other encryption(s), > try to refactor it. > > 2. Implementing with a package private interface of ZipCryption for next step. > This has two problems as your advice. > > We agree with that the "encryption" and "compression" should be > separated logically. > However, current implementation compress the encrypted data, and buffering it. > It is too tightly-coupled, so we need refactoring to separate the > managing buffer > processing and the stream processing of InflaterInputStream / > DeflaterOutputStream. > > About "push back the bytes belong to next entry", we think > InflaterInputStream.originBuf > of our PoC do not required the needed info. Do this implements have problem? > > http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/src/java.base/share/classes/java/util/zip/InflaterInputStream.java.cdiff.html > > Thanks, > Yuji > >> In fact I did have a draft implementation that supports WinZip AES about 5-6 >> years ago :-) >> (which also supports compression methods bzip and lzma, btw) Here is the >> top class, It appears >> a general interface might not be that helpful and it might be ideal to >> simply implement it inside >> the JDK, as what is proposed here, when it's really desired. >> >> http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java >> >> It is a ZipFile based implementation, so it does not have the headache that >> ZipInputStream has, >> such as to push back the bytes belong to next entry, since the loc might not >> have the needed >> info regarding the size/csize in stream mode. >> >> From abstract point of view. The "encryption" and "compression" are >> different layers, it would >> be ideal to have them in separate classes logically, instead of mixing the >> encryption into >> compression. Sure, it might be convenient and probably may have better >> performance to mix >> them in certain use scenario, but the "encryption" should never appear in >> the public interface >> of those compression classes. Package private interface should be fine, if >> have to. >> >> -Sherman >> >> >>> >>> 2016-01-06 7:10 GMT+09:00 Xueming Shen : >>>> >>>> it appears that instead of adding "password" specific method to these >>>> classes directly, it might be more appropriate to extend the ZipEntry >>>> class >>>> for such "password" functionality. For example, with a pair of new >>>> methods >>>> >>>> boolean ZipEntry.isTraditionalEncryption(). >>>> void ZipEntry.setTraditionalEncryption(String password); >>> >>> Thanks advice, I agree. We should re-design the API to extend the >>> ZipEntry class. >>> >>>> The encryption support should/can be added naturally/smoothly with >>>> ZipFile.getInputStream(e), ZipInputstream and >>>> ZipOutputStream.putNextEntry(e), >>>> with no extra new method in these two classes. The implementation checks >>>> the flag (bit0, no bit 6) first and then verifies the password, as an >>>> implementation details. >>> >>> Agree. For this proposal, we aim to support only traditional >>> encryption. So I think we should also check bit 6. >>> >>>> For ZipFile and ZipInputStream, we can add note to the api doc to force >>>> the >>>> invoker to check if the returned ZipEntry indicates it's an encrypted >>>> entry. >>>> If yes, it must to set the appropriate password to the returned ZipEntry >>>> via >>>> ZipEntry.setTraditionalEncryption(password); before reading any byte from >>>> the input stream. >>> >>> Yes, we have to add note the flow of codes to the JavaDoc. >>> >>>> Again, we should not have any "encryption" related public field/method in >>>> DeflaterOutputStream/InflaterInputStream. Ideally these two classes >>>> really >>>> should not be aware of "encryption" at all. >>> >>> Agree, but I think we might be faced technical difficulty about a >>> processing between zlib and the internal buffer of InflaterInputStream >>> / DeflaterOutputStream. Please give us time to implement. >>> >>>> -Sherman >>> >>> Thanks, >>> Yuji >>> >>> >>>> On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: >>>>> >>>>> Hi Sherman and all, >>>>> >>>>> Happy new year to everyone! >>>>> >>>>> Please let know your feedback about this proposal. :-) >>>>> >>>>> Thanks, >>>>> Yuji >>>>> >>>>> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >>>>>> >>>>>> Hi Sherman, >>>>>> >>>>>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>>>>>> >>>>>>> It is no longer necessary to touch the native code (zip_util.c/h) >>>>>>> after >>>>>>> the >>>>>>> native ZipFile implementation has been moved up to the java level. >>>>>>> Those >>>>>>> native code are for vm access only now, which I dont think care about >>>>>>> the >>>>>>> password support at all. >>>>>> >>>>>> Thanks for your information. We do not take care the native. >>>>>> >>>>>> I discussed with Yasumasa, and our thought is as below. >>>>>> >>>>>>> (1) what's the benefit of exposing the public interface ZipCryption? >>>>>>> the >>>>>>> real >>>>>>> question is whether or not this interface is good enough for other >>>>>>> encryption >>>>>>> implementation to plugin their implementation to support the >>>>>>> ZipFile/Input/ >>>>>>> OutputStream to their encryption spec. >>>>>> >>>>>> We aimed that the public interface ZipCryption supports the >>>>>> extensibillity for other encrypt engine. The JDK core libs developers >>>>>> have to implementation ZipyCryption only. If not provide, the JDK >>>>>> developers must implement ZipStream/Entry by JDK API to design the >>>>>> data structure of entry. >>>>>> If you want to use binary key data such as PKI, you can implement new >>>>>> encrypt/decrypt engine by ZipCryption interface. >>>>>> So we think we should provide this interface to be clearly how to >>>>>> implement a new engine, e.g., cipher algorithm, cipher strength and >>>>>> converting the header, etc. >>>>>> >>>>>>> (2) it seems like it might be possible to hide most of the >>>>>>> implementation >>>>>>> and only expose the "String password" (instead of the ZipCryption) as >>>>>>> the >>>>>>> public interface to support the "traditional" encryption. This depends >>>>>>> on the >>>>>>> result of (1) though. >>>>>> >>>>>> Thanks for your clues. We think the string password at first. However, >>>>>> we should also create a new binary interface given we support PKI in >>>>>> the future. >>>>>> >>>>>>> (3) I'm concerned of pushing ZipCryption into >>>>>>> InflaterInputStream/DeflaterOutputStream. >>>>>>> It might be worth considering to replace the ZipCryption >>>>>>> implementation >>>>>>> with >>>>>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>>>>> use >>>>>>> the FilterOutputStream for the ZipOutputStream and the >>>>>>> FilterInputStream >>>>>>> for the >>>>>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue >>>>>>> ... >>>>>> >>>>>> Thanks for your clues, too. Honestly speaking, we think the current >>>>>> zip implementation may break the data when used PushbackInputStream >>>>>> for the following reasons. >>>>>> >>>>>> * PushbackInputStream uses an unique internal buffer for re-read >>>>>> operation. >>>>>> * But, InflaterInputStream provide date to Inflater per reads and >>>>>> buffer by JNI (zlib). >>>>>> * So we think PushbackInputStream is poor compatibility with >>>>>> InflaterInputStream. >>>>>> >>>>>> We generally use InputStream through ZipEntry#getInputStream(). We do >>>>>> not touch FileInputStream for reading ZIP data. If we call unread() >>>>>> when we use PushbackInputStream as reading ZIP archive, we guess that >>>>>> it will break the reading data. >>>>>> So, our approach do not affect the PushbackInputStream. >>>>>> What do you think about this? >>>>>> >>>>>>> (4) It seems the ZipOutputStream only supports the "stream based" >>>>>>> password, while >>>>>>> the ZipInputStream supports the "entry based" password. Do we really >>>>>>> need >>>>>>> "entry based" support here? >>>>>> >>>>>> As your suggestion, we should support "entry based". We will start to >>>>>> implement "entry based" after this discussion is closed. >>>>>> >>>>>> Thanks, >>>>>> Yuji >>>>>> >>>>>>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>>>>>> >>>>>>>> Hi Jason, >>>>>>>> >>>>>>>> Thank you for your comment. >>>>>>>> I've fixed it in new webrev: >>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>>>>> >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> Yasumasa >>>>>>>> >>>>>>>> >>>>>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>>>>>> >>>>>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is >>>>>>>>> in >>>>>>>>> conflict with line 350 and 348. >>>>>>>>> >>>>>>>>> ________________________________________ >>>>>>>>> From: core-libs-dev on >>>>>>>>> behalf >>>>>>>>> of >>>>>>>>> Yasumasa Suenaga >>>>>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>>>>> protection to Zip entries >>>>>>>>> >>>>>>>>> Hi Sergey, >>>>>>>>> >>>>>>>>> Thank you for your comment. >>>>>>>>> >>>>>>>>> I added that description in new webrev: >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>>>>>> >>>>>>>>>> Should the new methods describe how they will work in case of null >>>>>>>>>> params? >>>>>>>>>> >>>>>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>>>>>> >>>>>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>>>>> >>>>>>>>>>> Could you review it? >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi Sherman, >>>>>>>>>>>> >>>>>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>>>>> jdk9/dev >>>>>>>>>>>> repos. >>>>>>>>>>>> >>>>>>>>>>>> Do you have any comments about current webrev? >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>> >>>>>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Sherman, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for your quick response :) >>>>>>>>>>>>> >>>>>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>>>>> below >>>>>>>>>>>>> reasons. >>>>>>>>>>>>> >>>>>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>>>>> like a >>>>>>>>>>>>> temporary file. >>>>>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>>>>> best >>>>>>>>>>>>> for >>>>>>>>>>>>> openjdk. >>>>>>>>>>>>> * PKWare claims that they have patents about the >>>>>>>>>>>>> "stronger" >>>>>>>>>>>>> on >>>>>>>>>>>>> Zip[1]. >>>>>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>>>>> "stronger" [2][3]. >>>>>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>>>>> interface >>>>>>>>>>>>> to >>>>>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>>>>> >>>>>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>>>>> easily >>>>>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>>>>> >>>>>>>>>>>>> [1] >>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption >>>>>>>>>>>>> Specification) >>>>>>>>>>>>> >>>>>>>>>>>>> [2] >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>>>>> >>>>>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> Yuji >>>>>>>>>>>>> >>>>>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming >>>>>>>>>>>>> Shen: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Yuji, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>>>>> bring >>>>>>>>>>>>>> in the >>>>>>>>>>>>>> security guy >>>>>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>>>>> in >>>>>>>>>>>>>> the >>>>>>>>>>>>>> "traditional PKWare >>>>>>>>>>>>>> decryption", which is, based on the wiki, "known to be >>>>>>>>>>>>>> seriously >>>>>>>>>>>>>> flawed, and >>>>>>>>>>>>>> in particular >>>>>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>>>>> support >>>>>>>>>>>>>> "stronger" encryption >>>>>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> Sherman >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>>>>> before >>>>>>>>>>>>>>>> get >>>>>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>>>>> However, >>>>>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we >>>>>>>>>>>>>>>> must >>>>>>>>>>>>>>>> use >>>>>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has >>>>>>>>>>>>>>>> been >>>>>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>>>>> zlib >>>>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Test code is as below. This code will let you know >>>>>>>>>>>>>>>> how >>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>> PoC >>>>>>>>>>>>>>>> works. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are >>>>>>>>>>>>>>>> providing >>>>>>>>>>>>>>>> many >>>>>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>>>>> implement to >>>>>>>>>>>>>>>> handle password-protected zip file. I guess that this >>>>>>>>>>>>>>>> proposal >>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> [1]: >>>>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >> From daniel.fuchs at oracle.com Mon Feb 1 09:24:31 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 1 Feb 2016 10:24:31 +0100 Subject: RFR 9: 8146773 java/lang/ref/CleanerTest.java CleanerTest.testRefSubtypes() fails In-Reply-To: <56ABB3AD.2030401@Oracle.com> References: <56ABB3AD.2030401@Oracle.com> Message-ID: <56AF244F.6060705@oracle.com> Hi Roger, Looks good to me - but I'd suggest to call fullGC at least once before waiting on the semaphore (e.g. add a call to whitebox.fullGC() when entering checkCleaned() before line 285). That might maximize the chances that the referred object will be GC'ed before you start waiting on the semaphore. Also I wonder whether you should raise the timeout in tryAcquire, 10ms is not much - and you want to give enough time so that the cleaner's thread is scheduled and calls the cleanup action... Maybe you could make that timeout depending on the timeoutFactor as well. best regards, -- daniel On 29/01/16 19:47, Roger Riggs wrote: > Please review a fix for two test issues. When run with -Xcomp and other > switches that change > GC behavior; the CleanerTest was not giving enough to time to see the > result of cleaning. > Also, added an adjustment of the number of cycles by the timeoutFactor. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-cleanertest-8146773/ > > 8146773: java/lang/ref/CleanerTest.java CleanerTest.testRefSubtypes() fails > 8148352: CleanerTest fails: Cleanable should have been freed > > Thanks, Roger > > From andrej.golovnin at gmail.com Mon Feb 1 09:32:48 2016 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Mon, 1 Feb 2016 10:32:48 +0100 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: <56AF237C.4040004@oracle.com> References: <56AF237C.4040004@oracle.com> Message-ID: Hi Aleksej, is it really needed to add the "@since 9" tag to the methods/constructors when the whole class is since JDK 9? As far as I know the @since tag should be only added to class members introduced in a later version than the class. Best regards, Andrej Golovnin On Mon, Feb 1, 2016 at 10:21 AM, Aleksey Shipilev wrote: > Hi, > > Please review this tiny update, that puts @since 9 tags over new Indify > String Concat JDK APIs: > http://cr.openjdk.java.net/~shade/8148730/webrev.01/ > > Cheers, > -Aleksey > From aleksey.shipilev at oracle.com Mon Feb 1 09:35:11 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 1 Feb 2016 12:35:11 +0300 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: References: <56AF237C.4040004@oracle.com> Message-ID: <56AF26CF.1040304@oracle.com> Hi, I don't know, you tell me -- you have the tools that enforce @since-ness! Certainly, we can put @since on classes only, if that is acceptable. -Aleksey On 02/01/2016 12:32 PM, Andrej Golovnin wrote: > Hi Aleksej, > > is it really needed to add the "@since 9" tag to the > methods/constructors when the whole class is since JDK 9? As far as I > know the @since tag should be only added to class members introduced > in a later version than the class. > > Best regards, > Andrej Golovnin > > On Mon, Feb 1, 2016 at 10:21 AM, Aleksey Shipilev > wrote: >> Hi, >> >> Please review this tiny update, that puts @since 9 tags over new Indify >> String Concat JDK APIs: >> http://cr.openjdk.java.net/~shade/8148730/webrev.01/ >> >> Cheers, >> -Aleksey >> From Alan.Bateman at oracle.com Mon Feb 1 09:42:21 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 Feb 2016 09:42:21 +0000 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: <56AF26CF.1040304@oracle.com> References: <56AF237C.4040004@oracle.com> <56AF26CF.1040304@oracle.com> Message-ID: <56AF287D.9090605@oracle.com> On 01/02/2016 09:35, Aleksey Shipilev wrote: > Hi, > > I don't know, you tell me -- you have the tools that enforce > @since-ness! Certainly, we can put @since on classes only, if that is > acceptable. > > -Aleksey > @since 9 on the class should be sufficient here. If it becomes necessary to add methods in Java SE 10 then the new methods will get @since. -Alan From andrej.golovnin at gmail.com Mon Feb 1 09:44:27 2016 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Mon, 1 Feb 2016 10:44:27 +0100 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: <56AF26CF.1040304@oracle.com> References: <56AF237C.4040004@oracle.com> <56AF26CF.1040304@oracle.com> Message-ID: Hi Aleksej, > I don't know, you tell me -- you have the tools that enforce > @since-ness! Certainly, we can put @since on classes only, if that is > acceptable. I don't have such tools. But when I write JavaDocs I try to follow the recommendations from this article: http://www.oracle.com/technetwork/java/javase/documentation/index-137868.html And here is the excerpt from the article regarding the @since tag: When a class (or interface) is introduced, specify one @since tag in its class description and no @since tags in the members. Add an @since tag only to members added in a later version than the class. This minimizes the number of @since tags. Best regards, Andrej Golovnin From aleksey.shipilev at oracle.com Mon Feb 1 09:56:23 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 1 Feb 2016 12:56:23 +0300 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: References: <56AF237C.4040004@oracle.com> <56AF26CF.1040304@oracle.com> Message-ID: <56AF2BC7.5030004@oracle.com> On 02/01/2016 12:44 PM, Andrej Golovnin wrote: >> I don't know, you tell me -- you have the tools that enforce >> @since-ness! Certainly, we can put @since on classes only, if that is >> acceptable. > > I don't have such tools. Apologies, I confused you with another Andrey :) Sure, let's do class-only: http://cr.openjdk.java.net/~shade/8148730/webrev.02/ Cheers, -Aleksey From Alan.Bateman at oracle.com Mon Feb 1 10:03:58 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 1 Feb 2016 10:03:58 +0000 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: <56AF2BC7.5030004@oracle.com> References: <56AF237C.4040004@oracle.com> <56AF26CF.1040304@oracle.com> <56AF2BC7.5030004@oracle.com> Message-ID: <56AF2D8E.1070909@oracle.com> On 01/02/2016 09:56, Aleksey Shipilev wrote: > : > > Sure, let's do class-only: > http://cr.openjdk.java.net/~shade/8148730/webrev.02/ > Looks good. From andrej.golovnin at gmail.com Mon Feb 1 10:04:23 2016 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Mon, 1 Feb 2016 11:04:23 +0100 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: <56AF2BC7.5030004@oracle.com> References: <56AF237C.4040004@oracle.com> <56AF26CF.1040304@oracle.com> <56AF2BC7.5030004@oracle.com> Message-ID: > Apologies, I confused you with another Andrey :) > > Sure, let's do class-only: > http://cr.openjdk.java.net/~shade/8148730/webrev.02/ No problem, Aleksey. Looks much better now. :) Best regards, Andrej Golovnin From aleksey.shipilev at oracle.com Mon Feb 1 12:12:44 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 1 Feb 2016 15:12:44 +0300 Subject: RFR (XS) 8148730: Add @since tags in new String concat APIs In-Reply-To: <56AF2D8E.1070909@oracle.com> References: <56AF237C.4040004@oracle.com> <56AF26CF.1040304@oracle.com> <56AF2BC7.5030004@oracle.com> <56AF2D8E.1070909@oracle.com> Message-ID: <56AF4BBC.7090407@oracle.com> On 02/01/2016 01:03 PM, Alan Bateman wrote: > On 01/02/2016 09:56, Aleksey Shipilev wrote: >> : >> >> Sure, let's do class-only: >> http://cr.openjdk.java.net/~shade/8148730/webrev.02/ >> > Looks good. Thanks guys, pushed. -Aleksey From Roger.Riggs at Oracle.com Mon Feb 1 14:47:02 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 09:47:02 -0500 Subject: RFR 9: 8146773 java/lang/ref/CleanerTest.java CleanerTest.testRefSubtypes() fails In-Reply-To: <56AF244F.6060705@oracle.com> References: <56ABB3AD.2030401@Oracle.com> <56AF244F.6060705@oracle.com> Message-ID: <56AF6FE6.6080604@Oracle.com> Hi Daniel, On 2/1/2016 4:24 AM, Daniel Fuchs wrote: > Hi Roger, > > Looks good to me - but I'd suggest to call fullGC at least once > before waiting on the semaphore (e.g. add a call to whitebox.fullGC() > when entering checkCleaned() before line 285). > That might maximize the chances that the referred object will be > GC'ed before you start waiting on the semaphore. Sure, I moved the GC to the beginning of the loop > > Also I wonder whether you should raise the timeout in tryAcquire, 10ms > is not much - and you want to give enough time so that the cleaner's > thread is scheduled and calls the cleanup action... > Maybe you could make that timeout depending on the timeoutFactor > as well. The number of cycles was scaled by the timeoutFactor but it is more intuitive to scale the timeout itself. checkCleaned is invoked for cases where the cleanup function will not be called so increasing the timeout will just increase the running time of the test. The refactoring was done to allow more time in cases where the cleaning is expected and not wait too long in cases where it is not expected. But the overall running time of the test is < 10s. Webrev updated in place. Thanks, Roger > > best regards, > > -- daniel > > On 29/01/16 19:47, Roger Riggs wrote: >> Please review a fix for two test issues. When run with -Xcomp and other >> switches that change >> GC behavior; the CleanerTest was not giving enough to time to see the >> result of cleaning. >> Also, added an adjustment of the number of cycles by the timeoutFactor. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-cleanertest-8146773/ >> >> 8146773: java/lang/ref/CleanerTest.java CleanerTest.testRefSubtypes() >> fails >> 8148352: CleanerTest fails: Cleanable should have been freed >> >> Thanks, Roger >> >> > From daniel.fuchs at oracle.com Mon Feb 1 15:09:43 2016 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 1 Feb 2016 16:09:43 +0100 Subject: RFR 9: 8146773 java/lang/ref/CleanerTest.java CleanerTest.testRefSubtypes() fails In-Reply-To: <56AF6FE6.6080604@Oracle.com> References: <56ABB3AD.2030401@Oracle.com> <56AF244F.6060705@oracle.com> <56AF6FE6.6080604@Oracle.com> Message-ID: <56AF7537.9010301@oracle.com> On 01/02/16 15:47, Roger Riggs wrote: > The number of cycles was scaled by the timeoutFactor but it is more > intuitive to > scale the timeout itself. > > checkCleaned is invoked for cases where the cleanup function will not be > called > so increasing the timeout will just increase the running time of the test. > The refactoring was done to allow more time in cases where the cleaning > is expected > and not wait too long in cases where it is not expected. But the > overall running time > of the test is < 10s. > > Webrev updated in place. Hi Roger, Looks good to me. -- daniel From Roger.Riggs at Oracle.com Mon Feb 1 16:02:36 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 11:02:36 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals Message-ID: <56AF819C.5020905@Oracle.com> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. This JEP 260 motivated alternative to sun.misc.Signal supports the use case for interactive applications that need to handle Control-C and other signals. The new java.util.Signal class provides a settable primary signal handler and a default signal handler. The primary signal handler can be unregistered and handling is restored to the default signal handler. System initialization registers default signal handlers to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires a permission if a SecurityManager is set. The sun.misc.Signal implementation is modified to be layered on a common thread and dispatch mechanism. The VM handling of native signals is not affected. The command option to reduce signal use by the runtime with -Xrs is unmodified. The changes to hotspot are minimal to rename the hardcoded callback to the Java Signal dispatcher. Please review and comment on the API and implementation. javadoc: http://cr.openjdk.java.net/~rriggs/signal-doc/ Webrev: jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ Issue: https://bugs.openjdk.java.net/browse/JDK-8087286 JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 Thanks, Roger From Roger.Riggs at Oracle.com Mon Feb 1 16:41:18 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 11:41:18 -0500 Subject: RFR 9: 8143016 : java/time/test/java/time/TestClock_System.java failed intermittently Message-ID: <56AF8AAE.9020000@Oracle.com> Please review a test improvement to address an intermittent test failure in the java.time System Clock test for microsecond support. webrev: http://cr.openjdk.java.net/~rriggs/webrev-micros-8143016/ Issue: https://bugs.openjdk.java.net/browse/JDK-8143016 Thanks, Roger From lance.andersen at oracle.com Mon Feb 1 16:43:40 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 1 Feb 2016 11:43:40 -0500 Subject: RFR 9: 8143016 : java/time/test/java/time/TestClock_System.java failed intermittently In-Reply-To: <56AF8AAE.9020000@Oracle.com> References: <56AF8AAE.9020000@Oracle.com> Message-ID: <36C64B86-D23F-4BBB-8A1A-A63A51F1B87E@oracle.com> Looks OK Roger. On Feb 1, 2016, at 11:41 AM, Roger Riggs wrote: > Please review a test improvement to address an intermittent test failure in > the java.time System Clock test for microsecond support. > > webrev: > http://cr.openjdk.java.net/~rriggs/webrev-micros-8143016/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8143016 > > Thanks, Roger > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From chris.hegarty at oracle.com Mon Feb 1 16:47:43 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 1 Feb 2016 16:47:43 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF819C.5020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> Message-ID: <56AF8C2F.2010003@oracle.com> Thank you for taking this on Roger. An initial question to help my understanding. [ I think I need to understand this, before I can comment further on the API ] I'm a little confused about how the default handler is supposed to work, so I looked at the implementation and become even more confused. The consumer is registered on a per instance basis, and the instance is added to the static map when it is created. So it appears that the registered handler is not dependent on when the last register() is called, but when the last instance of a Signal for a specific signal is created. Is this intended? -Chris. On 01/02/16 16:02, Roger Riggs wrote: > Please review an API addition to handle signals such as SIGINT, SIGHUP, > and SIGTERM. > This JEP 260 motivated alternative to sun.misc.Signal supports the use > case for > interactive applications that need to handle Control-C and other signals. > > The new java.util.Signal class provides a settable primary signal > handler and a default > signal handler. The primary signal handler can be unregistered and > handling is restored > to the default signal handler. System initialization registers default > signal handlers > to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API > requires > a permission if a SecurityManager is set. > > The sun.misc.Signal implementation is modified to be layered on a common > thread and dispatch mechanism. The VM handling of native signals is not > affected. > The command option to reduce signal use by the runtime with -Xrs is > unmodified. > > The changes to hotspot are minimal to rename the hardcoded callback to > the Java > Signal dispatcher. > > Please review and comment on the API and implementation. > > javadoc: > http://cr.openjdk.java.net/~rriggs/signal-doc/ > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8087286 > > JEP 260: > https://bugs.openjdk.java.net/browse/JDK-8132928 > > Thanks, Roger > > From scolebourne at joda.org Mon Feb 1 17:15:05 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 1 Feb 2016 17:15:05 +0000 Subject: RFR 9: 8143016 : java/time/test/java/time/TestClock_System.java failed intermittently In-Reply-To: <56AF8AAE.9020000@Oracle.com> References: <56AF8AAE.9020000@Oracle.com> Message-ID: Fine by me Stephen On 1 February 2016 at 16:41, Roger Riggs wrote: > Please review a test improvement to address an intermittent test failure in > the java.time System Clock test for microsecond support. > > webrev: > http://cr.openjdk.java.net/~rriggs/webrev-micros-8143016/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8143016 > > Thanks, Roger > From steve.drach at oracle.com Mon Feb 1 17:58:57 2016 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 1 Feb 2016 09:58:57 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AC6D86.7050101@oracle.com> References: <562EFF8F.6070501@oracle.com> <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> Message-ID: <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> > On Jan 30, 2016, at 12:00 AM, Alan Bateman wrote: > > > On 29/01/2016 17:39, Paul Sandoz wrote: >> : >> Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. >> >> My assumption was the traversal should by default be consistent with a calls to getEntry, thus: >> >> jarFile.stream().forEach(e -> { >> JarEntry je = jarFile.getJarEntry(e.getName()); >> assert e.equals(je); >> }); >> >> There might need to be another stream method that returns all entries. >> > Right, I'm mostly just wondering if entries()/streams() should override the entries in the stream with versioned entries and filter out the META-INF/versions/ tree. I don?t think so. That kind of behavior might be difficult to understand. Returning all the entries provides some flexibility. One can write code like this: jarfile.stream().map(JarEntry::getName).filter(s -> !s.startsWith(?META-INF?)).map(JarFile::getJarEntry).etc to get the versioned results for any version you specify when constructing the JarFile. > > If I've gone to trouble of specifying the a Release then it seems the right thing to do. On the other hand, it comes at a cost and there will be use-cases like "get the names of all entries" that would be more efficient to just build on the current entries()/stream(). I'm loath to suggest this might need a new method but it might be one of the options to consider here. Minimally there is a javadoc to specify on how these methods behave when the JAR is multi-release and opened by specifying a release. How?s this? diff -r 68867430065b src/java.base/share/classes/java/util/jar/JarFile.java --- a/src/java.base/share/classes/java/util/jar/JarFile.java Fri Jan 29 12:34:44 2016 -0800 +++ b/src/java.base/share/classes/java/util/jar/JarFile.java Mon Feb 01 09:48:05 2016 -0800 @@ -576,9 +576,11 @@ } /** - * Returns an enumeration of the jar file entries. + * Returns an enumeration of all the jar file entries. Constructing this + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} + * constructor does not modify the behavior of this method. * - * @return an enumeration of the jar file entries + * @return an enumeration of the all jar file entries * @throws IllegalStateException * may be thrown if the jar file has been closed */ @@ -587,11 +589,13 @@ } /** - * Returns an ordered {@code Stream} over the jar file entries. + * Returns an ordered {@code Stream} over all the jar file entries. * Entries appear in the {@code Stream} in the order they appear in - * the central directory of the jar file. + * the central directory of the jar file. Constructing this + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} + * constructor does not modify the behavior of this method. * - * @return an ordered {@code Stream} of entries in this jar file + * @return an ordered {@code Stream} of all entries in this jar file * @throws IllegalStateException if the jar file has been closed * @since 1.8 */ From martinrb at google.com Mon Feb 1 18:45:23 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 1 Feb 2016 10:45:23 -0800 Subject: RFR: jsr166 jdk9 integration wave 4 Message-ID: After much debate on what to do when CompleteableFuture.whenComplete encounters an exception in both the source and the action, we chose what was acceptable to the most people - add the action's exception to the source exception as a suppressed exception. And added usage guidelines. And gave handle "top billing" over whenComplete. http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ and the usual miscellaneous improvements. From gerard.ziemski at oracle.com Mon Feb 1 18:58:25 2016 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Mon, 1 Feb 2016 12:58:25 -0600 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF819C.5020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> Message-ID: <3C91D205-04E9-43E2-A527-2051FFB5C24E@oracle.com> hi Roger, I love that we are adding this Signal object. I have several questions, but please do not take them as criticism, I?m just seeking clarification and providing feedback: #1 Re: "Consumers for signals that are unknown or reserved by the Java implementation can not be registered.? - Why don't we want to allow unknown signals? This will make us have to update our implementation if we want to support new or platform specific signals in the future. #2 Re: "java.util.Signal.raise()? - That API raises the signal in the current process, but what about sending a signal to another process for interprocess communication? #3 Re: "Signal.of("SIGINT?)? - Is this a factory method that returns the same object if called more than once? #4 Re: "public boolean unregister(Consumer consumer)? - Why is this API returning a value? Wouldn?t having a Signal API like ?public Consumer getConsumer()? be more flexible? #5 Re: "public void registerDefault(Consumer consumer)? - Do we really need this API? Can?t the same be achieved with the plain vanilla ?public void register(Consumer consumer)? I guess I don?t really see what makes this API special. cheers > On Feb 1, 2016, at 10:02 AM, Roger Riggs wrote: > > Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. > This JEP 260 motivated alternative to sun.misc.Signal supports the use case for > interactive applications that need to handle Control-C and other signals. > > The new java.util.Signal class provides a settable primary signal handler and a default > signal handler. The primary signal handler can be unregistered and handling is restored > to the default signal handler. System initialization registers default signal handlers > to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires > a permission if a SecurityManager is set. > > The sun.misc.Signal implementation is modified to be layered on a common > thread and dispatch mechanism. The VM handling of native signals is not affected. > The command option to reduce signal use by the runtime with -Xrs is unmodified. > > The changes to hotspot are minimal to rename the hardcoded callback to the Java > Signal dispatcher. > > Please review and comment on the API and implementation. > > javadoc: > http://cr.openjdk.java.net/~rriggs/signal-doc/ > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8087286 > > JEP 260: > https://bugs.openjdk.java.net/browse/JDK-8132928 > > Thanks, Roger > > From Roger.Riggs at Oracle.com Mon Feb 1 19:10:56 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 14:10:56 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF8C2F.2010003@oracle.com> References: <56AF819C.5020905@Oracle.com> <56AF8C2F.2010003@oracle.com> Message-ID: <56AFADC0.50305@Oracle.com> Hi Chris, On 2/1/2016 11:47 AM, Chris Hegarty wrote: > Thank you for taking this on Roger. > > An initial question to help my understanding. [ I think I need to > understand this, before I can comment further on the API ] > > I'm a little confused about how the default handler is supposed to > work, so I looked at the implementation and become even more confused. > The consumer is registered on a per instance basis, and the instance > is added to the static map when it is created. So it appears that the > registered handler is not dependent on when the last register() is > called, but when the last instance of a Signal for a specific signal > is created. Is this intended? The Signal instance delegates to a singleton SignalImpl instance (based on the name/number). I chose to delegate from Signal to SignalImpl to be able to put SignalImpl in a separate package that could be exported to the implementation of sun.misc.Signal as needed. Otherwise, java.util.Signal would only be able to export only its public interface and s.m.Signal needs more functions to be available. The current and default consumers are stored in the singleton SignalImpl objects that are used for the dispatch and calling of the consumer. The SignalImpl instances are create on first use. Each call to register re-registers the SignalImpl with the VM to ensure it will be the current handler. Roger > > -Chris. > > On 01/02/16 16:02, Roger Riggs wrote: >> Please review an API addition to handle signals such as SIGINT, SIGHUP, >> and SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the use >> case for >> interactive applications that need to handle Control-C and other >> signals. >> >> The new java.util.Signal class provides a settable primary signal >> handler and a default >> signal handler. The primary signal handler can be unregistered and >> handling is restored >> to the default signal handler. System initialization registers default >> signal handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >> requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is not >> affected. >> The command option to reduce signal use by the runtime with -Xrs is >> unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback to >> the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> From xueming.shen at oracle.com Mon Feb 1 19:18:06 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 01 Feb 2016 11:18:06 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> References: <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> Message-ID: <56AFAF6E.4060806@oracle.com> On 02/01/2016 09:58 AM, Steve Drach wrote: >> On Jan 30, 2016, at 12:00 AM, Alan Bateman wrote: >> >> >> On 29/01/2016 17:39, Paul Sandoz wrote: >>> : >>> Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. >>> >>> My assumption was the traversal should by default be consistent with a calls to getEntry, thus: >>> >>> jarFile.stream().forEach(e -> { >>> JarEntry je = jarFile.getJarEntry(e.getName()); >>> assert e.equals(je); >>> }); >>> >>> There might need to be another stream method that returns all entries. >>> >> Right, I'm mostly just wondering if entries()/streams() should override the entries in the stream with versioned entries and filter out the META-INF/versions/ tree. > I don?t think so. That kind of behavior might be difficult to understand. Returning all the entries provides some flexibility. One can write code like this: > > jarfile.stream().map(JarEntry::getName).filter(s -> !s.startsWith(?META-INF?)).map(JarFile::getJarEntry).etc > > to get the versioned results for any version you specify when constructing the JarFile. The current specification treats those class files under meta-inf/releases like kind of "metadata" of those base entries. Ideally those files should not even be individual "files", but part of their corresponding entries. The consumer of the MR-Jar should not need to be aware of these version-ed entries at all to use this MR-jar file to load classes/resources. From this point of view, these entries probably should be "invisible" from entries()/stream(), when the jar file is opened with "version-enabled". And all returned entries should have all their "data" (size, csize, timestamps, crc ...) pointed to the corresponding version-ed entries, withe the only exception is the "name". On the other hand it might be desired to keep JarFile.entries()/stream() asis to match their "zip file" perspective, to return "all" raw entries. Then it might also be desired to have an alternative "versioned streamVersion()" ... something like public Stream stream(Release r); ? -sherman >> If I've gone to trouble of specifying the a Release then it seems the right thing to do. On the other hand, it comes at a cost and there will be use-cases like "get the names of all entries" that would be more efficient to just build on the current entries()/stream(). I'm loath to suggest this might need a new method but it might be one of the options to consider here. Minimally there is a javadoc to specify on how these methods behave when the JAR is multi-release and opened by specifying a release. > How?s this? > > diff -r 68867430065b src/java.base/share/classes/java/util/jar/JarFile.java > --- a/src/java.base/share/classes/java/util/jar/JarFile.java Fri Jan 29 12:34:44 2016 -0800 > +++ b/src/java.base/share/classes/java/util/jar/JarFile.java Mon Feb 01 09:48:05 2016 -0800 > @@ -576,9 +576,11 @@ > } > > /** > - * Returns an enumeration of the jar file entries. > + * Returns an enumeration of all the jar file entries. Constructing this > + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} > + * constructor does not modify the behavior of this method. > * > - * @return an enumeration of the jar file entries > + * @return an enumeration of the all jar file entries > * @throws IllegalStateException > * may be thrown if the jar file has been closed > */ > @@ -587,11 +589,13 @@ > } > > /** > - * Returns an ordered {@code Stream} over the jar file entries. > + * Returns an ordered {@code Stream} over all the jar file entries. > * Entries appear in the {@code Stream} in the order they appear in > - * the central directory of the jar file. > + * the central directory of the jar file. Constructing this > + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} > + * constructor does not modify the behavior of this method. > * > - * @return an ordered {@code Stream} of entries in this jar file > + * @return an ordered {@code Stream} of all entries in this jar file > * @throws IllegalStateException if the jar file has been closed > * @since 1.8 > */ > From Roger.Riggs at Oracle.com Mon Feb 1 19:25:04 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 14:25:04 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <3C91D205-04E9-43E2-A527-2051FFB5C24E@oracle.com> References: <56AF819C.5020905@Oracle.com> <3C91D205-04E9-43E2-A527-2051FFB5C24E@oracle.com> Message-ID: <56AFB110.3070902@Oracle.com> Hi Gerard, On 2/1/2016 1:58 PM, Gerard Ziemski wrote: > hi Roger, > > I love that we are adding this Signal object. I have several questions, but please do not take them as criticism, I?m just seeking clarification and providing feedback: > > > #1 Re: "Consumers for signals that are unknown or reserved by the Java implementation can not be registered.? > > - Why don't we want to allow unknown signals? This will make us have to update our implementation if we want to support new or platform specific signals in the future. > The statement was aimed primarily at the Java Signal API; there is quite a bit of detail oriented code in the VM to initialize and handle signals. Most of it is agnostic to the signal number and would just pass it through. If a signal is not supported by the OS (think SIGHUP on Windows) that should bubble up as being not available. The 'cannot be registered' might be re-worded to say it throws an exception, as the method javadoc does. The set of signals is a pretty slow moving target so updating implementations should not be a big issue. > #2 Re: "java.util.Signal.raise()? > > - That API raises the signal in the current process, but what about sending a signal to another process for interprocess communication? I left that for a separate issue but would be a straight-forward addition to java.lang.ProcessHandle/Process. > > > #3 Re: "Signal.of("SIGINT?)? > > - Is this a factory method that returns the same object if called more than once? Maybe, maybe not, why would it matter. The real state is encapsulate is in the SignalImpl instances which are singletons per signal. I was trying to keep the Signal object stateless to allow it to be a value-type and lighter weight some day. > > > #4 Re: "public boolean unregister(Consumer consumer)? > > - Why is this API returning a value? Wouldn?t having a Signal API like ?public Consumer getConsumer()? be more flexible? The return value reports whether it unregistered the specific consumer. If it was not the concurrently registered the caller might want to know it was currently registered. I expect the return would mostly be ignored. The getConsumer()/unregister consumer pair would be vulnerable to race conditions and require some external locking to get predictable behavior. > > > #5 Re: "public void registerDefault(Consumer consumer)? > > - Do we really need this API? Can?t the same be achieved with the plain vanilla ?public void register(Consumer consumer)? I guess I don?t really see what makes this API special. The java runtime currently registers termination handler to cleanly shutdown if someone types control-c. It is useful to be able to remove the application provided signal handler and be able to restore the system defaults. This API could be hidden as a pure implementation detail. So unregistering the signal handler would always restore the appropriate system ones. Thanks for the review and comments, Roger > > > cheers > >> On Feb 1, 2016, at 10:02 AM, Roger Riggs wrote: >> >> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >> interactive applications that need to handle Control-C and other signals. >> >> The new java.util.Signal class provides a settable primary signal handler and a default >> signal handler. The primary signal handler can be unregistered and handling is restored >> to the default signal handler. System initialization registers default signal handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is not affected. >> The command option to reduce signal use by the runtime with -Xrs is unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback to the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> From aleksey.shipilev at oracle.com Mon Feb 1 19:47:23 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 1 Feb 2016 22:47:23 +0300 Subject: RFR (S) 8148787: StringConcatFactory exactness check produces bad bytecode when a non-arg concat is requested Message-ID: <56AFB64B.5040901@oracle.com> Hi, Please review the fix for a corner case in StringConcatFactory exactness check, which produces invalid bytecode: https://bugs.openjdk.java.net/browse/JDK-8148787 Note that this happens when all three things align: a) BSM is called directly, as Java method -- javac would never produce such a String concat shape; b) BC_SB_SIZED_EXACT strategy is used, so exactness check can be applied; c) -Djava.lang.invoke.stringConcat.debug=true is set, forcing exactness check to run; The issue is with exactness debug check using a temporary local variable when the local variable table is small (like it is in non-arg case). The code can be reformulated without using temporary variables, with a creative use of "swap" instruction. Ironically, the bug is within the debugging code, so we don't care about its performance at all: http://cr.openjdk.java.net/~shade/8148787/webrev.01/ Cheers, -Aleksey From gerard.ziemski at oracle.com Mon Feb 1 20:17:33 2016 From: gerard.ziemski at oracle.com (Gerard Ziemski) Date: Mon, 1 Feb 2016 14:17:33 -0600 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AFB110.3070902@Oracle.com> References: <56AF819C.5020905@Oracle.com> <3C91D205-04E9-43E2-A527-2051FFB5C24E@oracle.com> <56AFB110.3070902@Oracle.com> Message-ID: > On Feb 1, 2016, at 1:25 PM, Roger Riggs wrote: > > Hi Gerard, > > On 2/1/2016 1:58 PM, Gerard Ziemski wrote: >> hi Roger, >> >> I love that we are adding this Signal object. I have several questions, but please do not take them as criticism, I?m just seeking clarification and providing feedback: >> >> >> #1 Re: "Consumers for signals that are unknown or reserved by the Java implementation can not be registered.? >> >> - Why don't we want to allow unknown signals? This will make us have to update our implementation if we want to support new or platform specific signals in the future. >> >> > The statement was aimed primarily at the Java Signal API; there is quite a bit of detail > oriented code in the VM to initialize and handle signals. Most of it is agnostic to the signal number > and would just pass it through. If a signal is not supported by the OS (think SIGHUP on Windows) > that should bubble up as being not available. The 'cannot be registered' might be re-worded to say > it throws an exception, as the method javadoc does. > > The set of signals is a pretty slow moving target so updating implementations should not be a big issue. Right, but you don?t actually answer why we don?t allow unknown (to us at the moment) signals. Why have a limit in place, unless there is a good reason? >> #2 Re: "java.util.Signal.raise()? >> >> - That API raises the signal in the current process, but what about sending a signal to another process for interprocess communication? >> > I left that for a separate issue but would be a straight-forward addition to java.lang.ProcessHandle/Process. The proposed Signal ?feels? incomplete to me without this, though I understand that it meets the original goal. I would love to see at the very least a followup enhancement filed to address this. >> >> >> #3 Re: "Signal.of("SIGINT?)? >> >> - Is this a factory method that returns the same object if called more than once? >> > Maybe, maybe not, why would it matter. > The real state is encapsulate is in the SignalImpl instances which are singletons per signal. > I was trying to keep the Signal object stateless to allow it to be a value-type and lighter weight > some day. If it doesn?t matter, the why not just use constructor ?Signal signal = new Signal(?SIGINT?)? ? >> >> >> #4 Re: "public boolean unregister(Consumer consumer)? >> >> - Why is this API returning a value? Wouldn?t having a Signal API like ?public Consumer getConsumer()? be more flexible? >> > > The return value reports whether it unregistered the specific consumer. If it was not > the concurrently registered the caller might want to know it was currently registered. > I expect the return would mostly be ignored. > > The getConsumer()/unregister consumer pair would be vulnerable to race conditions > and require some external locking to get predictable behavior. Isn?t it also true for register/unregister? > >> >> >> #5 Re: "public void registerDefault(Consumer consumer)? >> >> - Do we really need this API? Can?t the same be achieved with the plain vanilla ?public void register(Consumer consumer)? I guess I don?t really see what makes this API special. >> > The java runtime currently registers termination handler to cleanly shutdown if someone types control-c. > It is useful to be able to remove the application provided signal handler and be able to restore the > system defaults. > This API could be hidden as a pure implementation detail. So unregistering the signal handler > would always restore the appropriate system ones. I was hoping that behavior is all that?s needed. > > Thanks for the review and comments, Roger Thanks for all the work! > >> >> >> cheers >> >> >>> On Feb 1, 2016, at 10:02 AM, Roger Riggs >>> wrote: >>> >>> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >>> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >>> interactive applications that need to handle Control-C and other signals. >>> >>> The new java.util.Signal class provides a settable primary signal handler and a default >>> signal handler. The primary signal handler can be unregistered and handling is restored >>> to the default signal handler. System initialization registers default signal handlers >>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >>> a permission if a SecurityManager is set. >>> >>> The sun.misc.Signal implementation is modified to be layered on a common >>> thread and dispatch mechanism. The VM handling of native signals is not affected. >>> The command option to reduce signal use by the runtime with -Xrs is unmodified. >>> >>> The changes to hotspot are minimal to rename the hardcoded callback to the Java >>> Signal dispatcher. >>> >>> Please review and comment on the API and implementation. >>> >>> javadoc: >>> >>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>> >>> >>> Webrev: >>> jdk: >>> http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>> >>> hotspot: >>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>> >>> >>> Issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>> >>> >>> JEP 260: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> >>> Thanks, Roger >>> >>> >>> > From steve.drach at oracle.com Mon Feb 1 20:18:01 2016 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 1 Feb 2016 12:18:01 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56AFAF6E.4060806@oracle.com> References: <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> Message-ID: >>>> Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. >>>> >>>> My assumption was the traversal should by default be consistent with a calls to getEntry, thus: >>>> >>>> jarFile.stream().forEach(e -> { >>>> JarEntry je = jarFile.getJarEntry(e.getName()); >>>> assert e.equals(je); >>>> }); >>>> >>>> There might need to be another stream method that returns all entries. >>>> >>> Right, I'm mostly just wondering if entries()/streams() should override the entries in the stream with versioned entries and filter out the META-INF/versions/ tree. >> I don?t think so. That kind of behavior might be difficult to understand. Returning all the entries provides some flexibility. One can write code like this: >> >> jarfile.stream().map(JarEntry::getName).filter(s -> !s.startsWith(?META-INF?)).map(JarFile::getJarEntry).etc >> >> to get the versioned results for any version you specify when constructing the JarFile. > > The current specification treats those class files under meta-inf/releases like > kind of "metadata" of those base entries. Ideally those files should not even > be individual "files", but part of their corresponding entries. The consumer of > the MR-Jar should not need to be aware of these version-ed entries at all to use > this MR-jar file to load classes/resources. From this point of view, these entries > probably should be "invisible" from entries()/stream(), when the jar file is opened > with "version-enabled". And all returned entries should have all their "data" > (size, csize, timestamps, crc ...) pointed to the corresponding version-ed entries, > withe the only exception is the "name". > > On the other hand it might be desired to keep JarFile.entries()/stream() asis to > match their "zip file" perspective, to return "all" raw entries. Then it might also > be desired to have an alternative "versioned streamVersion()" ? It seems to that we have two reasonable alternatives: (1) return all entries, and (2) return all entries except those under the ?META-INF/versions/? directory and for any entries returned, return their versioned equivalent if it exists. If we choose alternative 2, we can still get alternative 1 by asking for JarFile.super.entries() and JarFile.super.stream(). Or we can do it both ways, leaving entries()/stream() as is and adding two new methods, versionedEntries() and versionedStream(). > > something like > > public Stream stream(Release r); ? We should not parametrize the methods with a Release, because what does it mean if we construct the JarFile with one Release but specify a different Release for the stream argument. Parameterizing methods with a Release object feels like we?re starting to slide down a slippery slope. I think adding the two new methods is the ?right? solution, but I?d like some consensus here. > > -sherman > > > > >>> If I've gone to trouble of specifying the a Release then it seems the right thing to do. On the other hand, it comes at a cost and there will be use-cases like "get the names of all entries" that would be more efficient to just build on the current entries()/stream(). I'm loath to suggest this might need a new method but it might be one of the options to consider here. Minimally there is a javadoc to specify on how these methods behave when the JAR is multi-release and opened by specifying a release. >> How?s this? >> >> diff -r 68867430065b src/java.base/share/classes/java/util/jar/JarFile.java >> --- a/src/java.base/share/classes/java/util/jar/JarFile.java Fri Jan 29 12:34:44 2016 -0800 >> +++ b/src/java.base/share/classes/java/util/jar/JarFile.java Mon Feb 01 09:48:05 2016 -0800 >> @@ -576,9 +576,11 @@ >> } >> >> /** >> - * Returns an enumeration of the jar file entries. >> + * Returns an enumeration of all the jar file entries. Constructing this >> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >> + * constructor does not modify the behavior of this method. >> * >> - * @return an enumeration of the jar file entries >> + * @return an enumeration of the all jar file entries >> * @throws IllegalStateException >> * may be thrown if the jar file has been closed >> */ >> @@ -587,11 +589,13 @@ >> } >> >> /** >> - * Returns an ordered {@code Stream} over the jar file entries. >> + * Returns an ordered {@code Stream} over all the jar file entries. >> * Entries appear in the {@code Stream} in the order they appear in >> - * the central directory of the jar file. >> + * the central directory of the jar file. Constructing this >> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >> + * constructor does not modify the behavior of this method. >> * >> - * @return an ordered {@code Stream} of entries in this jar file >> + * @return an ordered {@code Stream} of all entries in this jar file >> * @throws IllegalStateException if the jar file has been closed >> * @since 1.8 >> */ From Roger.Riggs at Oracle.com Mon Feb 1 20:26:23 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 15:26:23 -0500 Subject: RFR 9: 8143016 : java/time/test/java/time/TestClock_System.java failed intermittently In-Reply-To: <56AF8AAE.9020000@Oracle.com> References: <56AF8AAE.9020000@Oracle.com> Message-ID: <56AFBF6F.2070605@Oracle.com> Hi, I discovered a flaw in the fix; it needs to re-read the instant from the clock inside the inner loop. Please review. Thanks, Roger On 2/1/2016 11:41 AM, Roger Riggs wrote: > Please review a test improvement to address an intermittent test > failure in > the java.time System Clock test for microsecond support. > > webrev: > http://cr.openjdk.java.net/~rriggs/webrev-micros-8143016/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8143016 > > Thanks, Roger > From steve.drach at oracle.com Mon Feb 1 20:29:52 2016 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 1 Feb 2016 12:29:52 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> Message-ID: I?m sorry, I didn?t look at the code close enough before I started talking ;-) Right now entries()/stream() returns all entries and if the JarFile is constructed with a Release object != Release.BASE, the base entries that are returned are the versioned entries. I think this behavior is a bit confusing and we should just return all entries without regard to versioning. Then create the two new methods for specific versioned entries. > On Feb 1, 2016, at 12:18 PM, Steve Drach wrote: > >>>>> Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. >>>>> >>>>> My assumption was the traversal should by default be consistent with a calls to getEntry, thus: >>>>> >>>>> jarFile.stream().forEach(e -> { >>>>> JarEntry je = jarFile.getJarEntry(e.getName()); >>>>> assert e.equals(je); >>>>> }); >>>>> >>>>> There might need to be another stream method that returns all entries. >>>>> >>>> Right, I'm mostly just wondering if entries()/streams() should override the entries in the stream with versioned entries and filter out the META-INF/versions/ tree. >>> I don?t think so. That kind of behavior might be difficult to understand. Returning all the entries provides some flexibility. One can write code like this: >>> >>> jarfile.stream().map(JarEntry::getName).filter(s -> !s.startsWith(?META-INF?)).map(JarFile::getJarEntry).etc >>> >>> to get the versioned results for any version you specify when constructing the JarFile. >> >> The current specification treats those class files under meta-inf/releases like >> kind of "metadata" of those base entries. Ideally those files should not even >> be individual "files", but part of their corresponding entries. The consumer of >> the MR-Jar should not need to be aware of these version-ed entries at all to use >> this MR-jar file to load classes/resources. From this point of view, these entries >> probably should be "invisible" from entries()/stream(), when the jar file is opened >> with "version-enabled". And all returned entries should have all their "data" >> (size, csize, timestamps, crc ...) pointed to the corresponding version-ed entries, >> withe the only exception is the "name". >> >> On the other hand it might be desired to keep JarFile.entries()/stream() asis to >> match their "zip file" perspective, to return "all" raw entries. Then it might also >> be desired to have an alternative "versioned streamVersion()" ? > > It seems to that we have two reasonable alternatives: (1) return all entries, and (2) return all entries except those under the ?META-INF/versions/? directory and for any entries returned, return their versioned equivalent if it exists. If we choose alternative 2, we can still get alternative 1 by asking for JarFile.super.entries() and JarFile.super.stream(). > > Or we can do it both ways, leaving entries()/stream() as is and adding two new methods, versionedEntries() and versionedStream(). > >> >> something like >> >> public Stream stream(Release r); ? > > We should not parametrize the methods with a Release, because what does it mean if we construct the JarFile with one Release but specify a different Release for the stream argument. Parameterizing methods with a Release object feels like we?re starting to slide down a slippery slope. > > I think adding the two new methods is the ?right? solution, but I?d like some consensus here. > >> >> -sherman >> >> >> >> >>>> If I've gone to trouble of specifying the a Release then it seems the right thing to do. On the other hand, it comes at a cost and there will be use-cases like "get the names of all entries" that would be more efficient to just build on the current entries()/stream(). I'm loath to suggest this might need a new method but it might be one of the options to consider here. Minimally there is a javadoc to specify on how these methods behave when the JAR is multi-release and opened by specifying a release. >>> How?s this? >>> >>> diff -r 68867430065b src/java.base/share/classes/java/util/jar/JarFile.java >>> --- a/src/java.base/share/classes/java/util/jar/JarFile.java Fri Jan 29 12:34:44 2016 -0800 >>> +++ b/src/java.base/share/classes/java/util/jar/JarFile.java Mon Feb 01 09:48:05 2016 -0800 >>> @@ -576,9 +576,11 @@ >>> } >>> >>> /** >>> - * Returns an enumeration of the jar file entries. >>> + * Returns an enumeration of all the jar file entries. Constructing this >>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >>> + * constructor does not modify the behavior of this method. >>> * >>> - * @return an enumeration of the jar file entries >>> + * @return an enumeration of the all jar file entries >>> * @throws IllegalStateException >>> * may be thrown if the jar file has been closed >>> */ >>> @@ -587,11 +589,13 @@ >>> } >>> >>> /** >>> - * Returns an ordered {@code Stream} over the jar file entries. >>> + * Returns an ordered {@code Stream} over all the jar file entries. >>> * Entries appear in the {@code Stream} in the order they appear in >>> - * the central directory of the jar file. >>> + * the central directory of the jar file. Constructing this >>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >>> + * constructor does not modify the behavior of this method. >>> * >>> - * @return an ordered {@code Stream} of entries in this jar file >>> + * @return an ordered {@code Stream} of all entries in this jar file >>> * @throws IllegalStateException if the jar file has been closed >>> * @since 1.8 >>> */ From Roger.Riggs at Oracle.com Mon Feb 1 20:37:19 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 15:37:19 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: References: <56AF819C.5020905@Oracle.com> <3C91D205-04E9-43E2-A527-2051FFB5C24E@oracle.com> <56AFB110.3070902@Oracle.com> Message-ID: <56AFC1FF.7050907@Oracle.com> Hi, On 2/1/2016 3:17 PM, Gerard Ziemski wrote: >> On Feb 1, 2016, at 1:25 PM, Roger Riggs wrote: >> >> Hi Gerard, >> >> On 2/1/2016 1:58 PM, Gerard Ziemski wrote: >>> hi Roger, >>> >>> I love that we are adding this Signal object. I have several questions, but please do not take them as criticism, I?m just seeking clarification and providing feedback: >>> >>> >>> #1 Re: "Consumers for signals that are unknown or reserved by the Java implementation can not be registered.? >>> >>> - Why don't we want to allow unknown signals? This will make us have to update our implementation if we want to support new or platform specific signals in the future. >>> >>> >> The statement was aimed primarily at the Java Signal API; there is quite a bit of detail >> oriented code in the VM to initialize and handle signals. Most of it is agnostic to the signal number >> and would just pass it through. If a signal is not supported by the OS (think SIGHUP on Windows) >> that should bubble up as being not available. The 'cannot be registered' might be re-worded to say >> it throws an exception, as the method javadoc does. >> >> The set of signals is a pretty slow moving target so updating implementations should not be a big issue. > Right, but you don?t actually answer why we don?t allow unknown (to us at the moment) signals. Why have a limit in place, unless there is a good reason? There is no artificial limit; just the list of signal names. The implementation should not be expected to handle unknown inputs. > > >>> #2 Re: "java.util.Signal.raise()? >>> >>> - That API raises the signal in the current process, but what about sending a signal to another process for interprocess communication? >>> >> I left that for a separate issue but would be a straight-forward addition to java.lang.ProcessHandle/Process. > The proposed Signal ?feels? incomplete to me without this, though I understand that it meets the original goal. I would love to see at the very least a followup enhancement filed to address this. how about: 4914493 (process) Cannot send arbitary signals to UNIX process > >>> >>> #3 Re: "Signal.of("SIGINT?)? >>> >>> - Is this a factory method that returns the same object if called more than once? >>> >> Maybe, maybe not, why would it matter. >> The real state is encapsulate is in the SignalImpl instances which are singletons per signal. >> I was trying to keep the Signal object stateless to allow it to be a value-type and lighter weight >> some day. > If it doesn?t matter, the why not just use constructor ?Signal signal = new Signal(?SIGINT?)? ? Factory methods are preferred to constructors; it allows greater flexibility in the implementation, current and future. > > >>> >>> #4 Re: "public boolean unregister(Consumer consumer)? >>> >>> - Why is this API returning a value? Wouldn?t having a Signal API like ?public Consumer getConsumer()? be more flexible? >>> >> The return value reports whether it unregistered the specific consumer. If it was not >> the concurrently registered the caller might want to know it was currently registered. >> I expect the return would mostly be ignored. >> >> The getConsumer()/unregister consumer pair would be vulnerable to race conditions >> and require some external locking to get predictable behavior. > Isn?t it also true for register/unregister? Register is a strong takeover of responsibility for handling the signal. Unregister is safer if the caller has to say which handler to remove and not remove one the caller did not register. In practice, it requires a higher level coordination to avoid interference between competing interests in handling signals which makes it more complicated than necessary for the use case. > >>> >>> #5 Re: "public void registerDefault(Consumer consumer)? >>> >>> - Do we really need this API? Can?t the same be achieved with the plain vanilla ?public void register(Consumer consumer)? I guess I don?t really see what makes this API special. >>> >> The java runtime currently registers termination handler to cleanly shutdown if someone types control-c. >> It is useful to be able to remove the application provided signal handler and be able to restore the >> system defaults. >> This API could be hidden as a pure implementation detail. So unregistering the signal handler >> would always restore the appropriate system ones. > I was hoping that behavior is all that?s needed. Worth considering, will look for other comments on the subject. Thanks, Roger > > >> Thanks for the review and comments, Roger > Thanks for all the work! > > >>> >>> cheers >>> >>> >>>> On Feb 1, 2016, at 10:02 AM, Roger Riggs >>>> wrote: >>>> >>>> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >>>> interactive applications that need to handle Control-C and other signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal handler and a default >>>> signal handler. The primary signal handler can be unregistered and handling is restored >>>> to the default signal handler. System initialization registers default signal handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a common >>>> thread and dispatch mechanism. The VM handling of native signals is not affected. >>>> The command option to reduce signal use by the runtime with -Xrs is unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback to the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> >>>> Webrev: >>>> jdk: >>>> http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> >>>> hotspot: >>>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> >>>> Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> >>>> JEP 260: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> >>>> Thanks, Roger >>>> >>>> >>>> From pbenedict at apache.org Mon Feb 1 20:55:51 2016 From: pbenedict at apache.org (Paul Benedict) Date: Mon, 1 Feb 2016 14:55:51 -0600 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> Message-ID: I believe the answer should be based on your viewpoint of what "is" a JAR. Do you consider the JAR to be a dumb ZIP container or an artifact of the Java runtime? If it's the former, then return everything because the version folder is meaningless in this perspective. Otherwise, treat the JAR according to how Java runtime would load it, which is the versioned entries. Whatever your answer is, that should be the behavior for entries()/stream(). Those who don't like the behavior should be given a second set of methods to customize the return. Cheers, Paul On Mon, Feb 1, 2016 at 2:29 PM, Steve Drach wrote: > I?m sorry, I didn?t look at the code close enough before I started talking > ;-) Right now entries()/stream() returns all entries and if the JarFile is > constructed with a Release object != Release.BASE, the base entries that > are returned are the versioned entries. I think this behavior is a bit > confusing and we should just return all entries without regard to > versioning. Then create the two new methods for specific versioned entries. > > > On Feb 1, 2016, at 12:18 PM, Steve Drach wrote: > > > >>>>> Alan?s point is that traversing using entries()/stream() always > returns the versioned entries (if any) rather than all entries, thus in a > sense filters. > >>>>> > >>>>> My assumption was the traversal should by default be consistent with > a calls to getEntry, thus: > >>>>> > >>>>> jarFile.stream().forEach(e -> { > >>>>> JarEntry je = jarFile.getJarEntry(e.getName()); > >>>>> assert e.equals(je); > >>>>> }); > >>>>> > >>>>> There might need to be another stream method that returns all > entries. > >>>>> > >>>> Right, I'm mostly just wondering if entries()/streams() should > override the entries in the stream with versioned entries and filter out > the META-INF/versions/ tree. > >>> I don?t think so. That kind of behavior might be difficult to > understand. Returning all the entries provides some flexibility. One can > write code like this: > >>> > >>> jarfile.stream().map(JarEntry::getName).filter(s -> > !s.startsWith(?META-INF?)).map(JarFile::getJarEntry).etc > >>> > >>> to get the versioned results for any version you specify when > constructing the JarFile. > >> > >> The current specification treats those class files under > meta-inf/releases like > >> kind of "metadata" of those base entries. Ideally those files should > not even > >> be individual "files", but part of their corresponding entries. The > consumer of > >> the MR-Jar should not need to be aware of these version-ed entries at > all to use > >> this MR-jar file to load classes/resources. From this point of view, > these entries > >> probably should be "invisible" from entries()/stream(), when the jar > file is opened > >> with "version-enabled". And all returned entries should have all their > "data" > >> (size, csize, timestamps, crc ...) pointed to the corresponding > version-ed entries, > >> withe the only exception is the "name". > >> > >> On the other hand it might be desired to keep > JarFile.entries()/stream() asis to > >> match their "zip file" perspective, to return "all" raw entries. Then > it might also > >> be desired to have an alternative "versioned streamVersion()" ? > > > > It seems to that we have two reasonable alternatives: (1) return all > entries, and (2) return all entries except those under the > ?META-INF/versions/? directory and for any entries returned, return their > versioned equivalent if it exists. If we choose alternative 2, we can > still get alternative 1 by asking for JarFile.super.entries() and > JarFile.super.stream(). > > > > Or we can do it both ways, leaving entries()/stream() as is and adding > two new methods, versionedEntries() and versionedStream(). > > > >> > >> something like > >> > >> public Stream stream(Release r); ? > > > > We should not parametrize the methods with a Release, because what does > it mean if we construct the JarFile with one Release but specify a > different Release for the stream argument. Parameterizing methods with a > Release object feels like we?re starting to slide down a slippery slope. > > > > I think adding the two new methods is the ?right? solution, but I?d like > some consensus here. > > > >> > >> -sherman > >> > >> > >> > >> > >>>> If I've gone to trouble of specifying the a Release then it seems the > right thing to do. On the other hand, it comes at a cost and there will be > use-cases like "get the names of all entries" that would be more efficient > to just build on the current entries()/stream(). I'm loath to suggest this > might need a new method but it might be one of the options to consider > here. Minimally there is a javadoc to specify on how these methods behave > when the JAR is multi-release and opened by specifying a release. > >>> How?s this? > >>> > >>> diff -r 68867430065b > src/java.base/share/classes/java/util/jar/JarFile.java > >>> --- a/src/java.base/share/classes/java/util/jar/JarFile.java > Fri Jan 29 12:34:44 2016 -0800 > >>> +++ b/src/java.base/share/classes/java/util/jar/JarFile.java > Mon Feb 01 09:48:05 2016 -0800 > >>> @@ -576,9 +576,11 @@ > >>> } > >>> > >>> /** > >>> - * Returns an enumeration of the jar file entries. > >>> + * Returns an enumeration of all the jar file entries. > Constructing this > >>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, > Release)} > >>> + * constructor does not modify the behavior of this method. > >>> * > >>> - * @return an enumeration of the jar file entries > >>> + * @return an enumeration of the all jar file entries > >>> * @throws IllegalStateException > >>> * may be thrown if the jar file has been closed > >>> */ > >>> @@ -587,11 +589,13 @@ > >>> } > >>> > >>> /** > >>> - * Returns an ordered {@code Stream} over the jar file entries. > >>> + * Returns an ordered {@code Stream} over all the jar file > entries. > >>> * Entries appear in the {@code Stream} in the order they appear in > >>> - * the central directory of the jar file. > >>> + * the central directory of the jar file. Constructing this > >>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, > Release)} > >>> + * constructor does not modify the behavior of this method. > >>> * > >>> - * @return an ordered {@code Stream} of entries in this jar file > >>> + * @return an ordered {@code Stream} of all entries in this jar > file > >>> * @throws IllegalStateException if the jar file has been closed > >>> * @since 1.8 > >>> */ > > From lance.andersen at oracle.com Mon Feb 1 21:07:18 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Mon, 1 Feb 2016 16:07:18 -0500 Subject: RFR 9: 8143016 : java/time/test/java/time/TestClock_System.java failed intermittently In-Reply-To: <56AFBF6F.2070605@Oracle.com> References: <56AF8AAE.9020000@Oracle.com> <56AFBF6F.2070605@Oracle.com> Message-ID: <1EC56755-9D33-45FA-84FE-B3A001DCBCA5@oracle.com> See the change, looks OK On Feb 1, 2016, at 3:26 PM, Roger Riggs wrote: > Hi, > > I discovered a flaw in the fix; it needs to re-read the instant from the clock inside the inner loop. > Please review. > > Thanks, Roger > > On 2/1/2016 11:41 AM, Roger Riggs wrote: >> Please review a test improvement to address an intermittent test failure in >> the java.time System Clock test for microsecond support. >> >> webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-micros-8143016/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8143016 >> >> Thanks, Roger >> > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From david.holmes at oracle.com Mon Feb 1 21:26:34 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Feb 2016 07:26:34 +1000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF819C.5020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> Message-ID: <56AFCD8A.9000306@oracle.com> Hi Roger, Sorry I couldn't look into the code in extreme detail right now but would like to clarify the big picture ... Can you please clarify the exact flow of control from when a signal is sent to the process and when the Java handler for it gets to run - what is handled by which thread where, and in what context (signal handling context or regular execution context)? I'm assuming no Java code is ever being executed in a signal handling context, but I'm not clear on the threading model being used here. Is the VM still responsible for the initial receipt of all signals? Howe do you manage a user-defined signal handler, for ctrl-C say, with the default behaviour for that signal (orderly shutdown) ? Thanks, David On 2/02/2016 2:02 AM, Roger Riggs wrote: > Please review an API addition to handle signals such as SIGINT, SIGHUP, > and SIGTERM. > This JEP 260 motivated alternative to sun.misc.Signal supports the use > case for > interactive applications that need to handle Control-C and other signals. > > The new java.util.Signal class provides a settable primary signal > handler and a default > signal handler. The primary signal handler can be unregistered and > handling is restored > to the default signal handler. System initialization registers default > signal handlers > to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API > requires > a permission if a SecurityManager is set. > > The sun.misc.Signal implementation is modified to be layered on a common > thread and dispatch mechanism. The VM handling of native signals is not > affected. > The command option to reduce signal use by the runtime with -Xrs is > unmodified. > > The changes to hotspot are minimal to rename the hardcoded callback to > the Java > Signal dispatcher. > > Please review and comment on the API and implementation. > > javadoc: > http://cr.openjdk.java.net/~rriggs/signal-doc/ > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8087286 > > JEP 260: > https://bugs.openjdk.java.net/browse/JDK-8132928 > > Thanks, Roger > > From Roger.Riggs at Oracle.com Mon Feb 1 21:41:59 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 1 Feb 2016 16:41:59 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AFCD8A.9000306@oracle.com> References: <56AF819C.5020905@Oracle.com> <56AFCD8A.9000306@oracle.com> Message-ID: <56AFD127.50607@Oracle.com> HI David, On 2/1/2016 4:26 PM, David Holmes wrote: > Hi Roger, > > Sorry I couldn't look into the code in extreme detail right now but > would like to clarify the big picture ... > > Can you please clarify the exact flow of control from when a signal is > sent to the process and when the Java handler for it gets to run - > what is handled by which thread where, and in what context (signal > handling context or regular execution context)? I'm assuming no Java > code is ever being executed in a signal handling context, but I'm not > clear on the threading model being used here. The VM signal handling control flows are unchanged, I understand it roughly as the vm has a native signal handler that atomically sets an entry in an array indexed by signal number. See os.cpp. A separate dedicated thread waits to be woken up (signal_wait) and polls the array (os_linux.cpp:check_pending_signals). It then does an upcall on the dedicated thread to call the java.util.Signal.dispatch method. The java code takes it from there to find a registered java signal handler and then start a new thread to call the signal handler. The invocation of the java handler method is asynchronous to the native signal handler and due to thread scheduling for the dedicated thread and the spawning of a new thread, quite some time may pass before the registered java signal handler is invoked. > > Is the VM still responsible for the initial receipt of all signals? yes, no change. > > How do you manage a user-defined signal handler, for ctrl-C say, with > the default behaviour for that signal (orderly shutdown) ? System initialization registers a default signal handler for SIGINT, SIGTERM, and SIGHUP (not windows). If there is no registered signal handler, the default handler is invoked in the same manner. Roger > > Thanks, > David > > On 2/02/2016 2:02 AM, Roger Riggs wrote: >> Please review an API addition to handle signals such as SIGINT, SIGHUP, >> and SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the use >> case for >> interactive applications that need to handle Control-C and other >> signals. >> >> The new java.util.Signal class provides a settable primary signal >> handler and a default >> signal handler. The primary signal handler can be unregistered and >> handling is restored >> to the default signal handler. System initialization registers default >> signal handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >> requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is not >> affected. >> The command option to reduce signal use by the runtime with -Xrs is >> unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback to >> the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> From kim.barrett at oracle.com Tue Feb 2 00:05:10 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 1 Feb 2016 19:05:10 -0500 Subject: RFR: 8072777: java/lang/ref/ReferenceEnqueuePending.java: some references aren't queued In-Reply-To: <56ADD196.9040809@gmail.com> References: <9C4B1F92-29F6-4977-BB48-342ADF886B56@oracle.com> <56ADD196.9040809@gmail.com> Message-ID: > On Jan 31, 2016, at 4:19 AM, Peter Levart wrote: > > Hi Kim, Hi Peter - Thanks for looking at this. > This change will make it practically impossible to miss the enqueued references. Good. That?s the goal. > But this could be an opportunity to also clean-up the code a bit. The checkResult method has an 'obj' parameter whose purpose is not immediately obvious. [?] > > I don't know why such complications are necessary. Is the test supposed to also verify the requirement that a Reference whose referent is kept strongly reachable will not be enqueued? I was wondering about the odd stuff around obj as well. I don't think it is an attempt to "also verify the requirement that a Reference whose referent is kept strongly reachable will not be enqueued". Rather, I think it is a kludgy way to avoid having the final weaky be sometimes enqueued and sometimes not, depending on compiler optimizations. Using Reference.reachabilityFence to keep obj and weaky live across the checkResult as you suggested is one way to accomplish that, though keeping only the final obj alive (as in the existing code) suffices to keep the final weaky from being notified. I think reachabilityFence and a better comment would be clearer though, so changing accordingly. New webrevs: full: http://cr.openjdk.java.net/~kbarrett/8072777/webrev.01/ incr: http://cr.openjdk.java.net/~kbarrett/8072777/webrev.01.inc/ From david.holmes at oracle.com Tue Feb 2 00:41:06 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 2 Feb 2016 10:41:06 +1000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AFD127.50607@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56AFCD8A.9000306@oracle.com> <56AFD127.50607@Oracle.com> Message-ID: <56AFFB22.1090500@oracle.com> Hi Roger, On 2/02/2016 7:41 AM, Roger Riggs wrote: > HI David, > > > On 2/1/2016 4:26 PM, David Holmes wrote: >> Hi Roger, >> >> Sorry I couldn't look into the code in extreme detail right now but >> would like to clarify the big picture ... >> >> Can you please clarify the exact flow of control from when a signal is >> sent to the process and when the Java handler for it gets to run - >> what is handled by which thread where, and in what context (signal >> handling context or regular execution context)? I'm assuming no Java >> code is ever being executed in a signal handling context, but I'm not >> clear on the threading model being used here. > The VM signal handling control flows are unchanged, I understand it > roughly as the vm has a native signal handler > that atomically sets an entry in an array indexed by signal number. > See os.cpp. > > A separate dedicated thread waits to be woken up (signal_wait) and polls > the array (os_linux.cpp:check_pending_signals). > It then does an upcall on the dedicated thread to call the > java.util.Signal.dispatch method. > The java code takes it from there to find a registered java signal > handler and then start > a new thread to call the signal handler. > > The invocation of the java handler method is asynchronous to the native > signal handler > and due to thread scheduling for the dedicated thread and the spawning > of a new thread, > quite some time may pass before the registered java signal handler is > invoked. Ok. So this is primarily moving sun.misc.Signal to java.util.Signal with a supported API? >> >> Is the VM still responsible for the initial receipt of all signals? > yes, no change. I need to refresh myself as to how the set of acceptable signals is defined ie the sig masks. >> >> How do you manage a user-defined signal handler, for ctrl-C say, with >> the default behaviour for that signal (orderly shutdown) ? > System initialization registers a default signal handler for SIGINT, > SIGTERM, and SIGHUP (not windows). > If there is no registered signal handler, the default handler is invoked > in the same manner. Does that mean that a user registered ctrl-C handler will prevent the VM from being terminated by ctrl-C? I assume so. I'm dubious about the need for the average user to be able to provide Java level "signal" handlers - it is not as clear cut as it might seem. These are low-level system concepts that don't really fit in to the Java WORA principle in my opinion. We have provided lifecycle management of the VM by other means, including the use of ShutdownHooks. I guess I should have been aware of JEP 260's intentions re sun.misc.Signal. I don't think a public supported API is appropriate for this and I think it will open a can of worms when it comes to VM lifecycle management. Sorry. David ----- > Roger > >> >> Thanks, >> David >> >> On 2/02/2016 2:02 AM, Roger Riggs wrote: >>> Please review an API addition to handle signals such as SIGINT, SIGHUP, >>> and SIGTERM. >>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>> case for >>> interactive applications that need to handle Control-C and other >>> signals. >>> >>> The new java.util.Signal class provides a settable primary signal >>> handler and a default >>> signal handler. The primary signal handler can be unregistered and >>> handling is restored >>> to the default signal handler. System initialization registers default >>> signal handlers >>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>> requires >>> a permission if a SecurityManager is set. >>> >>> The sun.misc.Signal implementation is modified to be layered on a common >>> thread and dispatch mechanism. The VM handling of native signals is not >>> affected. >>> The command option to reduce signal use by the runtime with -Xrs is >>> unmodified. >>> >>> The changes to hotspot are minimal to rename the hardcoded callback to >>> the Java >>> Signal dispatcher. >>> >>> Please review and comment on the API and implementation. >>> >>> javadoc: >>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>> >>> Webrev: >>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>> >>> JEP 260: >>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> Thanks, Roger >>> >>> > From iris.clark at oracle.com Tue Feb 2 05:51:20 2016 From: iris.clark at oracle.com (Iris Clark) Date: Mon, 1 Feb 2016 21:51:20 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <04d501d14e7f$1ee19b50$5ca4d1f0$@tma.com.vn> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> <04d501d14e7f$1ee19b50$5ca4d1f0$@tma.com.vn> Message-ID: <7b5a0fda-6312-41ba-b211-ea56ae7031f1@default> Hi, Hong. Thanks again for looking closely at the regular expression and providing comments. > - The outer most quantifier in (((\.0)*\.[1-9][0-9]*)*)* is redundant > and a source of catastrophic backtracking. You only need > ((\.0)*\.[1-9][0-9]*)*. > - The outside - in PRE part (\-([a-zA-Z0-9]+))? doesn't need escaping. > Simplify it to (-([a-zA-Z0-9]+))? Both done. > - Do you want to allow only a plus without the number in the BUILD ? part? Why do you capture the +? ((\+)(0|[1-9][0-9]*)?)? I need to capture the plus to distinguish between cases where an empty build is allowed (e.g. "9+-foo") and when it is not ("9+"). See code in Version.java, line 226-230 and in Basic.java, line 98, 107-109. (Note that we use the empty "+" to distinguish "9-foo" from "9+-foo".) > - Dot loses meaning in the character class, and hyphen loses > meaning at the beginning or at the end of the character class. You > can simplify the OPT part to (-([-a-zA-Z0-9.]+))? Done. > - You might want to consider using named-capturing groups instead > of numbered capturing groups. Also, consider using non-capturing > groups for groups you don't need to extract. Done. This is the (hopefully) final version of the webrev and javadoc for the initial implementation of this API: http://cr.openjdk.java.net/~iris/verona/8072379/webrev.3/index.html http://cr.openjdk.java.net/~iris/verona/8072379/doc.3/jdk/Version.html I've filed the following bug to update the spec: 8148822: (spec) Regex in jdk.Version and JEP 223 should match https://bugs.openjdk.java.net/browse/JDK-8148822 These are the corresponding changes to the JEP which I'll apply in the next day or so: $ diff jep-open-mr11.md jep-open-mr12.md 84c84 < `^[1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*$`. The sequence may be of arbitrary --- > `^[1-9][0-9]*((\.0)*\.[1-9][0-9]*)*$`. The sequence may be of arbitrary 166c166 < - $OPT, matching `([-a-zA-Z0-9\.]+)` --- Additional --- > - $OPT, matching `([-a-zA-Z0-9.]+)` --- Additional Thanks again for the recommendations. Regards, iris -----Original Message----- From: Thanh Hong Dai [mailto:hdthanh at tma.com.vn] Sent: Wednesday, January 13, 2016 7:53 PM To: Iris Clark; Alan Bateman; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Some comment on the regex (and also the JEP): ([1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*)(\-([a-zA-Z0-9]+))?((\+)(0|[1-9][0-9]*)?)?(-([\-a-zA-Z0-9\.]+))? - The outer most quantifier in (((\.0)*\.[1-9][0-9]*)*)* is redundant and a source of catastrophic backtracking. You only need ((\.0)*\.[1-9][0-9]*)*. - The outside - in PRE part (\-([a-zA-Z0-9]+))? doesn't need escaping. Simplify it to (-([a-zA-Z0-9]+))? - Do you want to allow only a plus without the number in the BUILD part? Why do you capture the +? ((\+)(0|[1-9][0-9]*)?)? - Dot loses meaning in the character class, and hyphen loses meaning at the beginning or at the end of the character class. You can simplify the OPT part to (-([-a-zA-Z0-9.]+))? - You might want to consider using named-capturing groups instead of numbered capturing groups. Also, consider using non-capturing groups for groups you don't need to extract. Best regards, Hong Dai Thanh. -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Iris Clark Sent: Thursday, 14 January, 2016 4:55 AM To: Alan Bateman ; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Alan. Thanks for looking at this (hopefully) one last time. > It can't be java.base (see design principles in JEP 200). > If it's going into java.base temporarily then the top-level > modules.xml will need to be updated to export the "jdk" package. This diff has been applied to modules.xml: diff -r 6fefc5bce180 modules.xml --- a/modules.xml Wed Jan 13 13:56:19 2016 +0000 +++ b/modules.xml Wed Jan 13 13:46:56 2016 -0800 @@ -205,6 +205,9 @@ javax.security.cert + jdk + + jdk.net It essentially reverts your 8049422 change [0] to that file. I will not re-add jdk to the javadoc bundle for javac trees API since that is not an appropriate location. I filed the following bug to track publication of jdk.Version: 8144069: Determine correct publication for jdk.Version API https://bugs.openjdk.java.net/browse/JDK-8144069 When this came up earlier, I filed this bug to track finding a more appropriate module for jdk.Version: 8144062: Determine appropriate module for jdk.Version https://bugs.openjdk.java.net/browse/JDK-8144062 Thanks, iris [0] http://hg.openjdk.java.net/jdk9/dev/rev/1bee5efa73e3 -----Original Message----- From: Alan Bateman Sent: Tuesday, January 12, 2016 7:41 AM To: Iris Clark; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: Re: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion On 11/01/2016 21:44, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter break and > my vacation), I provided responses to your messages and an updated > webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume that > you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037 > 062.html > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036 > 904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > I think this looks good but we'll to decide which module to put this in. It can't be java.base (see design principles in JEP 200). If it's going into java.base temporarily then the top-level modules.xml will need to be updated to export the "jdk" package. -Alan From amaembo at gmail.com Tue Feb 2 06:19:10 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Tue, 2 Feb 2016 12:19:10 +0600 Subject: RFR: 8079136: Accessing a nested sublist leads to StackOverflowError In-Reply-To: <56AB8FE0.5080301@oracle.com> References: <55486EA5.8000009@oracle.com> <56A9D4D7.5040506@oracle.com> <85988769.20160129101608@gmail.com> <56AB8FE0.5080301@oracle.com> Message-ID: <35864771.20160202121910@gmail.com> Hello! IG> It's misfortune that the spec of subList() doesn't match the spec of IG> similar methods, like CharSequence.subSequence() or String.substring(). IG> It even contradicts the spec of List.subList(), which declares only IOOB IG> to be thrown! Yes, it's a pity. Isn't it the reason to change (fix) the spec making List.subList and AbstractList.subList conformant? >> AbstractList::rangeCheck could be replaced with Obejcts::checkIndex. >> The same for ArrayList class. IG> Right. IG> Here's the updated webrev: IG> http://cr.openjdk.java.net/~igerasim/8079136/06/webrev/ I have a doubt about replacing rangeCheckForAdd. What if size() == Integer.MAX_VALUE? This is not an issue for ArrayList as it's limited by MAX_ARRAY_SIZE which is Integer.MAX_VALUE - 8. However for user defined AbstractList having size = Integer.MAX_VALUE looks possible. In this case range check would fail for any index. With best regards, Tagir Valeev. IG> Sincerely yours, IG> Ivan >> Otherwise looks good to me. >> >> With best regards, >> Tagir Valeev. >> >> IG> Hello everyone! >> >> IG> I'd like to respin the discussion. >> IG> The previous attempt can be found here: >> IG> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033159.html >> >> IG> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8079136 >> IG> WEBREV: http://cr.openjdk.java.net/~igerasim/8079136/05/webrev/ >> >> IG> Here's the summary of the proposed changes: >> IG> 1) >> IG> Sublist of an AbstractList (AbstractList.SubList class) now maintains a >> IG> link to the root AbstractList, and not only to the immediate parent list. >> IG> This allows to perform modifications on the chain of sub-lists in a loop >> IG> instead of using recursion (which, in particular, helps avoid the >> IG> stack-overflow problem). >> IG> The sublist is still backed by its parent list, in sense that all the >> IG> modifications are correctly reflected in the backing list (as well as in >> IG> all the grand parents the sublist, if any), so the fix does not violate >> IG> the existing specification. >> >> IG> 2) >> IG> It is proposed to update the spec of AbstractList.subList() in the >> IG> @implSpec section by removing the words about private fields. >> IG> With the fix, some of those private fields are removed. >> >> IG> 3) >> IG> A similar change is proposed for the ArrayList.SubList class. >> >> IG> 4) >> IG> Two regression tests are added: >> IG> NestedSubList.java - demonstrates a stack-overflow problem when dealing >> IG> with a long chain of sublists, >> IG> SubList.java - tests basic functionality of sub-lists, which helps us >> IG> make sure nothing is broken with the proposed change. >> >> IG> If people agree that the fix is good, I'll file a CCC request for >> IG> changing the spec of AbstractList.subList(). >> >> IG> Sincerely yours, >> IG> Ivan >> >> From ivan.gerasimov at oracle.com Tue Feb 2 06:40:50 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 2 Feb 2016 09:40:50 +0300 Subject: RFR: 8079136: Accessing a nested sublist leads to StackOverflowError In-Reply-To: <35864771.20160202121910@gmail.com> References: <55486EA5.8000009@oracle.com> <56A9D4D7.5040506@oracle.com> <85988769.20160129101608@gmail.com> <56AB8FE0.5080301@oracle.com> <35864771.20160202121910@gmail.com> Message-ID: <56B04F72.6070207@oracle.com> Thanks Tagir for the comments! On 02.02.2016 9:19, Tagir F. Valeev wrote: > Hello! > > IG> It's misfortune that the spec of subList() doesn't match the spec of > IG> similar methods, like CharSequence.subSequence() or String.substring(). > IG> It even contradicts the spec of List.subList(), which declares only IOOB > IG> to be thrown! > > Yes, it's a pity. Isn't it the reason to change (fix) the spec making > List.subList and AbstractList.subList conformant? A request to fix that is already there: https://bugs.openjdk.java.net/browse/JDK-4506427 (it's 15 years old!) I'd rather keep it separate from this fix (JDK-8079136). >>> AbstractList::rangeCheck could be replaced with Obejcts::checkIndex. >>> The same for ArrayList class. > IG> Right. > IG> Here's the updated webrev: > IG> http://cr.openjdk.java.net/~igerasim/8079136/06/webrev/ > > I have a doubt about replacing rangeCheckForAdd. What if size() == > Integer.MAX_VALUE? This is not an issue for ArrayList as it's limited > by MAX_ARRAY_SIZE which is Integer.MAX_VALUE - 8. However for user > defined AbstractList having size = Integer.MAX_VALUE looks possible. > In this case range check would fail for any index. Ah, good point. I'll revert that part that part of the change back. Sincerely yours, Ivan > With best regards, > Tagir Valeev. > > IG> Sincerely yours, > IG> Ivan > >>> Otherwise looks good to me. >>> >>> With best regards, >>> Tagir Valeev. >>> >>> IG> Hello everyone! >>> >>> IG> I'd like to respin the discussion. >>> IG> The previous attempt can be found here: >>> IG> http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-May/033159.html >>> >>> IG> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8079136 >>> IG> WEBREV: http://cr.openjdk.java.net/~igerasim/8079136/05/webrev/ >>> >>> IG> Here's the summary of the proposed changes: >>> IG> 1) >>> IG> Sublist of an AbstractList (AbstractList.SubList class) now maintains a >>> IG> link to the root AbstractList, and not only to the immediate parent list. >>> IG> This allows to perform modifications on the chain of sub-lists in a loop >>> IG> instead of using recursion (which, in particular, helps avoid the >>> IG> stack-overflow problem). >>> IG> The sublist is still backed by its parent list, in sense that all the >>> IG> modifications are correctly reflected in the backing list (as well as in >>> IG> all the grand parents the sublist, if any), so the fix does not violate >>> IG> the existing specification. >>> >>> IG> 2) >>> IG> It is proposed to update the spec of AbstractList.subList() in the >>> IG> @implSpec section by removing the words about private fields. >>> IG> With the fix, some of those private fields are removed. >>> >>> IG> 3) >>> IG> A similar change is proposed for the ArrayList.SubList class. >>> >>> IG> 4) >>> IG> Two regression tests are added: >>> IG> NestedSubList.java - demonstrates a stack-overflow problem when dealing >>> IG> with a long chain of sublists, >>> IG> SubList.java - tests basic functionality of sub-lists, which helps us >>> IG> make sure nothing is broken with the proposed change. >>> >>> IG> If people agree that the fix is good, I'll file a CCC request for >>> IG> changing the spec of AbstractList.subList(). >>> >>> IG> Sincerely yours, >>> IG> Ivan >>> >>> > From martinrb at google.com Tue Feb 2 06:55:51 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 1 Feb 2016 22:55:51 -0800 Subject: RFR: 8079136: Accessing a nested sublist leads to StackOverflowError In-Reply-To: <35864771.20160202121910@gmail.com> References: <55486EA5.8000009@oracle.com> <56A9D4D7.5040506@oracle.com> <85988769.20160129101608@gmail.com> <56AB8FE0.5080301@oracle.com> <35864771.20160202121910@gmail.com> Message-ID: On Mon, Feb 1, 2016 at 10:19 PM, Tagir F. Valeev wrote: > I have a doubt about replacing rangeCheckForAdd. What if size() == > Integer.MAX_VALUE? This is not an issue for ArrayList as it's limited > by MAX_ARRAY_SIZE which is Integer.MAX_VALUE - 8. Actually, the limit is Integer.MAX_VALUE. But it only grows beyond MAX_ARRAY_SIZE if there's no choice. From aleksey.shipilev at oracle.com Tue Feb 2 07:00:57 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 2 Feb 2016 10:00:57 +0300 Subject: RFR (S) 8148787: StringConcatFactory exactness check produces bad bytecode when a non-arg concat is requested In-Reply-To: <56AFB64B.5040901@oracle.com> References: <56AFB64B.5040901@oracle.com> Message-ID: <56B05429.2000007@oracle.com> Anyone? This looks like a trivial fix. -Aleksey On 02/01/2016 10:47 PM, Aleksey Shipilev wrote: > Hi, > > Please review the fix for a corner case in StringConcatFactory exactness > check, which produces invalid bytecode: > https://bugs.openjdk.java.net/browse/JDK-8148787 > > Note that this happens when all three things align: > a) BSM is called directly, as Java method -- javac would never produce > such a String concat shape; > b) BC_SB_SIZED_EXACT strategy is used, so exactness check can be applied; > c) -Djava.lang.invoke.stringConcat.debug=true is set, forcing > exactness check to run; > > The issue is with exactness debug check using a temporary local variable > when the local variable table is small (like it is in non-arg case). The > code can be reformulated without using temporary variables, with a > creative use of "swap" instruction. > > Ironically, the bug is within the debugging code, so we don't care about > its performance at all: > http://cr.openjdk.java.net/~shade/8148787/webrev.01/ > > Cheers, > -Aleksey > From hdthanh at tma.com.vn Tue Feb 2 07:48:29 2016 From: hdthanh at tma.com.vn (Thanh Hong Dai) Date: Tue, 2 Feb 2016 14:48:29 +0700 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <7b5a0fda-6312-41ba-b211-ea56ae7031f1@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> <04d501d14e7f$1ee19b50$5ca4d1f0$@tma.com.vn> <7b5a0fda-6312-41ba-b211-ea56ae7031f1@default> Message-ID: <01a501d15d8e$1b683b50$5238b1f0$@tma.com.vn> Dear Iris, On closer look, there seems to be some conflicting definition of version string. In JEP: http://openjdk.java.net/jeps/223 $VNUM(-$PRE)?(\+$BUILD)?(-$OPT)? In Javadoc: http://cr.openjdk.java.net/~iris/verona/8072379/doc.3/jdk/Version.html $VNUM(-$PRE)?(\+($BUILD)?(-$OPT)?)? In implementation: The regex follows JEP's definition. The JEP & implementation allows -$OPT to be specified without +, but the Javadoc one does not allow that. For example, "9-pre-opt" is allowed by JEP, but disallowed by Javadoc. > I need to capture the plus to distinguish between cases where an empty build is allowed (e.g. "9+-foo") and when it is not ("9+"). > See code in Version.java, line 226-230 and in Basic.java, line 98, 107-109. (Note that we use the empty "+" to distinguish "9-foo" from "9+-foo".) Understood, but I didn't see any part of the JEP or the Javadoc explaining that + is needed to make the parser recognize the text followed as options instead of pre-release identifier. It would be great if that is added. Best regards, Hong Dai Thanh. -----Original Message----- From: Iris Clark [mailto:iris.clark at oracle.com] Sent: Tuesday, 2 February, 2016 12:51 PM To: Thanh Hong Dai ; Alan Bateman ; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Hong. Thanks again for looking closely at the regular expression and providing comments. > - The outer most quantifier in (((\.0)*\.[1-9][0-9]*)*)* is redundant > and a source of catastrophic backtracking. You only need > ((\.0)*\.[1-9][0-9]*)*. > - The outside - in PRE part (\-([a-zA-Z0-9]+))? doesn't need escaping. > Simplify it to (-([a-zA-Z0-9]+))? Both done. > - Do you want to allow only a plus without the number in the BUILD ? part? Why do you capture the +? ((\+)(0|[1-9][0-9]*)?)? I need to capture the plus to distinguish between cases where an empty build is allowed (e.g. "9+-foo") and when it is not ("9+"). See code in Version.java, line 226-230 and in Basic.java, line 98, 107-109. (Note that we use the empty "+" to distinguish "9-foo" from "9+-foo".) > - Dot loses meaning in the character class, and hyphen loses meaning > at the beginning or at the end of the character class. You can > simplify the OPT part to (-([-a-zA-Z0-9.]+))? Done. > - You might want to consider using named-capturing groups instead of > numbered capturing groups. Also, consider using non-capturing groups > for groups you don't need to extract. Done. This is the (hopefully) final version of the webrev and javadoc for the initial implementation of this API: http://cr.openjdk.java.net/~iris/verona/8072379/webrev.3/index.html http://cr.openjdk.java.net/~iris/verona/8072379/doc.3/jdk/Version.html I've filed the following bug to update the spec: 8148822: (spec) Regex in jdk.Version and JEP 223 should match https://bugs.openjdk.java.net/browse/JDK-8148822 These are the corresponding changes to the JEP which I'll apply in the next day or so: $ diff jep-open-mr11.md jep-open-mr12.md 84c84 < `^[1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*$`. The sequence may be of arbitrary --- > `^[1-9][0-9]*((\.0)*\.[1-9][0-9]*)*$`. The sequence may be of > arbitrary 166c166 < - $OPT, matching `([-a-zA-Z0-9\.]+)` --- Additional --- > - $OPT, matching `([-a-zA-Z0-9.]+)` --- > Additional Thanks again for the recommendations. Regards, iris -----Original Message----- From: Thanh Hong Dai [mailto:hdthanh at tma.com.vn] Sent: Wednesday, January 13, 2016 7:53 PM To: Iris Clark; Alan Bateman; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Some comment on the regex (and also the JEP): ([1-9][0-9]*(((\.0)*\.[1-9][0-9]*)*)*)(\-([a-zA-Z0-9]+))?((\+)(0|[1-9][0-9]*)?)?(-([\-a-zA-Z0-9\.]+))? - The outer most quantifier in (((\.0)*\.[1-9][0-9]*)*)* is redundant and a source of catastrophic backtracking. You only need ((\.0)*\.[1-9][0-9]*)*. - The outside - in PRE part (\-([a-zA-Z0-9]+))? doesn't need escaping. Simplify it to (-([a-zA-Z0-9]+))? - Do you want to allow only a plus without the number in the BUILD part? Why do you capture the +? ((\+)(0|[1-9][0-9]*)?)? - Dot loses meaning in the character class, and hyphen loses meaning at the beginning or at the end of the character class. You can simplify the OPT part to (-([-a-zA-Z0-9.]+))? - You might want to consider using named-capturing groups instead of numbered capturing groups. Also, consider using non-capturing groups for groups you don't need to extract. Best regards, Hong Dai Thanh. -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Iris Clark Sent: Thursday, 14 January, 2016 4:55 AM To: Alan Bateman ; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Alan. Thanks for looking at this (hopefully) one last time. > It can't be java.base (see design principles in JEP 200). > If it's going into java.base temporarily then the top-level > modules.xml will need to be updated to export the "jdk" package. This diff has been applied to modules.xml: diff -r 6fefc5bce180 modules.xml --- a/modules.xml Wed Jan 13 13:56:19 2016 +0000 +++ b/modules.xml Wed Jan 13 13:46:56 2016 -0800 @@ -205,6 +205,9 @@ javax.security.cert + jdk + + jdk.net It essentially reverts your 8049422 change [0] to that file. I will not re-add jdk to the javadoc bundle for javac trees API since that is not an appropriate location. I filed the following bug to track publication of jdk.Version: 8144069: Determine correct publication for jdk.Version API https://bugs.openjdk.java.net/browse/JDK-8144069 When this came up earlier, I filed this bug to track finding a more appropriate module for jdk.Version: 8144062: Determine appropriate module for jdk.Version https://bugs.openjdk.java.net/browse/JDK-8144062 Thanks, iris [0] http://hg.openjdk.java.net/jdk9/dev/rev/1bee5efa73e3 -----Original Message----- From: Alan Bateman Sent: Tuesday, January 12, 2016 7:41 AM To: Iris Clark; core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: Re: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion On 11/01/2016 21:44, Iris Clark wrote: > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter break and > my vacation), I provided responses to your messages and an updated > webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume that > you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037 > 062.html > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036 > 904.html > > I'd like to wrap up this work for the initial implementation of > jdk.Version soon. > I think this looks good but we'll to decide which module to put this in. It can't be java.base (see design principles in JEP 200). If it's going into java.base temporarily then the top-level modules.xml will need to be updated to export the "jdk" package. -Alan From amaembo at gmail.com Tue Feb 2 09:24:20 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Tue, 2 Feb 2016 15:24:20 +0600 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() Message-ID: <1591524905.20160202152420@gmail.com> Please review and sponsor this fix: https://bugs.openjdk.java.net/browse/JDK-8148838 http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r1/ When buffer traversal is already started, and split is requested, then the existing buffer should be carefully transferred to the prefix part. The only problem I see here is the testing time. Due to permuteFunctions() call it increases significantly what I added the flatMap() test. If this becomes an issue, I can write new simple test which tests flatMap() only (without permutations with other intermediate operations). With best regards, Tagir Valeev. From vladimir.x.ivanov at oracle.com Tue Feb 2 10:33:46 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 2 Feb 2016 13:33:46 +0300 Subject: RFR (S) 8148787: StringConcatFactory exactness check produces bad bytecode when a non-arg concat is requested In-Reply-To: <56B05429.2000007@oracle.com> References: <56AFB64B.5040901@oracle.com> <56B05429.2000007@oracle.com> Message-ID: <56B0860A.7080202@oracle.com> Looks good. Please, add -Xverify:all to all tests on string concat. Best regards, Vladimir Ivanov On 2/2/16 10:00 AM, Aleksey Shipilev wrote: > Anyone? This looks like a trivial fix. > > -Aleksey > > On 02/01/2016 10:47 PM, Aleksey Shipilev wrote: >> Hi, >> >> Please review the fix for a corner case in StringConcatFactory exactness >> check, which produces invalid bytecode: >> https://bugs.openjdk.java.net/browse/JDK-8148787 >> >> Note that this happens when all three things align: >> a) BSM is called directly, as Java method -- javac would never produce >> such a String concat shape; >> b) BC_SB_SIZED_EXACT strategy is used, so exactness check can be applied; >> c) -Djava.lang.invoke.stringConcat.debug=true is set, forcing >> exactness check to run; >> >> The issue is with exactness debug check using a temporary local variable >> when the local variable table is small (like it is in non-arg case). The >> code can be reformulated without using temporary variables, with a >> creative use of "swap" instruction. >> >> Ironically, the bug is within the debugging code, so we don't care about >> its performance at all: >> http://cr.openjdk.java.net/~shade/8148787/webrev.01/ >> >> Cheers, >> -Aleksey >> > > From ramanand.patil at oracle.com Tue Feb 2 11:00:06 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Tue, 2 Feb 2016 03:00:06 -0800 (PST) Subject: RFR: 8148446: (tz) Support tzdata2016a In-Reply-To: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> References: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> Message-ID: <7050b3f7-d566-4097-bdcc-9cbf76c272e4@default> HI all, Please review the latest TZDATA integration (tzdata2016a) to JDK9. Bug - https://bugs.openjdk.java.net/browse/JDK-8148446 Webrev - http://cr.openjdk.java.net/~rpatil/8148446/webrev.00/ All the TimeZone related tests are passed after integration. Regards, Ramanand. From peter.levart at gmail.com Tue Feb 2 11:44:04 2016 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 2 Feb 2016 12:44:04 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF819C.5020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> Message-ID: <56B09684.6010107@gmail.com> Hi Roger, Is this public API supposed to replace sun.misc.Signal? I don't see why not. Comparing the APIs, thought they are different in style, I can't find a feature of sun.misc.Signal that wouldn't be supported by java.util.Signal except for NativeSignalHandler which in my understanding is currently only used to implement the SignalHandler.SIG_DFL and SignalHandler.SIG_IGN special "native" handlers. Is there any use of NativeSignalHandler to dispatch to arbitrary native procedure? As to the implementation: Registering signal handlers with this API is protected by a runtime permission. Nevertheless, since this is public API and handlers are supplied by user code, should dispatching to the handlers be performed in a specially prepared Thread with no permissions? Current dispatch implementation constructs new Thread with constructor that uses the AccessController.getContext() inherited from the thread that calls the Signal.dispatch() method which is a special "Signal Dispatcher" thread spawned in the VM (os::signal_init). I haven't dug any deeper, but is it possible that "Signal Dispatcher" thread uses system protection domain? As to the API. Is registerDefault() really needed to be exposed to the public? I can see it is currently used to install default non-native handler(s) for TERM, INT, HUP signals in the boot-up sequence in j.l.System. Do you want users to be able to override these default handlers? Also the new implementation only sets the defaultConsumer field which is used only when some normal handler is unregister-ed, so this does not work correctly... Also if this is to become public API, There's a chance users would want to add a handler to the chain of existing handlers or override them. So what about an API that allows registering/unregistering a default (non-native) handler and other handlers above it in a uniform way, like: public final class Signal { private static final ConcurrentMap SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); private static final ConcurrentMap SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); public static Signal of(String name) { Signal signal = SIGNAL_BY_NAME.get(name); if (signal != null) { return signal; } int number; if (!name.startsWith("SIG") || name.length() <= 3 || (number = findSignal0(name.substring(3))) < 0) { throw new UnsupportedOperationException("Unknown signal: " + name); } signal = SIGNAL_BY_NUMBER.computeIfAbsent( number, new Function() { @Override public Signal apply(Integer number) { return new Signal(name, number); } } ); SIGNAL_BY_NAME.putIfAbsent(name, signal); return signal; } private final String name; private final int number; private volatile HandlerChain handlerChain; private long savedNativeHandler; private Signal(String name, int number) { this.name = name; this.number = number; } public String name() { return name; } public int number() { return number; } public void raise() { raise0(number); } public void register(BiConsumer handler) { synchronized (this) { HandlerChain oldChain = handlerChain; handlerChain = new HandlerChain(handler, oldChain); if (oldChain == null) { // set native to dispatch to Singnal.dispatch() savedNativeHandler = handle1(2); } } } public boolean unregister(BiConsumer handler) { synchronized (this) { HandlerChain oldChain = handlerChain; if (oldChain != null && oldChain.handler == handler) { if (oldChain.next == null) { // restore saved native handler long oldNativeHandler = handle1(savedNativeHandler); assert oldNativeHandler == 2L; } handlerChain = oldChain.next; return true; } else { return false; } } } // following two should probably be hidden from public API public void nativeIgnore() { synchronized (this) { if (handlerChain == null) { handle1(1); // ignore signal } else { throw new IllegalStateException( "Can't ignore signal after handlers have already been registered."); } } } public void nativeDefault() { synchronized (this) { if (handlerChain == null) { handle1(0); // default native handler } else { throw new IllegalStateException( "Can't restore signal after handlers have already been registered."); } } } private long handle1(long nativeHandler) { long oldNativeHandler = handle0(number, nativeHandler); if (oldNativeHandler == -1L) { throw new UnsupportedOperationException( "Signal already used by VM or OS: " + name); } return oldNativeHandler; } /* * Called by the VM to execute Java signal handlers. */ private static void dispatch(int number) { Signal signal = SIGNAL_BY_NUMBER.get(number); if (signal != null) { HandlerChain handlerChain = signal.handlerChain; if (handlerChain != null) { new InnocuousThread(() -> handlerChain.accept(signal)) .start(); } } } /** * Find the signal number, given a name. * * @param sigName the signal name * @return the signal number or -1 for unknown signals. */ private static native int findSignal0(String sigName); /* Registers a native signal handler, and returns the old handler. * Handler values: * 0 default handler * 1 ignore the signal * 2 call back to Signal.dispatch * other arbitrary native signal handlers * @param nativeH the index or address of the new signal handler * @return the previous index or address */ private static native long handle0(int sig, long nativeH); /* * Raise a given signal number. * @param sig the signal number to raise */ private static native void raise0(int sig); private static class HandlerChain implements Consumer { final BiConsumer handler; final HandlerChain next; HandlerChain(BiConsumer handler, HandlerChain next) { this.handler = handler; this.next = next; } @Override public void accept(Signal signal) { handler.accept(signal, () -> { if (next != null) { next.accept(signal); } }); } } } Regards, Peter On 02/01/2016 05:02 PM, Roger Riggs wrote: > Please review an API addition to handle signals such as SIGINT, > SIGHUP, and SIGTERM. > This JEP 260 motivated alternative to sun.misc.Signal supports the use > case for > interactive applications that need to handle Control-C and other signals. > > The new java.util.Signal class provides a settable primary signal > handler and a default > signal handler. The primary signal handler can be unregistered and > handling is restored > to the default signal handler. System initialization registers > default signal handlers > to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API > requires > a permission if a SecurityManager is set. > > The sun.misc.Signal implementation is modified to be layered on a common > thread and dispatch mechanism. The VM handling of native signals is > not affected. > The command option to reduce signal use by the runtime with -Xrs is > unmodified. > > The changes to hotspot are minimal to rename the hardcoded callback to > the Java > Signal dispatcher. > > Please review and comment on the API and implementation. > > javadoc: > http://cr.openjdk.java.net/~rriggs/signal-doc/ > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8087286 > > JEP 260: > https://bugs.openjdk.java.net/browse/JDK-8132928 > > Thanks, Roger > > From peter.levart at gmail.com Tue Feb 2 12:29:13 2016 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 2 Feb 2016 13:29:13 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B09684.6010107@gmail.com> References: <56AF819C.5020905@Oracle.com> <56B09684.6010107@gmail.com> Message-ID: <56B0A119.7020302@gmail.com> Hi Roger, One trick to have special methods like nativeDefault() / nativeIgnore() / nativeRegister(long nativeHandler) (if they are needed at all) hidden from public but be public for internal use is to: Create a public interface in non-exported internal package, like: package jdk.internal.misc; public interface NativeSignal { void nativeIgnore(); void nativeDefault(); void nativeRegister(long nativeHandler); } ...then create a private nested subclass of java.util.Signal inside it, like: public class Signal { ... private static final class NativeImpl extends Signal implements NativeSignal { NativeImpl(String name, int number) { super(name, number); } @Override public void nativeIgnore() { nativeRegister(1L); } @Override public void nativeDefault() { nativeRegister(0L); } @Override public synchronized void nativeRegister(long nativeHandler) { if (handlerChain == null) { handle1(nativeHandler); // set native handler } else { throw new IllegalStateException( "Can't set native handler after Java handlers have already been registered."); } } } } ...and then return instances of Signal.NativeImpl from Signal factory method instead... Internal code can thus cast a Signal instance to NativeSignal and invoke interface methods while external code can not access NativeSignal interface as it is not exported. Here's how it all looks together: public class Signal /* not final but constructor is package-private */ { private static final ConcurrentMap SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); private static final ConcurrentMap SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); public static Signal of(String name) { Signal signal = SIGNAL_BY_NAME.get(name); if (signal != null) { return signal; } int number; if (!name.startsWith("SIG") || name.length() <= 3 || (number = findSignal0(name.substring(3))) < 0) { throw new UnsupportedOperationException("Unknown signal: " + name); } signal = SIGNAL_BY_NUMBER.computeIfAbsent( number, new Function() { @Override public Signal apply(Integer number) { return new Signal.NativeImpl(name, number); } } ); SIGNAL_BY_NAME.putIfAbsent(name, signal); return signal; } private final String name; private final int number; volatile HandlerChain handlerChain; private long savedNativeHandler; Signal(String name, int number) { this.name = name; this.number = number; } public String name() { return name; } public int number() { return number; } public void raise() { raise0(number); } public void register(BiConsumer handler) { synchronized (this) { HandlerChain oldChain = handlerChain; handlerChain = new HandlerChain(handler, oldChain); if (oldChain == null) { // set native to dispatch to Singnal.dispatch() savedNativeHandler = handle1(2); } } } public boolean unregister(BiConsumer handler) { synchronized (this) { HandlerChain oldChain = handlerChain; if (oldChain != null && oldChain.handler == handler) { if (oldChain.next == null) { // restore saved native handler long oldNativeHandler = handle1(savedNativeHandler); assert oldNativeHandler == 2L; } handlerChain = oldChain.next; return true; } else { return false; } } } long handle1(long nativeHandler) { long oldNativeHandler = handle0(number, nativeHandler); if (oldNativeHandler == -1L) { throw new UnsupportedOperationException( "Signal already used by VM or OS: " + name); } return oldNativeHandler; } /* * Called by the VM to execute Java signal handlers. */ private static void dispatch(int number) { Signal signal = SIGNAL_BY_NUMBER.get(number); if (signal != null) { HandlerChain handlerChain = signal.handlerChain; if (handlerChain != null) { new InnocuousThread(() -> handlerChain.accept(signal)) .start(); } } } /** * Find the signal number, given a name. * * @param sigName the signal name * @return the signal number or -1 for unknown signals. */ private static native int findSignal0(String sigName); /* Registers a native signal handler, and returns the old handler. * Handler values: * 0 default handler * 1 ignore the signal * 2 call back to Signal.dispatch * other arbitrary native signal handlers * @param nativeH the index or address of the new signal handler * @return the previous index or address */ private static native long handle0(int sig, long nativeH); /* * Raise a given signal number. * @param sig the signal number to raise */ private static native void raise0(int sig); private static class HandlerChain implements Consumer { final BiConsumer handler; final HandlerChain next; HandlerChain(BiConsumer handler, HandlerChain next) { this.handler = handler; this.next = next; } @Override public void accept(Signal signal) { handler.accept(signal, () -> { if (next != null) { next.accept(signal); } }); } } private static final class NativeImpl extends Signal implements NativeSignal { NativeImpl(String name, int number) { super(name, number); } @Override public void nativeIgnore() { nativeRegister(1L); } @Override public void nativeDefault() { nativeRegister(0L); } @Override public synchronized void nativeRegister(long nativeHandler) { if (handlerChain == null) { handle1(nativeHandler); // set native handler } else { throw new IllegalStateException( "Can't set native handler after Java handlers have already been registered."); } } } } On 02/02/2016 12:44 PM, Peter Levart wrote: > Hi Roger, > > Is this public API supposed to replace sun.misc.Signal? I don't see > why not. Comparing the APIs, thought they are different in style, I > can't find a feature of sun.misc.Signal that wouldn't be supported by > java.util.Signal except for NativeSignalHandler which in my > understanding is currently only used to implement the > SignalHandler.SIG_DFL and SignalHandler.SIG_IGN special "native" > handlers. Is there any use of NativeSignalHandler to dispatch to > arbitrary native procedure? > > As to the implementation: Registering signal handlers with this API is > protected by a runtime permission. Nevertheless, since this is public > API and handlers are supplied by user code, should dispatching to the > handlers be performed in a specially prepared Thread with no > permissions? Current dispatch implementation constructs new Thread > with constructor that uses the AccessController.getContext() inherited > from the thread that calls the Signal.dispatch() method which is a > special "Signal Dispatcher" thread spawned in the VM > (os::signal_init). I haven't dug any deeper, but is it possible that > "Signal Dispatcher" thread uses system protection domain? > > As to the API. Is registerDefault() really needed to be exposed to the > public? I can see it is currently used to install default non-native > handler(s) for TERM, INT, HUP signals in the boot-up sequence in > j.l.System. Do you want users to be able to override these default > handlers? Also the new implementation only sets the defaultConsumer > field which is used only when some normal handler is unregister-ed, so > this does not work correctly... > > Also if this is to become public API, There's a chance users would > want to add a handler to the chain of existing handlers or override > them. So what about an API that allows registering/unregistering a > default (non-native) handler and other handlers above it in a uniform > way, like: > > > public final class Signal { > > private static final ConcurrentMap > SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); > private static final ConcurrentMap > SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); > > public static Signal of(String name) { > Signal signal = SIGNAL_BY_NAME.get(name); > if (signal != null) { > return signal; > } > > int number; > if (!name.startsWith("SIG") || name.length() <= 3 || > (number = findSignal0(name.substring(3))) < 0) { > throw new UnsupportedOperationException("Unknown signal: " > + name); > } > > signal = SIGNAL_BY_NUMBER.computeIfAbsent( > number, > new Function() { > @Override > public Signal apply(Integer number) { > return new Signal(name, number); > } > } > ); > > SIGNAL_BY_NAME.putIfAbsent(name, signal); > > return signal; > } > > private final String name; > private final int number; > private volatile HandlerChain handlerChain; > private long savedNativeHandler; > > private Signal(String name, int number) { > this.name = name; > this.number = number; > } > > public String name() { return name; } > > public int number() { return number; } > > public void raise() { raise0(number); } > > public void register(BiConsumer handler) { > synchronized (this) { > HandlerChain oldChain = handlerChain; > handlerChain = new HandlerChain(handler, oldChain); > if (oldChain == null) { > // set native to dispatch to Singnal.dispatch() > savedNativeHandler = handle1(2); > } > } > } > > public boolean unregister(BiConsumer handler) { > synchronized (this) { > HandlerChain oldChain = handlerChain; > if (oldChain != null && oldChain.handler == handler) { > if (oldChain.next == null) { > // restore saved native handler > long oldNativeHandler = handle1(savedNativeHandler); > assert oldNativeHandler == 2L; > } > handlerChain = oldChain.next; > return true; > } else { > return false; > } > } > } > > // following two should probably be hidden from public API > > public void nativeIgnore() { > synchronized (this) { > if (handlerChain == null) { > handle1(1); // ignore signal > } else { > throw new IllegalStateException( > "Can't ignore signal after handlers have already > been registered."); > } > } > } > > public void nativeDefault() { > synchronized (this) { > if (handlerChain == null) { > handle1(0); // default native handler > } else { > throw new IllegalStateException( > "Can't restore signal after handlers have already > been registered."); > } > } > } > > private long handle1(long nativeHandler) { > long oldNativeHandler = handle0(number, nativeHandler); > if (oldNativeHandler == -1L) { > throw new UnsupportedOperationException( > "Signal already used by VM or OS: " + name); > } > return oldNativeHandler; > } > > /* > * Called by the VM to execute Java signal handlers. > */ > private static void dispatch(int number) { > Signal signal = SIGNAL_BY_NUMBER.get(number); > if (signal != null) { > HandlerChain handlerChain = signal.handlerChain; > if (handlerChain != null) { > new InnocuousThread(() -> handlerChain.accept(signal)) > .start(); > } > } > } > > /** > * Find the signal number, given a name. > * > * @param sigName the signal name > * @return the signal number or -1 for unknown signals. > */ > private static native int findSignal0(String sigName); > > /* Registers a native signal handler, and returns the old handler. > * Handler values: > * 0 default handler > * 1 ignore the signal > * 2 call back to Signal.dispatch > * other arbitrary native signal handlers > * @param nativeH the index or address of the new signal handler > * @return the previous index or address > */ > private static native long handle0(int sig, long nativeH); > > /* > * Raise a given signal number. > * @param sig the signal number to raise > */ > private static native void raise0(int sig); > > > private static class HandlerChain implements Consumer { > final BiConsumer handler; > final HandlerChain next; > > HandlerChain(BiConsumer handler, > HandlerChain next) { > this.handler = handler; > this.next = next; > } > > @Override > public void accept(Signal signal) { > handler.accept(signal, () -> { > if (next != null) { > next.accept(signal); > } > }); > } > } > } > > > > > Regards, Peter > > > On 02/01/2016 05:02 PM, Roger Riggs wrote: >> Please review an API addition to handle signals such as SIGINT, >> SIGHUP, and SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the >> use case for >> interactive applications that need to handle Control-C and other >> signals. >> >> The new java.util.Signal class provides a settable primary signal >> handler and a default >> signal handler. The primary signal handler can be unregistered and >> handling is restored >> to the default signal handler. System initialization registers >> default signal handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >> requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is >> not affected. >> The command option to reduce signal use by the runtime with -Xrs is >> unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback >> to the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> > From aleksey.shipilev at oracle.com Tue Feb 2 13:13:18 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 2 Feb 2016 16:13:18 +0300 Subject: RFR (S) 8148787: StringConcatFactory exactness check produces bad bytecode when a non-arg concat is requested In-Reply-To: <56B0860A.7080202@oracle.com> References: <56AFB64B.5040901@oracle.com> <56B05429.2000007@oracle.com> <56B0860A.7080202@oracle.com> Message-ID: <56B0AB6E.6000009@oracle.com> On 02/02/2016 01:33 PM, Vladimir Ivanov wrote: > Looks good. Thanks Vladimir! > Please, add -Xverify:all to all tests on string concat. There is little need: all string concat tests are calling String concat stubs from non-privileged callers, and hence are verified. E.g. new regression test does fail with verification error even without -Xverify:all. But it is a good idea to force verification in concat tests just in case that ever fails: http://cr.openjdk.java.net/~shade/8148787/webrev.02/ This new patch flies through full JPRT without a hitch. Cheers, -Aleksey > Best regards, > Vladimir Ivanov > > On 2/2/16 10:00 AM, Aleksey Shipilev wrote: >> Anyone? This looks like a trivial fix. >> >> -Aleksey >> >> On 02/01/2016 10:47 PM, Aleksey Shipilev wrote: >>> Hi, >>> >>> Please review the fix for a corner case in StringConcatFactory exactness >>> check, which produces invalid bytecode: >>> https://bugs.openjdk.java.net/browse/JDK-8148787 >>> >>> Note that this happens when all three things align: >>> a) BSM is called directly, as Java method -- javac would never >>> produce >>> such a String concat shape; >>> b) BC_SB_SIZED_EXACT strategy is used, so exactness check can be >>> applied; >>> c) -Djava.lang.invoke.stringConcat.debug=true is set, forcing >>> exactness check to run; >>> >>> The issue is with exactness debug check using a temporary local variable >>> when the local variable table is small (like it is in non-arg case). The >>> code can be reformulated without using temporary variables, with a >>> creative use of "swap" instruction. >>> >>> Ironically, the bug is within the debugging code, so we don't care about >>> its performance at all: >>> http://cr.openjdk.java.net/~shade/8148787/webrev.01/ >>> >>> Cheers, >>> -Aleksey >>> >> >> From chris.hegarty at oracle.com Tue Feb 2 13:44:01 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Feb 2016 13:44:01 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AFADC0.50305@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56AF8C2F.2010003@oracle.com> <56AFADC0.50305@Oracle.com> Message-ID: <1A9E1057-8673-4197-AD4A-869A8BB00CA1@oracle.com> On 1 Feb 2016, at 19:10, Roger Riggs wrote: > Hi Chris, > > On 2/1/2016 11:47 AM, Chris Hegarty wrote: >> Thank you for taking this on Roger. >> >> An initial question to help my understanding. [ I think I need to >> understand this, before I can comment further on the API ] >> >> I'm a little confused about how the default handler is supposed to >> work, so I looked at the implementation and become even more confused. >> The consumer is registered on a per instance basis, and the instance >> is added to the static map when it is created. So it appears that the >> registered handler is not dependent on when the last register() is >> called, but when the last instance of a Signal for a specific signal >> is created. Is this intended? > The Signal instance delegates to a singleton SignalImpl instance > (based on the name/number). I chose to delegate from Signal > to SignalImpl to be able to put SignalImpl in a separate package that > could be exported to the implementation of sun.misc.Signal as needed. > Otherwise, java.util.Signal would only be able to export only its public interface > and s.m.Signal needs more functions to be available. > > The current and default consumers are stored in the singleton SignalImpl objects > that are used for the dispatch and calling of the consumer. > > The SignalImpl instances are create on first use. Each call to register > re-registers the SignalImpl with the VM to ensure it will be the current handler. Thanks Roger. I see this now ( glasses cleaned ;-) ). The interaction between Signal and SignalImpl initially confused me. Signal.of always returns a new instance of Signal. I wonder if it is worth, somewhat, simplify this, with the use of a SharedSecret. Signal: ? static { SharedSecrets.setJavaUtilSignalAccess( new JavaUtilSignalAccess() { public Signal newSignal(SignalImpl impl) { return new Signal(impl); } } ); } /** Private constructor. */ private Signal(SignalImpl impl) { this.impl = impl; } public static Signal of(String name) { Objects.requireNonNull(name, "name"); return SignalImpl.of(name).signal(); } SignalImpl: ? private SignalImpl(String name, int number) { this.name = name; this.number = number; signal = SharedSecrets.getJavaUtilSignalAccess().newSignal(this); } public static SignalImpl of(String name) { SecurityManager sm = System.getSecurityManager(); if (sm != null) { sm.checkPermission(new RuntimePermission("handleSignal")); } synchronized (nameToSignal) { // Note: Terminator initialization is too early to use lambda with putIfAbsent SignalImpl impl = nameToSignal.get(name); if (impl == null) { int number; if (name.length() < 3 || (number = findSignal0(name.substring(3))) < 0) { throw new UnsupportedOperationException("Unknown signal: " + name); } impl = new SignalImpl(name, number); nameToSignal.put(name, impl); numberToSignal.put(impl.number, impl); } return impl; } } public Signal signal() { return signal; } There just seems to be some tension between these types, but maybe that is coming, at least somewhat, from sun.misc.Signal. -Chris. > Roger > > >> >> -Chris. >> >> On 01/02/16 16:02, Roger Riggs wrote: >>> Please review an API addition to handle signals such as SIGINT, SIGHUP, >>> and SIGTERM. >>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>> case for >>> interactive applications that need to handle Control-C and other signals. >>> >>> The new java.util.Signal class provides a settable primary signal >>> handler and a default >>> signal handler. The primary signal handler can be unregistered and >>> handling is restored >>> to the default signal handler. System initialization registers default >>> signal handlers >>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>> requires >>> a permission if a SecurityManager is set. >>> >>> The sun.misc.Signal implementation is modified to be layered on a common >>> thread and dispatch mechanism. The VM handling of native signals is not >>> affected. >>> The command option to reduce signal use by the runtime with -Xrs is >>> unmodified. >>> >>> The changes to hotspot are minimal to rename the hardcoded callback to >>> the Java >>> Signal dispatcher. >>> >>> Please review and comment on the API and implementation. >>> >>> javadoc: >>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>> >>> Webrev: >>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>> >>> JEP 260: >>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> Thanks, Roger >>> >>> > From chris.hegarty at oracle.com Tue Feb 2 13:58:55 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Feb 2016 13:58:55 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF819C.5020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> Message-ID: I think this API should be a good replacement for sun.misc.Signal. Some comments: - The second @implNote, referring to s.m.Signal, I assume can be removed? - I agree with Peter, about the permission regarding the thread that executes the consumer. Does it make sense to support a ThreadFactory, or may that is overkill. - Is it necessary for ?raise' to declare that it may throw UOE? Since the Signal and impl are already retrieved. Or maybe there is another possibility of failure? - Signal.dispatch just calls SignalImpl.dispatch. Should the VM just do this directly? - unregister is based on the consumer, possibly lambda?s, identity ? This seems strange. - I?m still confused by the registerDefault, and I?m not sure if it is actually needed. Why can Terminator not just register as normal? - Should the instance methods that register/unregister/raise perform a security check ( in case the Signal escapes, for example in the consumer ) -Chris. On 1 Feb 2016, at 16:02, Roger Riggs wrote: > Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. > This JEP 260 motivated alternative to sun.misc.Signal supports the use case for > interactive applications that need to handle Control-C and other signals. > > The new java.util.Signal class provides a settable primary signal handler and a default > signal handler. The primary signal handler can be unregistered and handling is restored > to the default signal handler. System initialization registers default signal handlers > to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires > a permission if a SecurityManager is set. > > The sun.misc.Signal implementation is modified to be layered on a common > thread and dispatch mechanism. The VM handling of native signals is not affected. > The command option to reduce signal use by the runtime with -Xrs is unmodified. > > The changes to hotspot are minimal to rename the hardcoded callback to the Java > Signal dispatcher. > > Please review and comment on the API and implementation. > > javadoc: > http://cr.openjdk.java.net/~rriggs/signal-doc/ > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8087286 > > JEP 260: > https://bugs.openjdk.java.net/browse/JDK-8132928 > > Thanks, Roger > > From vladimir.x.ivanov at oracle.com Tue Feb 2 14:24:11 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 2 Feb 2016 17:24:11 +0300 Subject: RFR (S) 8148787: StringConcatFactory exactness check produces bad bytecode when a non-arg concat is requested In-Reply-To: <56B0AB6E.6000009@oracle.com> References: <56AFB64B.5040901@oracle.com> <56B05429.2000007@oracle.com> <56B0860A.7080202@oracle.com> <56B0AB6E.6000009@oracle.com> Message-ID: <56B0BC0B.50601@oracle.com> > But it is a good idea to force verification in concat tests just in case > that ever fails: > http://cr.openjdk.java.net/~shade/8148787/webrev.02/ Still looks good. Thanks for adding the flag. Best regards, Vladimir Ivanov From chris.hegarty at oracle.com Tue Feb 2 14:37:41 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Feb 2016 14:37:41 +0000 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: Message-ID: On 1 Feb 2016, at 18:45, Martin Buchholz wrote: > After much debate on what to do when CompleteableFuture.whenComplete > encounters an exception in both the source and the action, we chose > what was acceptable to the most people - add the action's exception to > the source exception as a suppressed exception. And added usage > guidelines. And gave handle "top billing" over whenComplete. > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ This all looks fine to me. So I assume you only need a small CCC request for CompletionStage, right? Everything else is implementation. -Chris. From paul.sandoz at oracle.com Tue Feb 2 14:57:43 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Feb 2016 14:57:43 +0000 Subject: RFR (S) 8148787: StringConcatFactory exactness check produces bad bytecode when a non-arg concat is requested In-Reply-To: <56B05429.2000007@oracle.com> References: <56AFB64B.5040901@oracle.com> <56B05429.2000007@oracle.com> Message-ID: > On 2 Feb 2016, at 07:00, Aleksey Shipilev wrote: > > Anyone? This looks like a trivial fix. > +1. The comment on stack logic is really helpful. Paul. > -Aleksey > > On 02/01/2016 10:47 PM, Aleksey Shipilev wrote: >> Hi, >> >> Please review the fix for a corner case in StringConcatFactory exactness >> check, which produces invalid bytecode: >> https://bugs.openjdk.java.net/browse/JDK-8148787 >> >> Note that this happens when all three things align: >> a) BSM is called directly, as Java method -- javac would never produce >> such a String concat shape; >> b) BC_SB_SIZED_EXACT strategy is used, so exactness check can be applied; >> c) -Djava.lang.invoke.stringConcat.debug=true is set, forcing >> exactness check to run; >> >> The issue is with exactness debug check using a temporary local variable >> when the local variable table is small (like it is in non-arg case). The >> code can be reformulated without using temporary variables, with a >> creative use of "swap" instruction. >> >> Ironically, the bug is within the debugging code, so we don't care about >> its performance at all: >> http://cr.openjdk.java.net/~shade/8148787/webrev.01/ >> >> Cheers, >> -Aleksey >> > > From paul.sandoz at oracle.com Tue Feb 2 15:10:30 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Feb 2016 15:10:30 +0000 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <1591524905.20160202152420@gmail.com> References: <1591524905.20160202152420@gmail.com> Message-ID: <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> Hi Tagir, Nice find. My inclination is to keep this simple and not support splitting after partial traversal. Sometimes splitting after partial traversal might be more complex not support (e.g. ArrayList). In other cases it?s more complex to support and in such cases i would argue it is not worth it since this kind of functionality is an edge case. Testing wise you are right to be concerned about the increase in test execution time. The lack of flatMap is definitely an omission. In this case I think it is ok to replace map with flatMap, as both result in stuff going into the holding buffer, and we can use the smaller data sets, e.g. "StreamTestData.small?, that were recently added. Paul. > On 2 Feb 2016, at 09:24, Tagir F. Valeev wrote: > > Please review and sponsor this fix: > > https://bugs.openjdk.java.net/browse/JDK-8148838 > http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r1/ > > When buffer traversal is already started, and split is requested, then > the existing buffer should be carefully transferred to the prefix > part. > > The only problem I see here is the testing time. Due to > permuteFunctions() call it increases significantly what I added the > flatMap() test. If this becomes an issue, I can write new simple test > which tests flatMap() only (without permutations with other > intermediate operations). > > With best regards, > Tagir Valeev. > From Roger.Riggs at Oracle.com Tue Feb 2 15:16:41 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 10:16:41 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B09684.6010107@gmail.com> References: <56AF819C.5020905@Oracle.com> <56B09684.6010107@gmail.com> Message-ID: <56B0C859.8070504@Oracle.com> Hi Peter, On 2/2/2016 6:44 AM, Peter Levart wrote: > Hi Roger, > > Is this public API supposed to replace sun.misc.Signal? I don't see > why not. Comparing the APIs, thought they are different in style, I > can't find a feature of sun.misc.Signal that wouldn't be supported by > java.util.Signal except for NativeSignalHandler which in my > understanding is currently only used to implement the > SignalHandler.SIG_DFL and SignalHandler.SIG_IGN special "native" > handlers. Is there any use of NativeSignalHandler to dispatch to > arbitrary native procedure? I don't think so and have not included it in the replacement API. > > As to the implementation: Registering signal handlers with this API is > protected by a runtime permission. Nevertheless, since this is public > API and handlers are supplied by user code, should dispatching to the > handlers be performed in a specially prepared Thread with no > permissions? Current dispatch implementation constructs new Thread > with constructor that uses the AccessController.getContext() inherited > from the thread that calls the Signal.dispatch() method which is a > special "Signal Dispatcher" thread spawned in the VM > (os::signal_init). I haven't dug any deeper, but is it possible that > "Signal Dispatcher" thread uses system protection domain? yes, the spawned thread should follow the typical use of InnocuousThread. The Signal Dispatcher thread has very limited code it executes and does not need any privs either. Will check. > > As to the API. Is registerDefault() really needed to be exposed to the > public? I can see it is currently used to install default non-native > handler(s) for TERM, INT, HUP signals in the boot-up sequence in > j.l.System. Do you want users to be able to override these default > handlers? There is not a strong use case and it can be removed from the public API. > Also the new implementation only sets the defaultConsumer field which > is used only when some normal handler is unregister-ed, so this does > not work correctly... WIll fix. > > Also if this is to become public API, There's a chance users would > want to add a handler to the chain of existing handlers or override > them. So what about an API that allows registering/unregistering a > default (non-native) handler and other handlers above it in a uniform > way, like: The problem with chaining, as in the current API, is that there is no way to know what the next handler in the chain will do. If it is the default one for INT, TERM, HUP, it will call Shutdown and exit. So without extra information and cooperation chaining is risky. If the handler knows something about the other actors in the environment, it can coordinate with them directly. For the use cases that have been raised for existing use of sun.misc.Signal, they are simple interactive environments that want to give the appearance of being able to interrupt using control-c. I've been aiming for the simplest API that would support the current use cases. Thanks, Roger > > > public final class Signal { > > private static final ConcurrentMap > SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); > private static final ConcurrentMap > SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); > > public static Signal of(String name) { > Signal signal = SIGNAL_BY_NAME.get(name); > if (signal != null) { > return signal; > } > > int number; > if (!name.startsWith("SIG") || name.length() <= 3 || > (number = findSignal0(name.substring(3))) < 0) { > throw new UnsupportedOperationException("Unknown signal: " > + name); > } > > signal = SIGNAL_BY_NUMBER.computeIfAbsent( > number, > new Function() { > @Override > public Signal apply(Integer number) { > return new Signal(name, number); > } > } > ); > > SIGNAL_BY_NAME.putIfAbsent(name, signal); > > return signal; > } > > private final String name; > private final int number; > private volatile HandlerChain handlerChain; > private long savedNativeHandler; > > private Signal(String name, int number) { > this.name = name; > this.number = number; > } > > public String name() { return name; } > > public int number() { return number; } > > public void raise() { raise0(number); } > > public void register(BiConsumer handler) { > synchronized (this) { > HandlerChain oldChain = handlerChain; > handlerChain = new HandlerChain(handler, oldChain); > if (oldChain == null) { > // set native to dispatch to Singnal.dispatch() > savedNativeHandler = handle1(2); > } > } > } > > public boolean unregister(BiConsumer handler) { > synchronized (this) { > HandlerChain oldChain = handlerChain; > if (oldChain != null && oldChain.handler == handler) { > if (oldChain.next == null) { > // restore saved native handler > long oldNativeHandler = handle1(savedNativeHandler); > assert oldNativeHandler == 2L; > } > handlerChain = oldChain.next; > return true; > } else { > return false; > } > } > } > > // following two should probably be hidden from public API > > public void nativeIgnore() { > synchronized (this) { > if (handlerChain == null) { > handle1(1); // ignore signal > } else { > throw new IllegalStateException( > "Can't ignore signal after handlers have already > been registered."); > } > } > } > > public void nativeDefault() { > synchronized (this) { > if (handlerChain == null) { > handle1(0); // default native handler > } else { > throw new IllegalStateException( > "Can't restore signal after handlers have already > been registered."); > } > } > } > > private long handle1(long nativeHandler) { > long oldNativeHandler = handle0(number, nativeHandler); > if (oldNativeHandler == -1L) { > throw new UnsupportedOperationException( > "Signal already used by VM or OS: " + name); > } > return oldNativeHandler; > } > > /* > * Called by the VM to execute Java signal handlers. > */ > private static void dispatch(int number) { > Signal signal = SIGNAL_BY_NUMBER.get(number); > if (signal != null) { > HandlerChain handlerChain = signal.handlerChain; > if (handlerChain != null) { > new InnocuousThread(() -> handlerChain.accept(signal)) > .start(); > } > } > } > > /** > * Find the signal number, given a name. > * > * @param sigName the signal name > * @return the signal number or -1 for unknown signals. > */ > private static native int findSignal0(String sigName); > > /* Registers a native signal handler, and returns the old handler. > * Handler values: > * 0 default handler > * 1 ignore the signal > * 2 call back to Signal.dispatch > * other arbitrary native signal handlers > * @param nativeH the index or address of the new signal handler > * @return the previous index or address > */ > private static native long handle0(int sig, long nativeH); > > /* > * Raise a given signal number. > * @param sig the signal number to raise > */ > private static native void raise0(int sig); > > > private static class HandlerChain implements Consumer { > final BiConsumer handler; > final HandlerChain next; > > HandlerChain(BiConsumer handler, > HandlerChain next) { > this.handler = handler; > this.next = next; > } > > @Override > public void accept(Signal signal) { > handler.accept(signal, () -> { > if (next != null) { > next.accept(signal); > } > }); > } > } > } > > > > > Regards, Peter > > > On 02/01/2016 05:02 PM, Roger Riggs wrote: >> Please review an API addition to handle signals such as SIGINT, >> SIGHUP, and SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the >> use case for >> interactive applications that need to handle Control-C and other >> signals. >> >> The new java.util.Signal class provides a settable primary signal >> handler and a default >> signal handler. The primary signal handler can be unregistered and >> handling is restored >> to the default signal handler. System initialization registers >> default signal handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >> requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is >> not affected. >> The command option to reduce signal use by the runtime with -Xrs is >> unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback >> to the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> > From aleksey.shipilev at oracle.com Tue Feb 2 15:16:58 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 2 Feb 2016 18:16:58 +0300 Subject: RFR (S) 8148787: StringConcatFactory exactness check produces bad bytecode when a non-arg concat is requested In-Reply-To: References: <56AFB64B.5040901@oracle.com> <56B05429.2000007@oracle.com> Message-ID: <56B0C86A.1020409@oracle.com> On 02/02/2016 05:57 PM, Paul Sandoz wrote: >> On 2 Feb 2016, at 07:00, Aleksey Shipilev wrote: >> Anyone? This looks like a trivial fix. > > +1. The comment on stack logic is really helpful. Thank you, Vladimir and Paul, pushed. Cheers, -Aleksey From Roger.Riggs at Oracle.com Tue Feb 2 15:18:29 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 10:18:29 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <1A9E1057-8673-4197-AD4A-869A8BB00CA1@oracle.com> References: <56AF819C.5020905@Oracle.com> <56AF8C2F.2010003@oracle.com> <56AFADC0.50305@Oracle.com> <1A9E1057-8673-4197-AD4A-869A8BB00CA1@oracle.com> Message-ID: <56B0C8C5.5060509@Oracle.com> Hi Chris, On 2/2/2016 8:44 AM, Chris Hegarty wrote: > On 1 Feb 2016, at 19:10, Roger Riggs wrote: > >> Hi Chris, >> >> On 2/1/2016 11:47 AM, Chris Hegarty wrote: >>> Thank you for taking this on Roger. >>> >>> An initial question to help my understanding. [ I think I need to >>> understand this, before I can comment further on the API ] >>> >>> I'm a little confused about how the default handler is supposed to >>> work, so I looked at the implementation and become even more confused. >>> The consumer is registered on a per instance basis, and the instance >>> is added to the static map when it is created. So it appears that the >>> registered handler is not dependent on when the last register() is >>> called, but when the last instance of a Signal for a specific signal >>> is created. Is this intended? >> The Signal instance delegates to a singleton SignalImpl instance >> (based on the name/number). I chose to delegate from Signal >> to SignalImpl to be able to put SignalImpl in a separate package that >> could be exported to the implementation of sun.misc.Signal as needed. >> Otherwise, java.util.Signal would only be able to export only its public interface >> and s.m.Signal needs more functions to be available. >> >> The current and default consumers are stored in the singleton SignalImpl objects >> that are used for the dispatch and calling of the consumer. >> >> The SignalImpl instances are create on first use. Each call to register >> re-registers the SignalImpl with the VM to ensure it will be the current handler. > Thanks Roger. I see this now ( glasses cleaned ;-) ). The interaction between > Signal and SignalImpl initially confused me. > > Signal.of always returns a new instance of Signal. I wonder if it is worth, somewhat, > simplify this, with the use of a SharedSecret. Shared Secret always seems like a hack to me and I would hope it can be phased out with module private access. I'll take another look and at Pete's suggestion for an interface and a subclass. Thanks, Roger > > Signal: > ? > static { > SharedSecrets.setJavaUtilSignalAccess( > new JavaUtilSignalAccess() { > public Signal newSignal(SignalImpl impl) { > return new Signal(impl); > } > } > ); > } > > /** Private constructor. */ > private Signal(SignalImpl impl) { > this.impl = impl; > } > > public static Signal of(String name) { > Objects.requireNonNull(name, "name"); > return SignalImpl.of(name).signal(); > } > > > SignalImpl: > > ? > private SignalImpl(String name, int number) { > this.name = name; > this.number = number; > signal = SharedSecrets.getJavaUtilSignalAccess().newSignal(this); > } > > public static SignalImpl of(String name) { > SecurityManager sm = System.getSecurityManager(); > if (sm != null) { > sm.checkPermission(new RuntimePermission("handleSignal")); > } > > synchronized (nameToSignal) { > // Note: Terminator initialization is too early to use lambda with putIfAbsent > SignalImpl impl = nameToSignal.get(name); > if (impl == null) { > int number; > if (name.length() < 3 || > (number = findSignal0(name.substring(3))) < 0) { > throw new UnsupportedOperationException("Unknown signal: " + name); > } > > impl = new SignalImpl(name, number); > nameToSignal.put(name, impl); > numberToSignal.put(impl.number, impl); > } > return impl; > } > } > > public Signal signal() { return signal; } > > > There just seems to be some tension between these types, but maybe that is coming, > at least somewhat, from sun.misc.Signal. > > -Chris. > >> Roger >> >> >>> -Chris. >>> >>> On 01/02/16 16:02, Roger Riggs wrote: >>>> Please review an API addition to handle signals such as SIGINT, SIGHUP, >>>> and SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>>> case for >>>> interactive applications that need to handle Control-C and other signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal >>>> handler and a default >>>> signal handler. The primary signal handler can be unregistered and >>>> handling is restored >>>> to the default signal handler. System initialization registers default >>>> signal handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>> requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a common >>>> thread and dispatch mechanism. The VM handling of native signals is not >>>> affected. >>>> The command option to reduce signal use by the runtime with -Xrs is >>>> unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback to >>>> the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> Webrev: >>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> JEP 260: >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> Thanks, Roger >>>> >>>> From martinrb at google.com Tue Feb 2 15:23:32 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 2 Feb 2016 07:23:32 -0800 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: Message-ID: On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty wrote: > > On 1 Feb 2016, at 18:45, Martin Buchholz wrote: > >> After much debate on what to do when CompleteableFuture.whenComplete >> encounters an exception in both the source and the action, we chose >> what was acceptable to the most people - add the action's exception to >> the source exception as a suppressed exception. And added usage >> guidelines. And gave handle "top billing" over whenComplete. >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ > > This all looks fine to me. > > So I assume you only need a small CCC request for CompletionStage, right? > Everything else is implementation. If you squint you might argue that CompletionStage's contract hasn't actually changed, but yeah, go ahead and do a CCC for CompletionStage. Publishing a specdiff would be nice - method reordering (for "top billing") has made the diffs harder to review. Thanks. From Roger.Riggs at Oracle.com Tue Feb 2 15:31:02 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 10:31:02 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: References: <56AF819C.5020905@Oracle.com> Message-ID: <56B0CBB6.9080903@Oracle.com> Hi Chris, On 2/2/2016 8:58 AM, Chris Hegarty wrote: > I think this API should be a good replacement for sun.misc.Signal. > > Some comments: > > - The second @implNote, referring to s.m.Signal, I assume can be removed? yes, except that sun.misc.Signal is still around for 9; there there should be note somewhere about the interactions, perhaps it should be in s.m.Signal instead. > > - I agree with Peter, about the permission regarding the thread that executes > the consumer. Does it make sense to support a ThreadFactory, or may that > is overkill. Using InnocuousThread should be sufficient. Adding a ThreadFactory to the register method would give more control but is probably more than necessary. > > - Is it necessary for ?raise' to declare that it may throw UOE? Since the Signal > and impl are already retrieved. Or maybe there is another possibility of failure? As currently implemented in the VM, some signals can be found and handled but not raised. So UOE might occur on raise even if the signal can be handled. > > - Signal.dispatch just calls SignalImpl.dispatch. Should the VM just do this > directly? Yes, but since the VM has a hard coded class/method entry point; I thought it would be be lower maintenance to keep it in a well known class. (I'm open to other registration mechanisms for the callback too). I suspect the VM folks want to own that initial Signal Dispatch thread or I would propose just a blocking call to native to get the next signal; it returns when there is a signal to deliver. But that change can be factored out for later if it is a good idea. > > - unregister is based on the consumer, possibly lambda?s, identity ? This seems > strange. Unless I'm mistaken, lambda's have identity and if the caller needs to unregister it, they would need to save the reference. Using an explicit class implementing Consumer would ensure that. > > - I?m still confused by the registerDefault, and I?m not sure if it is actually needed. > Why can Terminator not just register as normal? The next register would overwrite the handler and the implementation would not know that that handler should be the one restored when a client (like jshell) unregisters its control-c handler. > > - Should the instance methods that register/unregister/raise perform a > security check ( in case the Signal escapes, for example in the consumer ) The Signal is a capability, if you have a Signal reference you have the power; similar to Process and ProcessHandle the security check is done to get the reference to begin with. From an auditing point of view, there is no ambiguity in code that has access to a Signal reference; if has the reference it can control the process. (I haven't done a review with the security folks on this yet). Thanks, Roger > > -Chris. > > On 1 Feb 2016, at 16:02, Roger Riggs wrote: > >> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >> interactive applications that need to handle Control-C and other signals. >> >> The new java.util.Signal class provides a settable primary signal handler and a default >> signal handler. The primary signal handler can be unregistered and handling is restored >> to the default signal handler. System initialization registers default signal handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is not affected. >> The command option to reduce signal use by the runtime with -Xrs is unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback to the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> From Roger.Riggs at Oracle.com Tue Feb 2 16:05:46 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 11:05:46 -0500 Subject: JDK 9 RFR of JDK-8148653: Problem list B5086147.java on windows In-Reply-To: <56AC43B9.8030701@oracle.com> References: <56AC43B9.8030701@oracle.com> Message-ID: <56B0D3DA.8080000@Oracle.com> Hi Joe, Looks ok. Roger On 1/30/2016 12:01 AM, joe darcy wrote: > The test > > java/net/URL/B5086147.java > > has been failing on windows since the changesets for JDK-8147462 and > JDK-8145104 were pushed. The test should be problem listed until the > root cause of JDK-8148626 is found and fixed. > > Please review the patch to do this below. > > Thanks, > > -Joe > > diff -r 3bce90b8839e test/ProblemList.txt > --- a/test/ProblemList.txt Fri Jan 29 17:03:17 2016 -0800 > +++ b/test/ProblemList.txt Fri Jan 29 21:01:06 2016 -0800 > @@ -180,6 +180,9 @@ > # 7143960 > java/net/DatagramSocket/SendDatagramToBadAddress.java macosx-all > > +# 8148626 > +java/net/URL/B5086147.java windows-all > + > ############################################################################ > > > # jdk_nio > From chris.hegarty at oracle.com Tue Feb 2 16:12:46 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Feb 2016 16:12:46 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B0CBB6.9080903@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56B0CBB6.9080903@Oracle.com> Message-ID: Roger, On 2 Feb 2016, at 15:31, Roger Riggs wrote: > Hi Chris, > > On 2/2/2016 8:58 AM, Chris Hegarty wrote: >> I think this API should be a good replacement for sun.misc.Signal. >> >> Some comments: >> >> - The second @implNote, referring to s.m.Signal, I assume can be removed? >> > yes, except that sun.misc.Signal is still around for 9; there there should be note somewhere about the > interactions, perhaps it should be in s.m.Signal instead. Yes, I think it should be in s.m.Signal. >> - I agree with Peter, about the permission regarding the thread that executes >> the consumer. Does it make sense to support a ThreadFactory, or may that >> is overkill. >> > Using InnocuousThread should be sufficient. Right, but then it should be specified. > Adding a ThreadFactory to the register method would give more control but is probably more than necessary. Hmmm? maybe. But then it is very restrictive, unless there is some mechanism to override it. >> - Is it necessary for ?raise' to declare that it may throw UOE? Since the Signal >> and impl are already retrieved. Or maybe there is another possibility of failure? >> > As currently implemented in the VM, some signals can be found and handled but not raised. > So UOE might occur on raise even if the signal can be handled. Ok, thanks. >> - Signal.dispatch just calls SignalImpl.dispatch. Should the VM just do this >> directly? >> > Yes, but since the VM has a hard coded class/method entry point; I thought it would be be lower > maintenance to keep it in a well known class. (I'm open to other registration mechanisms for the callback too). > I suspect the VM folks want to own that initial Signal Dispatch thread or I would propose just a blocking call to native > to get the next signal; it returns when there is a signal to deliver. But that change can be factored out for later > if it is a good idea. Agreed. >> - unregister is based on the consumer, possibly lambda?s, identity ? This seems >> strange. >> > Unless I'm mistaken, lambda's have identity and if the caller needs to unregister it, they would need to save > the reference. Using an explicit class implementing Consumer would ensure that. Yes, they do, I?m just questioning whether this is what we want. >> - I?m still confused by the registerDefault, and I?m not sure if it is actually needed. >> Why can Terminator not just register as normal? >> > The next register would overwrite the handler and the implementation would not know that > that handler should be the one restored when a client (like jshell) unregisters its control-c handler. So the use case is to register a handler and later restore the previously set handler. It is not possible for register to return the previously registered handler, and then it is up the user to store it and reset it, as necessary. Rather than the default notion. >> - Should the instance methods that register/unregister/raise perform a >> security check ( in case the Signal escapes, for example in the consumer ) >> > The Signal is a capability, It is in the new API, but not in s.m.Signal ( it is just pair of string name and number ). > if you have a Signal reference you have the power; similar to Process and ProcessHandle > the security check is done to get the reference to begin with. From an auditing point of view, > there is no ambiguity in code that has access to a Signal reference; if has the reference it can control > the process. There is a potential issue when the Signal is delivered to the consumer when it is raised, if, there is any thread, other than an innocuous thread, executing the consumer. Another small implementation question: is it still necessary for s.m.Signal.dispatch to start a new Thread ? -Chris. > (I haven't done a review with the security folks on this yet). > > Thanks, Roger > > >> >> -Chris. >> >> On 1 Feb 2016, at 16:02, Roger Riggs >> >> wrote: >> >> >>> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >>> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >>> interactive applications that need to handle Control-C and other signals. >>> >>> The new java.util.Signal class provides a settable primary signal handler and a default >>> signal handler. The primary signal handler can be unregistered and handling is restored >>> to the default signal handler. System initialization registers default signal handlers >>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >>> a permission if a SecurityManager is set. >>> >>> The sun.misc.Signal implementation is modified to be layered on a common >>> thread and dispatch mechanism. The VM handling of native signals is not affected. >>> The command option to reduce signal use by the runtime with -Xrs is unmodified. >>> >>> The changes to hotspot are minimal to rename the hardcoded callback to the Java >>> Signal dispatcher. >>> >>> Please review and comment on the API and implementation. >>> >>> javadoc: >>> >>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>> >>> >>> Webrev: >>> jdk: >>> http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>> >>> hotspot: >>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>> >>> >>> Issue: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>> >>> >>> JEP 260: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> >>> Thanks, Roger >>> >>> >>> > From peter.levart at gmail.com Tue Feb 2 16:14:21 2016 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 2 Feb 2016 17:14:21 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B0C859.8070504@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56B09684.6010107@gmail.com> <56B0C859.8070504@Oracle.com> Message-ID: <56B0D5DD.3070501@gmail.com> Hi Roger, On 02/02/2016 04:16 PM, Roger Riggs wrote: > Hi Peter, > > On 2/2/2016 6:44 AM, Peter Levart wrote: >> > ... >> >> Also if this is to become public API, There's a chance users would >> want to add a handler to the chain of existing handlers or override >> them. So what about an API that allows registering/unregistering a >> default (non-native) handler and other handlers above it in a uniform >> way, like: > The problem with chaining, as in the current API, is that there is no > way to know what the next > handler in the chain will do. If it is the default one for INT, TERM, > HUP, it will call Shutdown and exit. > So without extra information and cooperation chaining is risky. > If the handler knows something about the other actors in the > environment, it can coordinate with them directly. > For the use cases that have been raised for existing use of > sun.misc.Signal, they are simple interactive > environments that want to give the appearance of being able to > interrupt using control-c. > > I've been aiming for the simplest API that would support the current > use cases. I noticed that sun.misc.Signal[Handler] is a critical API according to JEP 260, so it can't be removed in JDK9 yet. I wanted to see if sun.misc.Signal[Handler] could be modified to use java.util.Signal directly while making java.util.Signal support chaining and restoring of previous handler on deregister. Here's what it looks like in code: http://cr.openjdk.java.net/~plevart/jdk9-dev/Signal/webrev.01/ But if chaining is not desired, then at least restoring of previous handler could be implemented in a uniform way and for arbitrary registration depth (no need for register/registerDefault distinction). What do you think? Regards, Peter > > Thanks, Roger > > >> >> >> public final class Signal { >> >> private static final ConcurrentMap >> SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); >> private static final ConcurrentMap >> SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); >> >> public static Signal of(String name) { >> Signal signal = SIGNAL_BY_NAME.get(name); >> if (signal != null) { >> return signal; >> } >> >> int number; >> if (!name.startsWith("SIG") || name.length() <= 3 || >> (number = findSignal0(name.substring(3))) < 0) { >> throw new UnsupportedOperationException("Unknown signal: >> " + name); >> } >> >> signal = SIGNAL_BY_NUMBER.computeIfAbsent( >> number, >> new Function() { >> @Override >> public Signal apply(Integer number) { >> return new Signal(name, number); >> } >> } >> ); >> >> SIGNAL_BY_NAME.putIfAbsent(name, signal); >> >> return signal; >> } >> >> private final String name; >> private final int number; >> private volatile HandlerChain handlerChain; >> private long savedNativeHandler; >> >> private Signal(String name, int number) { >> this.name = name; >> this.number = number; >> } >> >> public String name() { return name; } >> >> public int number() { return number; } >> >> public void raise() { raise0(number); } >> >> public void register(BiConsumer handler) { >> synchronized (this) { >> HandlerChain oldChain = handlerChain; >> handlerChain = new HandlerChain(handler, oldChain); >> if (oldChain == null) { >> // set native to dispatch to Singnal.dispatch() >> savedNativeHandler = handle1(2); >> } >> } >> } >> >> public boolean unregister(BiConsumer handler) { >> synchronized (this) { >> HandlerChain oldChain = handlerChain; >> if (oldChain != null && oldChain.handler == handler) { >> if (oldChain.next == null) { >> // restore saved native handler >> long oldNativeHandler = handle1(savedNativeHandler); >> assert oldNativeHandler == 2L; >> } >> handlerChain = oldChain.next; >> return true; >> } else { >> return false; >> } >> } >> } >> >> // following two should probably be hidden from public API >> >> public void nativeIgnore() { >> synchronized (this) { >> if (handlerChain == null) { >> handle1(1); // ignore signal >> } else { >> throw new IllegalStateException( >> "Can't ignore signal after handlers have already >> been registered."); >> } >> } >> } >> >> public void nativeDefault() { >> synchronized (this) { >> if (handlerChain == null) { >> handle1(0); // default native handler >> } else { >> throw new IllegalStateException( >> "Can't restore signal after handlers have already >> been registered."); >> } >> } >> } >> >> private long handle1(long nativeHandler) { >> long oldNativeHandler = handle0(number, nativeHandler); >> if (oldNativeHandler == -1L) { >> throw new UnsupportedOperationException( >> "Signal already used by VM or OS: " + name); >> } >> return oldNativeHandler; >> } >> >> /* >> * Called by the VM to execute Java signal handlers. >> */ >> private static void dispatch(int number) { >> Signal signal = SIGNAL_BY_NUMBER.get(number); >> if (signal != null) { >> HandlerChain handlerChain = signal.handlerChain; >> if (handlerChain != null) { >> new InnocuousThread(() -> handlerChain.accept(signal)) >> .start(); >> } >> } >> } >> >> /** >> * Find the signal number, given a name. >> * >> * @param sigName the signal name >> * @return the signal number or -1 for unknown signals. >> */ >> private static native int findSignal0(String sigName); >> >> /* Registers a native signal handler, and returns the old handler. >> * Handler values: >> * 0 default handler >> * 1 ignore the signal >> * 2 call back to Signal.dispatch >> * other arbitrary native signal handlers >> * @param nativeH the index or address of the new signal handler >> * @return the previous index or address >> */ >> private static native long handle0(int sig, long nativeH); >> >> /* >> * Raise a given signal number. >> * @param sig the signal number to raise >> */ >> private static native void raise0(int sig); >> >> >> private static class HandlerChain implements Consumer { >> final BiConsumer handler; >> final HandlerChain next; >> >> HandlerChain(BiConsumer handler, >> HandlerChain next) { >> this.handler = handler; >> this.next = next; >> } >> >> @Override >> public void accept(Signal signal) { >> handler.accept(signal, () -> { >> if (next != null) { >> next.accept(signal); >> } >> }); >> } >> } >> } >> >> >> >> >> Regards, Peter >> >> >> On 02/01/2016 05:02 PM, Roger Riggs wrote: >>> Please review an API addition to handle signals such as SIGINT, >>> SIGHUP, and SIGTERM. >>> This JEP 260 motivated alternative to sun.misc.Signal supports the >>> use case for >>> interactive applications that need to handle Control-C and other >>> signals. >>> >>> The new java.util.Signal class provides a settable primary signal >>> handler and a default >>> signal handler. The primary signal handler can be unregistered and >>> handling is restored >>> to the default signal handler. System initialization registers >>> default signal handlers >>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>> requires >>> a permission if a SecurityManager is set. >>> >>> The sun.misc.Signal implementation is modified to be layered on a >>> common >>> thread and dispatch mechanism. The VM handling of native signals is >>> not affected. >>> The command option to reduce signal use by the runtime with -Xrs is >>> unmodified. >>> >>> The changes to hotspot are minimal to rename the hardcoded callback >>> to the Java >>> Signal dispatcher. >>> >>> Please review and comment on the API and implementation. >>> >>> javadoc: >>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>> >>> Webrev: >>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>> >>> JEP 260: >>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> Thanks, Roger >>> >>> >> > From chris.hegarty at oracle.com Tue Feb 2 16:22:54 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Feb 2016 16:22:54 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B0D5DD.3070501@gmail.com> References: <56AF819C.5020905@Oracle.com> <56B09684.6010107@gmail.com> <56B0C859.8070504@Oracle.com> <56B0D5DD.3070501@gmail.com> Message-ID: On 2 Feb 2016, at 16:14, Peter Levart wrote: > Hi Roger, > > On 02/02/2016 04:16 PM, Roger Riggs wrote: >> Hi Peter, >> >> On 2/2/2016 6:44 AM, Peter Levart wrote: >>> >> ... >>> >>> Also if this is to become public API, There's a chance users would want to add a handler to the chain of existing handlers or override them. So what about an API that allows registering/unregistering a default (non-native) handler and other handlers above it in a uniform way, like: >> The problem with chaining, as in the current API, is that there is no way to know what the next >> handler in the chain will do. If it is the default one for INT, TERM, HUP, it will call Shutdown and exit. >> So without extra information and cooperation chaining is risky. >> If the handler knows something about the other actors in the environment, it can coordinate with them directly. >> For the use cases that have been raised for existing use of sun.misc.Signal, they are simple interactive >> environments that want to give the appearance of being able to interrupt using control-c. >> >> I've been aiming for the simplest API that would support the current use cases. > > I noticed that sun.misc.Signal[Handler] is a critical API according to JEP 260, so it can't be removed in JDK9 yet. Right. It must stay, pretty much, as is in JDK 9. > I wanted to see if sun.misc.Signal[Handler] could be modified to use java.util.Signal I originally though about doing something like this: public interface SignalHandler extends Consumer { /** * Handle the given signal * * @param sig a signal object */ public void handle(Signal sig); + + @Override default void accept(java.util.Signal signal) { + handle(new Signal(signal)); + } + + static SignalHandler from(Consumer h) { + return new SignalHandler() { + final Consumer handler = h; + @Override + public void handle(Signal sig) { + accept(sig.signal()); + } + + @Override + public void accept(java.util.Signal signal) { + handler.accept(signal); + } + }; + } } -Chris. > directly while making java.util.Signal support chaining and restoring of previous handler on deregister. Here's what it looks like in code: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Signal/webrev.01/ > > But if chaining is not desired, then at least restoring of previous handler could be implemented in a uniform way and for arbitrary registration depth (no need for register/registerDefault distinction). > > What do you think? > > Regards, Peter > >> >> Thanks, Roger >> >> >>> >>> >>> public final class Signal { >>> >>> private static final ConcurrentMap >>> SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); >>> private static final ConcurrentMap >>> SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); >>> >>> public static Signal of(String name) { >>> Signal signal = SIGNAL_BY_NAME.get(name); >>> if (signal != null) { >>> return signal; >>> } >>> >>> int number; >>> if (!name.startsWith("SIG") || name.length() <= 3 || >>> (number = findSignal0(name.substring(3))) < 0) { >>> throw new UnsupportedOperationException("Unknown signal: " + name); >>> } >>> >>> signal = SIGNAL_BY_NUMBER.computeIfAbsent( >>> number, >>> new Function() { >>> @Override >>> public Signal apply(Integer number) { >>> return new Signal(name, number); >>> } >>> } >>> ); >>> >>> SIGNAL_BY_NAME.putIfAbsent(name, signal); >>> >>> return signal; >>> } >>> >>> private final String name; >>> private final int number; >>> private volatile HandlerChain handlerChain; >>> private long savedNativeHandler; >>> >>> private Signal(String name, int number) { >>> this.name = name; >>> this.number = number; >>> } >>> >>> public String name() { return name; } >>> >>> public int number() { return number; } >>> >>> public void raise() { raise0(number); } >>> >>> public void register(BiConsumer handler) { >>> synchronized (this) { >>> HandlerChain oldChain = handlerChain; >>> handlerChain = new HandlerChain(handler, oldChain); >>> if (oldChain == null) { >>> // set native to dispatch to Singnal.dispatch() >>> savedNativeHandler = handle1(2); >>> } >>> } >>> } >>> >>> public boolean unregister(BiConsumer handler) { >>> synchronized (this) { >>> HandlerChain oldChain = handlerChain; >>> if (oldChain != null && oldChain.handler == handler) { >>> if (oldChain.next == null) { >>> // restore saved native handler >>> long oldNativeHandler = handle1(savedNativeHandler); >>> assert oldNativeHandler == 2L; >>> } >>> handlerChain = oldChain.next; >>> return true; >>> } else { >>> return false; >>> } >>> } >>> } >>> >>> // following two should probably be hidden from public API >>> >>> public void nativeIgnore() { >>> synchronized (this) { >>> if (handlerChain == null) { >>> handle1(1); // ignore signal >>> } else { >>> throw new IllegalStateException( >>> "Can't ignore signal after handlers have already been registered."); >>> } >>> } >>> } >>> >>> public void nativeDefault() { >>> synchronized (this) { >>> if (handlerChain == null) { >>> handle1(0); // default native handler >>> } else { >>> throw new IllegalStateException( >>> "Can't restore signal after handlers have already been registered."); >>> } >>> } >>> } >>> >>> private long handle1(long nativeHandler) { >>> long oldNativeHandler = handle0(number, nativeHandler); >>> if (oldNativeHandler == -1L) { >>> throw new UnsupportedOperationException( >>> "Signal already used by VM or OS: " + name); >>> } >>> return oldNativeHandler; >>> } >>> >>> /* >>> * Called by the VM to execute Java signal handlers. >>> */ >>> private static void dispatch(int number) { >>> Signal signal = SIGNAL_BY_NUMBER.get(number); >>> if (signal != null) { >>> HandlerChain handlerChain = signal.handlerChain; >>> if (handlerChain != null) { >>> new InnocuousThread(() -> handlerChain.accept(signal)) >>> .start(); >>> } >>> } >>> } >>> >>> /** >>> * Find the signal number, given a name. >>> * >>> * @param sigName the signal name >>> * @return the signal number or -1 for unknown signals. >>> */ >>> private static native int findSignal0(String sigName); >>> >>> /* Registers a native signal handler, and returns the old handler. >>> * Handler values: >>> * 0 default handler >>> * 1 ignore the signal >>> * 2 call back to Signal.dispatch >>> * other arbitrary native signal handlers >>> * @param nativeH the index or address of the new signal handler >>> * @return the previous index or address >>> */ >>> private static native long handle0(int sig, long nativeH); >>> >>> /* >>> * Raise a given signal number. >>> * @param sig the signal number to raise >>> */ >>> private static native void raise0(int sig); >>> >>> >>> private static class HandlerChain implements Consumer { >>> final BiConsumer handler; >>> final HandlerChain next; >>> >>> HandlerChain(BiConsumer handler, HandlerChain next) { >>> this.handler = handler; >>> this.next = next; >>> } >>> >>> @Override >>> public void accept(Signal signal) { >>> handler.accept(signal, () -> { >>> if (next != null) { >>> next.accept(signal); >>> } >>> }); >>> } >>> } >>> } >>> >>> >>> >>> >>> Regards, Peter >>> >>> >>> On 02/01/2016 05:02 PM, Roger Riggs wrote: >>>> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >>>> interactive applications that need to handle Control-C and other signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal handler and a default >>>> signal handler. The primary signal handler can be unregistered and handling is restored >>>> to the default signal handler. System initialization registers default signal handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a common >>>> thread and dispatch mechanism. The VM handling of native signals is not affected. >>>> The command option to reduce signal use by the runtime with -Xrs is unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback to the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> Webrev: >>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> JEP 260: >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> Thanks, Roger From Roger.Riggs at Oracle.com Tue Feb 2 16:35:15 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 11:35:15 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: References: <56AF819C.5020905@Oracle.com> <56B0CBB6.9080903@Oracle.com> Message-ID: <56B0DAC3.8080105@Oracle.com> Hi Chris, On 2/2/2016 11:12 AM, Chris Hegarty wrote: > Roger, > > On 2 Feb 2016, at 15:31, Roger Riggs wrote: > >> Hi Chris, >> >> On 2/2/2016 8:58 AM, Chris Hegarty wrote: >>> I think this API should be a good replacement for sun.misc.Signal. >>> >>> Some comments: >>> >>> - The second @implNote, referring to s.m.Signal, I assume can be removed? >>> >> yes, except that sun.misc.Signal is still around for 9; there there should be note somewhere about the >> interactions, perhaps it should be in s.m.Signal instead. > Yes, I think it should be in s.m.Signal. > >>> - I agree with Peter, about the permission regarding the thread that executes >>> the consumer. Does it make sense to support a ThreadFactory, or may that >>> is overkill. >>> >> Using InnocuousThread should be sufficient. > Right, but then it should be specified. ok > >> Adding a ThreadFactory to the register method would give more control but is probably more than necessary. > Hmmm? maybe. But then it is very restrictive, unless there is some mechanism > to override it. same as current s.m.Signal implementation. > >>> - Is it necessary for ?raise' to declare that it may throw UOE? Since the Signal >>> and impl are already retrieved. Or maybe there is another possibility of failure? >>> >> As currently implemented in the VM, some signals can be found and handled but not raised. >> So UOE might occur on raise even if the signal can be handled. > Ok, thanks. > >>> - Signal.dispatch just calls SignalImpl.dispatch. Should the VM just do this >>> directly? >>> >> Yes, but since the VM has a hard coded class/method entry point; I thought it would be be lower >> maintenance to keep it in a well known class. (I'm open to other registration mechanisms for the callback too). >> I suspect the VM folks want to own that initial Signal Dispatch thread or I would propose just a blocking call to native >> to get the next signal; it returns when there is a signal to deliver. But that change can be factored out for later >> if it is a good idea. > Agreed. > >>> - unregister is based on the consumer, possibly lambda?s, identity ? This seems >>> strange. >>> >> Unless I'm mistaken, lambda's have identity and if the caller needs to unregister it, they would need to save >> the reference. Using an explicit class implementing Consumer would ensure that. > Yes, they do, I?m just questioning whether this is what we want. The handler needs some identity to be able to unregister it, either the method returns some kind of unique handle for the registration or it uses the identity of the consumer itself. It is simpler to hold on to the reference of the consumer. > >>> - I?m still confused by the registerDefault, and I?m not sure if it is actually needed. >>> Why can Terminator not just register as normal? >>> >> The next register would overwrite the handler and the implementation would not know that >> that handler should be the one restored when a client (like jshell) unregisters its control-c handler. > So the use case is to register a handler and later restore the previously > set handler. It is not possible for register to return the previously registered > handler, and then it is up the user to store it and reset it, as necessary. > Rather than the default notion. Returning the previously set handler exposes a reference to some other subsystem's object. In some cases it might be a security risk. > >>> - Should the instance methods that register/unregister/raise perform a >>> security check ( in case the Signal escapes, for example in the consumer ) >>> >> The Signal is a capability, > It is in the new API, but not in s.m.Signal ( it is just pair of string name and number ). > >> if you have a Signal reference you have the power; similar to Process and ProcessHandle >> the security check is done to get the reference to begin with. From an auditing point of view, >> there is no ambiguity in code that has access to a Signal reference; if has the reference it can control >> the process. > There is a potential issue when the Signal is delivered to the consumer when it > is raised, if, there is any thread, other than an innocuous thread, executing the > consumer. > > Another small implementation question: is it still necessary for s.m.Signal.dispatch > to start a new Thread ? If it re-uses the VM's Signal Dispatch thread, the latency in the signal consumer might delay other signal handling. Like with the Cleaner threads and functions, there is a concern about the impact of one of the clients on the shared resource (Thread). Starting a new thread allows the impact to be reduced. There might be an argument for using the a common executor but it might also introduce additional latency in processing the signal depending on the state of the executor. Starting a new thread has more predictable behavior. Thanks, Roger > > -Chris. > >> (I haven't done a review with the security folks on this yet). >> >> Thanks, Roger >> >> >>> -Chris. >>> >>> On 1 Feb 2016, at 16:02, Roger Riggs >>> >>> wrote: >>> >>> >>>> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >>>> interactive applications that need to handle Control-C and other signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal handler and a default >>>> signal handler. The primary signal handler can be unregistered and handling is restored >>>> to the default signal handler. System initialization registers default signal handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a common >>>> thread and dispatch mechanism. The VM handling of native signals is not affected. >>>> The command option to reduce signal use by the runtime with -Xrs is unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback to the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> >>>> Webrev: >>>> jdk: >>>> http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> >>>> hotspot: >>>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> >>>> Issue: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> >>>> JEP 260: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> >>>> Thanks, Roger >>>> >>>> >>>> From Roger.Riggs at Oracle.com Tue Feb 2 16:45:43 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 11:45:43 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B0D5DD.3070501@gmail.com> References: <56AF819C.5020905@Oracle.com> <56B09684.6010107@gmail.com> <56B0C859.8070504@Oracle.com> <56B0D5DD.3070501@gmail.com> Message-ID: <56B0DD37.4090407@Oracle.com> Hi Peter, On 2/2/2016 11:14 AM, Peter Levart wrote: > Hi Roger, > > On 02/02/2016 04:16 PM, Roger Riggs wrote: >> Hi Peter, >> >> On 2/2/2016 6:44 AM, Peter Levart wrote: >>> >> ... >>> >>> Also if this is to become public API, There's a chance users would >>> want to add a handler to the chain of existing handlers or override >>> them. So what about an API that allows registering/unregistering a >>> default (non-native) handler and other handlers above it in a >>> uniform way, like: >> The problem with chaining, as in the current API, is that there is no >> way to know what the next >> handler in the chain will do. If it is the default one for INT, >> TERM, HUP, it will call Shutdown and exit. >> So without extra information and cooperation chaining is risky. >> If the handler knows something about the other actors in the >> environment, it can coordinate with them directly. >> For the use cases that have been raised for existing use of >> sun.misc.Signal, they are simple interactive >> environments that want to give the appearance of being able to >> interrupt using control-c. >> >> I've been aiming for the simplest API that would support the current >> use cases. > > I noticed that sun.misc.Signal[Handler] is a critical API according to > JEP 260, so it can't be removed in JDK9 yet. I wanted to see if > sun.misc.Signal[Handler] could be modified to use java.util.Signal > directly while making java.util.Signal support chaining and restoring > of previous handler on deregister. Here's what it looks like in code: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Signal/webrev.01/ > > But if chaining is not desired, then at least restoring of previous > handler could be implemented in a uniform way and for arbitrary > registration depth (no need for register/registerDefault distinction). A stack based model can work but still needs backup handlers for when all the specified handlers to unregistered. I'm ok with submerging registerDefault into the implementation used only by the system Termination class. I don't think the use cases call for a stack and it makes the API more complex for register/unregister. I'd rather see a good simple j.u.Signal API that meets the requirements without considering how to support the legacy s.m.Signal, if it needs a bit of cruft to support it for a while that's ok. Someday we will want to rip out all that code and have a simple API. Roger > > What do you think? > > Regards, Peter > >> >> Thanks, Roger >> >> >>> >>> >>> public final class Signal { >>> >>> private static final ConcurrentMap >>> SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); >>> private static final ConcurrentMap >>> SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); >>> >>> public static Signal of(String name) { >>> Signal signal = SIGNAL_BY_NAME.get(name); >>> if (signal != null) { >>> return signal; >>> } >>> >>> int number; >>> if (!name.startsWith("SIG") || name.length() <= 3 || >>> (number = findSignal0(name.substring(3))) < 0) { >>> throw new UnsupportedOperationException("Unknown signal: >>> " + name); >>> } >>> >>> signal = SIGNAL_BY_NUMBER.computeIfAbsent( >>> number, >>> new Function() { >>> @Override >>> public Signal apply(Integer number) { >>> return new Signal(name, number); >>> } >>> } >>> ); >>> >>> SIGNAL_BY_NAME.putIfAbsent(name, signal); >>> >>> return signal; >>> } >>> >>> private final String name; >>> private final int number; >>> private volatile HandlerChain handlerChain; >>> private long savedNativeHandler; >>> >>> private Signal(String name, int number) { >>> this.name = name; >>> this.number = number; >>> } >>> >>> public String name() { return name; } >>> >>> public int number() { return number; } >>> >>> public void raise() { raise0(number); } >>> >>> public void register(BiConsumer handler) { >>> synchronized (this) { >>> HandlerChain oldChain = handlerChain; >>> handlerChain = new HandlerChain(handler, oldChain); >>> if (oldChain == null) { >>> // set native to dispatch to Singnal.dispatch() >>> savedNativeHandler = handle1(2); >>> } >>> } >>> } >>> >>> public boolean unregister(BiConsumer handler) { >>> synchronized (this) { >>> HandlerChain oldChain = handlerChain; >>> if (oldChain != null && oldChain.handler == handler) { >>> if (oldChain.next == null) { >>> // restore saved native handler >>> long oldNativeHandler = handle1(savedNativeHandler); >>> assert oldNativeHandler == 2L; >>> } >>> handlerChain = oldChain.next; >>> return true; >>> } else { >>> return false; >>> } >>> } >>> } >>> >>> // following two should probably be hidden from public API >>> >>> public void nativeIgnore() { >>> synchronized (this) { >>> if (handlerChain == null) { >>> handle1(1); // ignore signal >>> } else { >>> throw new IllegalStateException( >>> "Can't ignore signal after handlers have already >>> been registered."); >>> } >>> } >>> } >>> >>> public void nativeDefault() { >>> synchronized (this) { >>> if (handlerChain == null) { >>> handle1(0); // default native handler >>> } else { >>> throw new IllegalStateException( >>> "Can't restore signal after handlers have >>> already been registered."); >>> } >>> } >>> } >>> >>> private long handle1(long nativeHandler) { >>> long oldNativeHandler = handle0(number, nativeHandler); >>> if (oldNativeHandler == -1L) { >>> throw new UnsupportedOperationException( >>> "Signal already used by VM or OS: " + name); >>> } >>> return oldNativeHandler; >>> } >>> >>> /* >>> * Called by the VM to execute Java signal handlers. >>> */ >>> private static void dispatch(int number) { >>> Signal signal = SIGNAL_BY_NUMBER.get(number); >>> if (signal != null) { >>> HandlerChain handlerChain = signal.handlerChain; >>> if (handlerChain != null) { >>> new InnocuousThread(() -> handlerChain.accept(signal)) >>> .start(); >>> } >>> } >>> } >>> >>> /** >>> * Find the signal number, given a name. >>> * >>> * @param sigName the signal name >>> * @return the signal number or -1 for unknown signals. >>> */ >>> private static native int findSignal0(String sigName); >>> >>> /* Registers a native signal handler, and returns the old handler. >>> * Handler values: >>> * 0 default handler >>> * 1 ignore the signal >>> * 2 call back to Signal.dispatch >>> * other arbitrary native signal handlers >>> * @param nativeH the index or address of the new signal handler >>> * @return the previous index or address >>> */ >>> private static native long handle0(int sig, long nativeH); >>> >>> /* >>> * Raise a given signal number. >>> * @param sig the signal number to raise >>> */ >>> private static native void raise0(int sig); >>> >>> >>> private static class HandlerChain implements Consumer { >>> final BiConsumer handler; >>> final HandlerChain next; >>> >>> HandlerChain(BiConsumer handler, >>> HandlerChain next) { >>> this.handler = handler; >>> this.next = next; >>> } >>> >>> @Override >>> public void accept(Signal signal) { >>> handler.accept(signal, () -> { >>> if (next != null) { >>> next.accept(signal); >>> } >>> }); >>> } >>> } >>> } >>> >>> >>> >>> >>> Regards, Peter >>> >>> >>> On 02/01/2016 05:02 PM, Roger Riggs wrote: >>>> Please review an API addition to handle signals such as SIGINT, >>>> SIGHUP, and SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the >>>> use case for >>>> interactive applications that need to handle Control-C and other >>>> signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal >>>> handler and a default >>>> signal handler. The primary signal handler can be unregistered and >>>> handling is restored >>>> to the default signal handler. System initialization registers >>>> default signal handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>> requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a >>>> common >>>> thread and dispatch mechanism. The VM handling of native signals is >>>> not affected. >>>> The command option to reduce signal use by the runtime with -Xrs is >>>> unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback >>>> to the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> Webrev: >>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> JEP 260: >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> Thanks, Roger >>>> >>>> >>> >> > From chris.hegarty at oracle.com Tue Feb 2 16:58:30 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 2 Feb 2016 16:58:30 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B0DAC3.8080105@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56B0CBB6.9080903@Oracle.com> <56B0DAC3.8080105@Oracle.com> Message-ID: <102BFA99-7D9D-4FDB-AEAF-6E10D959C6B0@oracle.com> On 2 Feb 2016, at 16:35, Roger Riggs wrote: : >>>> - I agree with Peter, about the permission regarding the thread that executes >>>> the consumer. Does it make sense to support a ThreadFactory, or may that >>>> is overkill. >>>> >>>> >>> Using InnocuousThread should be sufficient. >>> >> Right, but then it should be specified. > ok >> >>> Adding a ThreadFactory to the register method would give more control but is probably more than necessary. >>> >> Hmmm? maybe. But then it is very restrictive, unless there is some mechanism >> to override it. >> > same as current s.m.Signal implementation. But, doesn?t s.m.Signal start a new thread from the VM created signal dispatcher thread which has all permissions ( inherited ACC )? j.u.Signal will be different, it would execute the consumer with no permissions. : >> So the use case is to register a handler and later restore the previously >> set handler. It is not possible for register to return the previously registered >> handler, and then it is up the user to store it and reset it, as necessary. >> Rather than the default notion. >> > Returning the previously set handler exposes a reference to some other subsystem's object. > In some cases it might be a security risk. Right, but the user will have to have permission. : >>> if you have a Signal reference you have the power; similar to Process and ProcessHandle >>> the security check is done to get the reference to begin with. From an auditing point of view, >>> there is no ambiguity in code that has access to a Signal reference; if has the reference it can control >>> the process. >>> >> There is a potential issue when the Signal is delivered to the consumer when it >> is raised, if, there is any thread, other than an innocuous thread, executing the >> consumer. >> >> Another small implementation question: is it still necessary for s.m.Signal.dispatch >> to start a new Thread ? >> > If it re-uses the VM's Signal Dispatch thread, the latency in the signal consumer might delay other signal > handling. I was not suggesting that it use the VM Signal Dispatch Thread. SignalImpl.dispatch, the common dispatch path, already starts a new thread and executes s.m.dispatch. > Like with the Cleaner threads and functions, there is a concern about the impact of one of the clients > on the shared resource (Thread). Starting a new thread allows the impact to be reduced. > > There might be an argument for using the a common executor but it might also introduce additional > latency in processing the signal depending on the state of the executor. > Starting a new thread has more predictable behaviour. I was actually suggesting supporting a variant taking a thread factory, since the API will specify that the default handler thread, created per signal raised, will execute with no permissions. -Chris. > Thanks, Roger > >> >> -Chris. >> >> >>> (I haven't done a review with the security folks on this yet). >>> >>> Thanks, Roger >>> >>> >>> >>>> -Chris. >>>> >>>> On 1 Feb 2016, at 16:02, Roger Riggs >>>> >>>> >>>> >>>> wrote: >>>> >>>> >>>> >>>>> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >>>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >>>>> interactive applications that need to handle Control-C and other signals. >>>>> >>>>> The new java.util.Signal class provides a settable primary signal handler and a default >>>>> signal handler. The primary signal handler can be unregistered and handling is restored >>>>> to the default signal handler. System initialization registers default signal handlers >>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >>>>> a permission if a SecurityManager is set. >>>>> >>>>> The sun.misc.Signal implementation is modified to be layered on a common >>>>> thread and dispatch mechanism. The VM handling of native signals is not affected. >>>>> The command option to reduce signal use by the runtime with -Xrs is unmodified. >>>>> >>>>> The changes to hotspot are minimal to rename the hardcoded callback to the Java >>>>> Signal dispatcher. >>>>> >>>>> Please review and comment on the API and implementation. >>>>> >>>>> javadoc: >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>> >>>>> >>>>> >>>>> Webrev: >>>>> jdk: >>>>> >>>>> http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>> >>>>> >>>>> hotspot: >>>>> >>>>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>> >>>>> >>>>> >>>>> Issue: >>>>> >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>> >>>>> >>>>> >>>>> JEP 260: >>>>> >>>>> >>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>> >>>>> >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> >>>>> > From Roger.Riggs at Oracle.com Tue Feb 2 17:15:48 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 12:15:48 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <102BFA99-7D9D-4FDB-AEAF-6E10D959C6B0@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B0CBB6.9080903@Oracle.com> <56B0DAC3.8080105@Oracle.com> <102BFA99-7D9D-4FDB-AEAF-6E10D959C6B0@oracle.com> Message-ID: <56B0E444.6060902@Oracle.com> Hi Chris, On 2/2/2016 11:58 AM, Chris Hegarty wrote: >>>> Adding a ThreadFactory to the register method would give more control but is probably more than necessary. >>>> >>> Hmmm? maybe. But then it is very restrictive, unless there is some mechanism >>> to override it. >>> >> same as current s.m.Signal implementation. > But, doesn?t s.m.Signal start a new thread from the VM created signal dispatcher thread > which has all permissions ( inherited ACC )? j.u.Signal will be different, it would execute > the consumer with no permissions. Yes, for compatibility, s.m.Signal would need all Permissions. Is there any way to determine if that's really necessary. For the new java.util.Signal if elevated permissions are needed, the code could use doPriv as necessary. In most cases, it should just be notifying some other thread of the client. > > : > >>> So the use case is to register a handler and later restore the previously >>> set handler. It is not possible for register to return the previously registered >>> handler, and then it is up the user to store it and reset it, as necessary. >>> Rather than the default notion. >>> >> Returning the previously set handler exposes a reference to some other subsystem's object. >> In some cases it might be a security risk. > Right, but the user will have to have permission. > > : > >>>> if you have a Signal reference you have the power; similar to Process and ProcessHandle >>>> the security check is done to get the reference to begin with. From an auditing point of view, >>>> there is no ambiguity in code that has access to a Signal reference; if has the reference it can control >>>> the process. >>>> >>> There is a potential issue when the Signal is delivered to the consumer when it >>> is raised, if, there is any thread, other than an innocuous thread, executing the >>> consumer. The thread safety of the consumer is their responsibility. >>> >>> Another small implementation question: is it still necessary for s.m.Signal.dispatch >>> to start a new Thread ? >>> >> If it re-uses the VM's Signal Dispatch thread, the latency in the signal consumer might delay other signal >> handling. > I was not suggesting that it use the VM Signal Dispatch Thread. SignalImpl.dispatch, > the common dispatch path, already starts a new thread and executes s.m.dispatch. Right, I missed your meaning. Only 1 new thread is needed, s.m.Signal.dispatch can use the thread as is. >> Like with the Cleaner threads and functions, there is a concern about the impact of one of the clients >> on the shared resource (Thread). Starting a new thread allows the impact to be reduced. >> >> There might be an argument for using the a common executor but it might also introduce additional >> latency in processing the signal depending on the state of the executor. >> Starting a new thread has more predictable behaviour. > I was actually suggesting supporting a variant taking a thread factory, since the > API will specify that the default handler thread, created per signal raised, will > execute with no permissions. I don't know whether to try to anticipate and build that in or expect that with doPriv the thread could gain any permissions or priority it needed. Roger > > -Chris. > >> Thanks, Roger >> >>> -Chris. >>> >>> >>>> (I haven't done a review with the security folks on this yet). >>>> >>>> Thanks, Roger >>>> >>>> >>>> >>>>> -Chris. >>>>> >>>>> On 1 Feb 2016, at 16:02, Roger Riggs >>>>> >>>>> >>>>> >>>>> wrote: >>>>> >>>>> >>>>> >>>>>> Please review an API addition to handle signals such as SIGINT, SIGHUP, and SIGTERM. >>>>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use case for >>>>>> interactive applications that need to handle Control-C and other signals. >>>>>> >>>>>> The new java.util.Signal class provides a settable primary signal handler and a default >>>>>> signal handler. The primary signal handler can be unregistered and handling is restored >>>>>> to the default signal handler. System initialization registers default signal handlers >>>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires >>>>>> a permission if a SecurityManager is set. >>>>>> >>>>>> The sun.misc.Signal implementation is modified to be layered on a common >>>>>> thread and dispatch mechanism. The VM handling of native signals is not affected. >>>>>> The command option to reduce signal use by the runtime with -Xrs is unmodified. >>>>>> >>>>>> The changes to hotspot are minimal to rename the hardcoded callback to the Java >>>>>> Signal dispatcher. >>>>>> >>>>>> Please review and comment on the API and implementation. >>>>>> >>>>>> javadoc: >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>>> >>>>>> >>>>>> >>>>>> Webrev: >>>>>> jdk: >>>>>> >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>>> >>>>>> >>>>>> hotspot: >>>>>> >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>>> >>>>>> >>>>>> >>>>>> Issue: >>>>>> >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>>> >>>>>> >>>>>> >>>>>> JEP 260: >>>>>> >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>>> >>>>>> >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>>> >>>>>> >>>>>> From steve.drach at oracle.com Tue Feb 2 17:32:18 2016 From: steve.drach at oracle.com (Steve Drach) Date: Tue, 2 Feb 2016 09:32:18 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> Message-ID: I think the following is a reasonable solution. If the JarFile is constructed with any of the 5 constructors that do not contain a Release argument, then entries()/stream() returns the set of all entries in the jar file including those under the META-INF/versions/directory. The base entries are ?raw?, they are not aliases for versioned entries. If the JarFile is constructed with the 1 constructor that does include a Release argument, then entries()/stream() returns the set of appropriately versioned entries that would result from invoking getEntry()/getJarEntry() with the name of each base entry. The entries in the tree rooted at the directory META-INF/versions/ are not returned. > On Feb 1, 2016, at 12:29 PM, Steve Drach wrote: > > I?m sorry, I didn?t look at the code close enough before I started talking ;-) Right now entries()/stream() returns all entries and if the JarFile is constructed with a Release object != Release.BASE, the base entries that are returned are the versioned entries. I think this behavior is a bit confusing and we should just return all entries without regard to versioning. Then create the two new methods for specific versioned entries. > >> On Feb 1, 2016, at 12:18 PM, Steve Drach wrote: >> >>>>>> Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. >>>>>> >>>>>> My assumption was the traversal should by default be consistent with a calls to getEntry, thus: >>>>>> >>>>>> jarFile.stream().forEach(e -> { >>>>>> JarEntry je = jarFile.getJarEntry(e.getName()); >>>>>> assert e.equals(je); >>>>>> }); >>>>>> >>>>>> There might need to be another stream method that returns all entries. >>>>>> >>>>> Right, I'm mostly just wondering if entries()/streams() should override the entries in the stream with versioned entries and filter out the META-INF/versions/ tree. >>>> I don?t think so. That kind of behavior might be difficult to understand. Returning all the entries provides some flexibility. One can write code like this: >>>> >>>> jarfile.stream().map(JarEntry::getName).filter(s -> !s.startsWith(?META-INF?)).map(JarFile::getJarEntry).etc >>>> >>>> to get the versioned results for any version you specify when constructing the JarFile. >>> >>> The current specification treats those class files under meta-inf/releases like >>> kind of "metadata" of those base entries. Ideally those files should not even >>> be individual "files", but part of their corresponding entries. The consumer of >>> the MR-Jar should not need to be aware of these version-ed entries at all to use >>> this MR-jar file to load classes/resources. From this point of view, these entries >>> probably should be "invisible" from entries()/stream(), when the jar file is opened >>> with "version-enabled". And all returned entries should have all their "data" >>> (size, csize, timestamps, crc ...) pointed to the corresponding version-ed entries, >>> withe the only exception is the "name". >>> >>> On the other hand it might be desired to keep JarFile.entries()/stream() asis to >>> match their "zip file" perspective, to return "all" raw entries. Then it might also >>> be desired to have an alternative "versioned streamVersion()" ? >> >> It seems to that we have two reasonable alternatives: (1) return all entries, and (2) return all entries except those under the ?META-INF/versions/? directory and for any entries returned, return their versioned equivalent if it exists. If we choose alternative 2, we can still get alternative 1 by asking for JarFile.super.entries() and JarFile.super.stream(). >> >> Or we can do it both ways, leaving entries()/stream() as is and adding two new methods, versionedEntries() and versionedStream(). >> >>> >>> something like >>> >>> public Stream stream(Release r); ? >> >> We should not parametrize the methods with a Release, because what does it mean if we construct the JarFile with one Release but specify a different Release for the stream argument. Parameterizing methods with a Release object feels like we?re starting to slide down a slippery slope. >> >> I think adding the two new methods is the ?right? solution, but I?d like some consensus here. >> >>> >>> -sherman >>> >>> >>> >>> >>>>> If I've gone to trouble of specifying the a Release then it seems the right thing to do. On the other hand, it comes at a cost and there will be use-cases like "get the names of all entries" that would be more efficient to just build on the current entries()/stream(). I'm loath to suggest this might need a new method but it might be one of the options to consider here. Minimally there is a javadoc to specify on how these methods behave when the JAR is multi-release and opened by specifying a release. >>>> How?s this? >>>> >>>> diff -r 68867430065b src/java.base/share/classes/java/util/jar/JarFile.java >>>> --- a/src/java.base/share/classes/java/util/jar/JarFile.java Fri Jan 29 12:34:44 2016 -0800 >>>> +++ b/src/java.base/share/classes/java/util/jar/JarFile.java Mon Feb 01 09:48:05 2016 -0800 >>>> @@ -576,9 +576,11 @@ >>>> } >>>> >>>> /** >>>> - * Returns an enumeration of the jar file entries. >>>> + * Returns an enumeration of all the jar file entries. Constructing this >>>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >>>> + * constructor does not modify the behavior of this method. >>>> * >>>> - * @return an enumeration of the jar file entries >>>> + * @return an enumeration of the all jar file entries >>>> * @throws IllegalStateException >>>> * may be thrown if the jar file has been closed >>>> */ >>>> @@ -587,11 +589,13 @@ >>>> } >>>> >>>> /** >>>> - * Returns an ordered {@code Stream} over the jar file entries. >>>> + * Returns an ordered {@code Stream} over all the jar file entries. >>>> * Entries appear in the {@code Stream} in the order they appear in >>>> - * the central directory of the jar file. >>>> + * the central directory of the jar file. Constructing this >>>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >>>> + * constructor does not modify the behavior of this method. >>>> * >>>> - * @return an ordered {@code Stream} of entries in this jar file >>>> + * @return an ordered {@code Stream} of all entries in this jar file >>>> * @throws IllegalStateException if the jar file has been closed >>>> * @since 1.8 >>>> */ > From Roger.Riggs at Oracle.com Tue Feb 2 18:27:59 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 13:27:59 -0500 Subject: RFR:JDK-8146747:java.time.Duration.toNanos() and toMillis() exception on negative durations In-Reply-To: <56AEF8B8.4070703@oracle.com> References: <56AEF8B8.4070703@oracle.com> Message-ID: <56B0F52F.1090905@Oracle.com> Hi Nadeesh, Looks fine. Roger On 2/1/2016 1:18 AM, nadeesh tv wrote: > Hi all, > > Please review following > > Bug Id : https://bugs.openjdk.java.net/browse/JDK-8146747 > > > Solution: Handled the negative duration separately > > webrev : http://cr.openjdk.java.net/~ntv/8146747/webrev.00/ > > -- > Thanks and Regards, > Nadeesh TV > From mikael.vidstedt at oracle.com Tue Feb 2 19:25:33 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 2 Feb 2016 11:25:33 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56A96B55.7050301@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> Message-ID: <56B102AD.7020800@oracle.com> Please review this change which introduces a Copy::conjoint_swap and an Unsafe.copySwapMemory method to call it from Java, along with the necessary changes to have java.nio.Bits call it instead of the Bits.c code. http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ On the jdk/ side I don't think there should be a lot of surprises. Bits.c is gone and that required a mapfile-vers to be changed accordingly. I also added a relatively extensive jdk/internal/misc/Unsafe/CopySwap.java test which exercises all the various copySwap configurations and verifies that the resulting data is correct. There are also a handful of negative tests in there. On the hotspot/ side: * the copy logic in copy.cpp is leveraging templates to help the C++ compiler produce tight copy loops for the various configurations {element type, copy direction, src aligned, dst aligned}. * Unsafe_CopySwapMemory is a leaf to not stall safe points more than necessary. Only if needed (THROW, copy involves heap objects) will it enter VM using a new JVM_ENTRY_FROM_LEAF macro. * JVM_ENTRY_FROM_LEAF calls a new VM_ENTRY_BASE_FROM_LEAF helper macro, which mimics what VM_ENTRY_BASE does, but also does a debug_only(ResetNoHandleMark __rnhm;) - this is because JVM_LEAF/VM_LEAF_BASE does debug_only(NoHandleMark __hm;). I'm in the process of getting the last performance numbers, but from what I've seen so far this will outperform the earlier implementation. Cheers, Mikeal On 2016-01-27 17:13, Mikael Vidstedt wrote: > > Just an FYI: > > I'm working on moving all of this to the Hotspot Copy class and > bridging to it via jdk.internal.misc.Unsafe, removing Bits.c > altogether. The implementation is working, and the preliminary > performance numbers beat the pants off of any of the suggested Bits.c > implementations (yay!). > > I'm currently in the progress of getting some unit tests in place for > it all to make sure it covers all the corner cases and then I'll run > some real benchmarks to see if it actually lives up to the expectations. > > Cheers, > Mikael > > On 2016-01-26 11:13, John Rose wrote: >> On Jan 26, 2016, at 11:08 AM, Andrew Haley wrote: >>> On 01/26/2016 07:04 PM, John Rose wrote: >>>> Unsafe.copyMemory bottoms out to Copy::conjoint_memory_atomic. >>>> IMO that's a better starting point than memcpy. Perhaps it can be >>>> given an additional parameter (or overloading) to specify a swap size. >>> OK, but conjoint_memory_atomic doesn't guarantee that destination >>> words won't be torn if their source is misaligned: in fact it >>> guarantees that they will will be. >> That's a good point, and argues for a new function with the >> stronger guarantee. Actually, it would be perfectly reasonable >> to strengthen the guarantee on the existing function. I don't >> think anyone will care about the slight performance change, >> especially since it is probably favorable. Since it's Unsafe, >> they are not supposed to care, either. >> >> ? John > From aleksey.shipilev at oracle.com Tue Feb 2 19:41:39 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 2 Feb 2016 22:41:39 +0300 Subject: RFR (S) 8148869: StringConcatFactory MH_INLINE_SIZED_EXACT strategy does not work with -XX:-CompactStrings Message-ID: <56B10673.2090704@oracle.com> Hi, Please review another simple fix in String concat: https://bugs.openjdk.java.net/browse/JDK-8148869 The failure happens because MH_INLINE_SIZED_EXACT strategy has to figure out the coder for the String concat result, as the binary "or" of initial coder and all argument's (possible) coders. And it mistakenly starts with coder=0, which is String.LATIN1. Under -XX:+CompactStrings (default), if all concat arguments can be encoded in LATIN1, it should produce LATIN1 result; and if at least one argument is UTF16, then the result should be UTF16. The trouble comes with -XX:-CompactStrings, which should always produce UTF16 concat result. Then, even if all arguments can be encoded into LATIN1, the result should be UTF16. But it isn't, because the initial coder is always LATIN1! The fix is to make initial coder depend on Strings.COMPACT_STRINGS: http://cr.openjdk.java.net/~shade/8148869/webrev.01/ Since the failure is predicated on -XX:-CompactStrings, our current tests fail only on platforms where CompactStrings are turned off by default, e.g. AArch64. Added a new regression test that concats differently encodeable chars under +|-CompactStrings. Cheers, -Aleksey From artem.smotrakov at oracle.com Tue Feb 2 19:48:57 2016 From: artem.smotrakov at oracle.com (Artem Smotrakov) Date: Tue, 2 Feb 2016 11:48:57 -0800 Subject: [9] RFR: 8146249: libjimage should use delete[] with new[] Message-ID: <56B10829.3040206@oracle.com> Hello, Please review this small fix for jdk9/dev repo. It updates a couple of files in libjimage to use delete[] operator together with new[]. This fix is similar to https://bugs.openjdk.java.net/browse/JDK-8140649 Bug: https://bugs.openjdk.java.net/browse/JDK-8146249 Webrev: http://cr.openjdk.java.net/~asmotrak/8146249/webrev.00/ Artem From james.laskey at oracle.com Tue Feb 2 19:53:27 2016 From: james.laskey at oracle.com (Jim Laskey (Oracle)) Date: Tue, 2 Feb 2016 15:53:27 -0400 Subject: [9] RFR: 8146249: libjimage should use delete[] with new[] In-Reply-To: <56B10829.3040206@oracle.com> References: <56B10829.3040206@oracle.com> Message-ID: <903203D9-24A0-4D2A-B0CC-5DB1DFF26F93@oracle.com> +1 Note that these changes will be superceded (thrown out) by the Jake merge. ? Jim > On Feb 2, 2016, at 3:48 PM, Artem Smotrakov wrote: > > Hello, > > Please review this small fix for jdk9/dev repo. > > It updates a couple of files in libjimage to use delete[] operator together with new[]. This fix is similar to https://bugs.openjdk.java.net/browse/JDK-8140649 > > Bug: https://bugs.openjdk.java.net/browse/JDK-8146249 > Webrev: http://cr.openjdk.java.net/~asmotrak/8146249/webrev.00/ > > Artem From paul.sandoz at oracle.com Tue Feb 2 20:00:32 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 2 Feb 2016 20:00:32 +0000 Subject: RFR (S) 8148869: StringConcatFactory MH_INLINE_SIZED_EXACT strategy does not work with -XX:-CompactStrings In-Reply-To: <56B10673.2090704@oracle.com> References: <56B10673.2090704@oracle.com> Message-ID: <072804AE-0B36-4ADF-BAFF-64D3FE2EEE7D@oracle.com> +1 Paul. > On 2 Feb 2016, at 19:41, Aleksey Shipilev wrote: > > Hi, > > Please review another simple fix in String concat: > https://bugs.openjdk.java.net/browse/JDK-8148869 > > The failure happens because MH_INLINE_SIZED_EXACT strategy has to figure > out the coder for the String concat result, as the binary "or" of > initial coder and all argument's (possible) coders. And it mistakenly > starts with coder=0, which is String.LATIN1. > > Under -XX:+CompactStrings (default), if all concat arguments can be > encoded in LATIN1, it should produce LATIN1 result; and if at least one > argument is UTF16, then the result should be UTF16. > > The trouble comes with -XX:-CompactStrings, which should always produce > UTF16 concat result. Then, even if all arguments can be encoded into > LATIN1, the result should be UTF16. But it isn't, because the initial > coder is always LATIN1! > > The fix is to make initial coder depend on Strings.COMPACT_STRINGS: > http://cr.openjdk.java.net/~shade/8148869/webrev.01/ > > Since the failure is predicated on -XX:-CompactStrings, our current > tests fail only on platforms where CompactStrings are turned off by > default, e.g. AArch64. Added a new regression test that concats > differently encodeable chars under +|-CompactStrings. > > Cheers, > -Aleksey > From vladimir.x.ivanov at oracle.com Tue Feb 2 20:07:09 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 2 Feb 2016 23:07:09 +0300 Subject: RFR (S) 8148869: StringConcatFactory MH_INLINE_SIZED_EXACT strategy does not work with -XX:-CompactStrings In-Reply-To: <56B10673.2090704@oracle.com> References: <56B10673.2090704@oracle.com> Message-ID: Looks good. Best regards, Vladimir Ivanov > On 2 ????. 2016 ?., at 22:41, Aleksey Shipilev wrote: > > Hi, > > Please review another simple fix in String concat: > https://bugs.openjdk.java.net/browse/JDK-8148869 > > The failure happens because MH_INLINE_SIZED_EXACT strategy has to figure > out the coder for the String concat result, as the binary "or" of > initial coder and all argument's (possible) coders. And it mistakenly > starts with coder=0, which is String.LATIN1. > > Under -XX:+CompactStrings (default), if all concat arguments can be > encoded in LATIN1, it should produce LATIN1 result; and if at least one > argument is UTF16, then the result should be UTF16. > > The trouble comes with -XX:-CompactStrings, which should always produce > UTF16 concat result. Then, even if all arguments can be encoded into > LATIN1, the result should be UTF16. But it isn't, because the initial > coder is always LATIN1! > > The fix is to make initial coder depend on Strings.COMPACT_STRINGS: > http://cr.openjdk.java.net/~shade/8148869/webrev.01/ > > Since the failure is predicated on -XX:-CompactStrings, our current > tests fail only on platforms where CompactStrings are turned off by > default, e.g. AArch64. Added a new regression test that concats > differently encodeable chars under +|-CompactStrings. > > Cheers, > -Aleksey > From nick.radov at optum.com Tue Feb 2 20:43:27 2016 From: nick.radov at optum.com (Radov, Nicholas O) Date: Tue, 2 Feb 2016 20:43:27 +0000 Subject: JDK-8019345 (java.net.URI should be updated for RFC 3986): duplicate of JDK-6791060? Message-ID: <0D30AB9F36B3B24DA8D29EB394F7F21442ADA638@APSWP0948.ms.ds.uhc.com> It looks like issue JDK-8019345 < https://bugs.openjdk.java.net/browse/JDK-8019345> is a duplicate of issue JDK-6791060 . Should one of those be closed? Regarding the comment on JDK-8019345 about how the implementation differs from that required by RFC 3986, I am aware of two minor differences. 1. They added the IPvFuture rule. So a URI like "http://[v0.0]" should be valid but currently it fails. 2. They added the path-empty rule. So a URI like "foo:" should be valid but currently it fails. Would someone be willing to update the issue with that additional information? I don't have an official contributor account so I can't add comments myself. This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately. From peter.levart at gmail.com Tue Feb 2 20:58:54 2016 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 2 Feb 2016 21:58:54 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B0DD37.4090407@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56B09684.6010107@gmail.com> <56B0C859.8070504@Oracle.com> <56B0D5DD.3070501@gmail.com> <56B0DD37.4090407@Oracle.com> Message-ID: <56B1188E.90602@gmail.com> On 02/02/2016 05:45 PM, Roger Riggs wrote: > Hi Peter, > > > On 2/2/2016 11:14 AM, Peter Levart wrote: >> Hi Roger, >> >> On 02/02/2016 04:16 PM, Roger Riggs wrote: >>> Hi Peter, >>> >>> On 2/2/2016 6:44 AM, Peter Levart wrote: >>>> >>> ... >>>> >>>> Also if this is to become public API, There's a chance users would >>>> want to add a handler to the chain of existing handlers or override >>>> them. So what about an API that allows registering/unregistering a >>>> default (non-native) handler and other handlers above it in a >>>> uniform way, like: >>> The problem with chaining, as in the current API, is that there is >>> no way to know what the next >>> handler in the chain will do. If it is the default one for INT, >>> TERM, HUP, it will call Shutdown and exit. >>> So without extra information and cooperation chaining is risky. >>> If the handler knows something about the other actors in the >>> environment, it can coordinate with them directly. >>> For the use cases that have been raised for existing use of >>> sun.misc.Signal, they are simple interactive >>> environments that want to give the appearance of being able to >>> interrupt using control-c. >>> >>> I've been aiming for the simplest API that would support the current >>> use cases. >> >> I noticed that sun.misc.Signal[Handler] is a critical API according >> to JEP 260, so it can't be removed in JDK9 yet. I wanted to see if >> sun.misc.Signal[Handler] could be modified to use java.util.Signal >> directly while making java.util.Signal support chaining and restoring >> of previous handler on deregister. Here's what it looks like in code: >> >> http://cr.openjdk.java.net/~plevart/jdk9-dev/Signal/webrev.01/ >> >> But if chaining is not desired, then at least restoring of previous >> handler could be implemented in a uniform way and for arbitrary >> registration depth (no need for register/registerDefault distinction). > > A stack based model can work but still needs backup handlers for when > all the specified handlers to unregistered. > I'm ok with submerging registerDefault into the implementation used > only by the system Termination class. > I don't think the use cases call for a stack and it makes the API more > complex for register/unregister. > > I'd rather see a good simple j.u.Signal API that meets the > requirements without considering how to > support the legacy s.m.Signal, if it needs a bit of cruft to support > it for a while that's ok. > Someday we will want to rip out all that code and have a simple API. Here's another attempt that removes the possibility to chain handlers, but keeps a stack of previous handlers so that unregistration returns to exactly the same state as before registration. j.l.System can therefore use plain register() method for the default handlers and nesting registrations/unregistrations are possible: http://cr.openjdk.java.net/~plevart/jdk9-dev/Signal/webrev.02/ The API is practically the same as your's with the exception of the specification of unregister() which documents the restoration of previous state and does not mention any "default" handlers. The implementation is much simpler now (counting the lines of NativeSignal interface + j.u.Singnal class vs. your SignalImpl + j.u.Signal results in 392 vs. 513) What do you think? Regards, Peter > > Roger > > >> >> What do you think? >> >> Regards, Peter >> >>> >>> Thanks, Roger >>> >>> >>>> >>>> >>>> public final class Signal { >>>> >>>> private static final ConcurrentMap >>>> SIGNAL_BY_NAME = new ConcurrentHashMap<>(4); >>>> private static final ConcurrentMap >>>> SIGNAL_BY_NUMBER = new ConcurrentHashMap<>(4); >>>> >>>> public static Signal of(String name) { >>>> Signal signal = SIGNAL_BY_NAME.get(name); >>>> if (signal != null) { >>>> return signal; >>>> } >>>> >>>> int number; >>>> if (!name.startsWith("SIG") || name.length() <= 3 || >>>> (number = findSignal0(name.substring(3))) < 0) { >>>> throw new UnsupportedOperationException("Unknown >>>> signal: " + name); >>>> } >>>> >>>> signal = SIGNAL_BY_NUMBER.computeIfAbsent( >>>> number, >>>> new Function() { >>>> @Override >>>> public Signal apply(Integer number) { >>>> return new Signal(name, number); >>>> } >>>> } >>>> ); >>>> >>>> SIGNAL_BY_NAME.putIfAbsent(name, signal); >>>> >>>> return signal; >>>> } >>>> >>>> private final String name; >>>> private final int number; >>>> private volatile HandlerChain handlerChain; >>>> private long savedNativeHandler; >>>> >>>> private Signal(String name, int number) { >>>> this.name = name; >>>> this.number = number; >>>> } >>>> >>>> public String name() { return name; } >>>> >>>> public int number() { return number; } >>>> >>>> public void raise() { raise0(number); } >>>> >>>> public void register(BiConsumer handler) { >>>> synchronized (this) { >>>> HandlerChain oldChain = handlerChain; >>>> handlerChain = new HandlerChain(handler, oldChain); >>>> if (oldChain == null) { >>>> // set native to dispatch to Singnal.dispatch() >>>> savedNativeHandler = handle1(2); >>>> } >>>> } >>>> } >>>> >>>> public boolean unregister(BiConsumer handler) { >>>> synchronized (this) { >>>> HandlerChain oldChain = handlerChain; >>>> if (oldChain != null && oldChain.handler == handler) { >>>> if (oldChain.next == null) { >>>> // restore saved native handler >>>> long oldNativeHandler = >>>> handle1(savedNativeHandler); >>>> assert oldNativeHandler == 2L; >>>> } >>>> handlerChain = oldChain.next; >>>> return true; >>>> } else { >>>> return false; >>>> } >>>> } >>>> } >>>> >>>> // following two should probably be hidden from public API >>>> >>>> public void nativeIgnore() { >>>> synchronized (this) { >>>> if (handlerChain == null) { >>>> handle1(1); // ignore signal >>>> } else { >>>> throw new IllegalStateException( >>>> "Can't ignore signal after handlers have >>>> already been registered."); >>>> } >>>> } >>>> } >>>> >>>> public void nativeDefault() { >>>> synchronized (this) { >>>> if (handlerChain == null) { >>>> handle1(0); // default native handler >>>> } else { >>>> throw new IllegalStateException( >>>> "Can't restore signal after handlers have >>>> already been registered."); >>>> } >>>> } >>>> } >>>> >>>> private long handle1(long nativeHandler) { >>>> long oldNativeHandler = handle0(number, nativeHandler); >>>> if (oldNativeHandler == -1L) { >>>> throw new UnsupportedOperationException( >>>> "Signal already used by VM or OS: " + name); >>>> } >>>> return oldNativeHandler; >>>> } >>>> >>>> /* >>>> * Called by the VM to execute Java signal handlers. >>>> */ >>>> private static void dispatch(int number) { >>>> Signal signal = SIGNAL_BY_NUMBER.get(number); >>>> if (signal != null) { >>>> HandlerChain handlerChain = signal.handlerChain; >>>> if (handlerChain != null) { >>>> new InnocuousThread(() -> handlerChain.accept(signal)) >>>> .start(); >>>> } >>>> } >>>> } >>>> >>>> /** >>>> * Find the signal number, given a name. >>>> * >>>> * @param sigName the signal name >>>> * @return the signal number or -1 for unknown signals. >>>> */ >>>> private static native int findSignal0(String sigName); >>>> >>>> /* Registers a native signal handler, and returns the old handler. >>>> * Handler values: >>>> * 0 default handler >>>> * 1 ignore the signal >>>> * 2 call back to Signal.dispatch >>>> * other arbitrary native signal handlers >>>> * @param nativeH the index or address of the new signal handler >>>> * @return the previous index or address >>>> */ >>>> private static native long handle0(int sig, long nativeH); >>>> >>>> /* >>>> * Raise a given signal number. >>>> * @param sig the signal number to raise >>>> */ >>>> private static native void raise0(int sig); >>>> >>>> >>>> private static class HandlerChain implements Consumer { >>>> final BiConsumer handler; >>>> final HandlerChain next; >>>> >>>> HandlerChain(BiConsumer handler, >>>> HandlerChain next) { >>>> this.handler = handler; >>>> this.next = next; >>>> } >>>> >>>> @Override >>>> public void accept(Signal signal) { >>>> handler.accept(signal, () -> { >>>> if (next != null) { >>>> next.accept(signal); >>>> } >>>> }); >>>> } >>>> } >>>> } >>>> >>>> >>>> >>>> >>>> Regards, Peter >>>> >>>> >>>> On 02/01/2016 05:02 PM, Roger Riggs wrote: >>>>> Please review an API addition to handle signals such as SIGINT, >>>>> SIGHUP, and SIGTERM. >>>>> This JEP 260 motivated alternative to sun.misc.Signal supports the >>>>> use case for >>>>> interactive applications that need to handle Control-C and other >>>>> signals. >>>>> >>>>> The new java.util.Signal class provides a settable primary signal >>>>> handler and a default >>>>> signal handler. The primary signal handler can be unregistered >>>>> and handling is restored >>>>> to the default signal handler. System initialization registers >>>>> default signal handlers >>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal >>>>> API requires >>>>> a permission if a SecurityManager is set. >>>>> >>>>> The sun.misc.Signal implementation is modified to be layered on a >>>>> common >>>>> thread and dispatch mechanism. The VM handling of native signals >>>>> is not affected. >>>>> The command option to reduce signal use by the runtime with -Xrs >>>>> is unmodified. >>>>> >>>>> The changes to hotspot are minimal to rename the hardcoded >>>>> callback to the Java >>>>> Signal dispatcher. >>>>> >>>>> Please review and comment on the API and implementation. >>>>> >>>>> javadoc: >>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>> >>>>> Webrev: >>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>> >>>>> JEP 260: >>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>> >>> >> > From xueming.shen at oracle.com Tue Feb 2 21:25:23 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 02 Feb 2016 13:25:23 -0800 Subject: RFR JDK-7071819: To support Extended Grapheme Clusters in Regex In-Reply-To: <569DEB35.7020006@oracle.com> References: <569DEB35.7020006@oracle.com> Message-ID: <56B11EC3.3010505@oracle.com> Hi, Have not heard any feedback on this one so far. I'm adding a little more to make it attractive for reviewers :-) On top of the \N now the webrev includes the proposal to add two more matchers, \X for unicode extended grapheme cluster and \b{g} for the corresponding boundary. Issue: https://bugs.openjdk.java.net/browse/JDK-7071819 Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 webrev: http://cr.openjdk.java.net/~sherman/8147531_7071819/webrev/ Thanks! Sherman On 01/18/2016 11:52 PM, Xueming Shen wrote: > Hi, > > Please help review the change to add \N support in regex. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 > webrev: http://cr.openjdk.java.net/~sherman/8147531/webrev > > This is one of the items we were planning to address via JEP111 > http://openjdk.java.net/jeps/111 > https://bugs.openjdk.java.net/browse/JDK-8046101 > > Some of the constructs had been added already in early release. I'm > planning to address the rest as individual rfe separately. > > Thanks, > Sherman From brian.burkhalter at oracle.com Tue Feb 2 21:29:47 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 2 Feb 2016 13:29:47 -0800 Subject: JDK 9 RFR of 8148891: Multiple failing javax/imageio/plugins after client integration Message-ID: <34A7AAAD-038F-437B-B9B2-E7BA3DC71339@oracle.com> Please review at your convenience. Issue: https://bugs.openjdk.java.net/browse/JDK-8148891 Patch: diff --git a/test/ProblemList.txt b/test/ProblemList.txt --- a/test/ProblemList.txt +++ b/test/ProblemList.txt @@ -315,6 +315,9 @@ ############################################################################ # jdk_imageio +javax/imageio/plugins/shared/CanWriteSequence.java generic-all +javax/imageio/plugins/tiff/MultiPageTest/MultiPageTest.java generic-all +javax/imageio/plugins/tiff/WriteToSequenceAfterAbort.java generic-all ############################################################################ Summary: Put tests in the problem list until JDK-8148454 is fixed. Thanks, Brian From philip.race at oracle.com Tue Feb 2 21:36:44 2016 From: philip.race at oracle.com (Phil Race) Date: Tue, 2 Feb 2016 13:36:44 -0800 Subject: JDK 9 RFR of 8148891: Multiple failing javax/imageio/plugins after client integration In-Reply-To: <34A7AAAD-038F-437B-B9B2-E7BA3DC71339@oracle.com> References: <34A7AAAD-038F-437B-B9B2-E7BA3DC71339@oracle.com> Message-ID: <56B1216C.3050103@oracle.com> +1 -phil. On 2/2/2016 1:29 PM, Brian Burkhalter wrote: > Please review at your convenience. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8148891 > Patch: > > diff --git a/test/ProblemList.txt b/test/ProblemList.txt > --- a/test/ProblemList.txt > +++ b/test/ProblemList.txt > @@ -315,6 +315,9 @@ > ############################################################################ > > # jdk_imageio > +javax/imageio/plugins/shared/CanWriteSequence.java generic-all > +javax/imageio/plugins/tiff/MultiPageTest/MultiPageTest.java generic-all > +javax/imageio/plugins/tiff/WriteToSequenceAfterAbort.java generic-all > > ############################################################################ > > Summary: > Put tests in the problem list until JDK-8148454 is fixed. > > Thanks, > > Brian From aleksey.shipilev at oracle.com Tue Feb 2 21:53:47 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 3 Feb 2016 00:53:47 +0300 Subject: RFR (S) 8148869: StringConcatFactory MH_INLINE_SIZED_EXACT strategy does not work with -XX:-CompactStrings In-Reply-To: References: <56B10673.2090704@oracle.com> Message-ID: <56B1256B.8000801@oracle.com> Thanks guys, pushed. -Aleksey On 02/02/2016 11:07 PM, Vladimir Ivanov wrote: > Looks good. > > Best regards, > Vladimir Ivanov > >> On 2 ????. 2016 ?., at 22:41, Aleksey Shipilev wrote: >> >> Hi, >> >> Please review another simple fix in String concat: >> https://bugs.openjdk.java.net/browse/JDK-8148869 >> >> The failure happens because MH_INLINE_SIZED_EXACT strategy has to figure >> out the coder for the String concat result, as the binary "or" of >> initial coder and all argument's (possible) coders. And it mistakenly >> starts with coder=0, which is String.LATIN1. >> >> Under -XX:+CompactStrings (default), if all concat arguments can be >> encoded in LATIN1, it should produce LATIN1 result; and if at least one >> argument is UTF16, then the result should be UTF16. >> >> The trouble comes with -XX:-CompactStrings, which should always produce >> UTF16 concat result. Then, even if all arguments can be encoded into >> LATIN1, the result should be UTF16. But it isn't, because the initial >> coder is always LATIN1! >> >> The fix is to make initial coder depend on Strings.COMPACT_STRINGS: >> http://cr.openjdk.java.net/~shade/8148869/webrev.01/ >> >> Since the failure is predicated on -XX:-CompactStrings, our current >> tests fail only on platforms where CompactStrings are turned off by >> default, e.g. AArch64. Added a new regression test that concats >> differently encodeable chars under +|-CompactStrings. >> >> Cheers, >> -Aleksey >> From stuart.marks at oracle.com Tue Feb 2 22:08:18 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 2 Feb 2016 14:08:18 -0800 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF819C.5020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> Message-ID: <56B128D2.2050301@oracle.com> Hi Roger, It will be good to get this into the JDK. Lots of people have been asking for this. I have a few comments on the API. 1) Is there a way to query the set of signals supported? This might be a Set returned by a static method, for example. I agree that signal strings outside this set shouldn't be supported. 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM explicitly. Are these required to be implemented on all platforms, or just on "unix-like" platforms, are they just examples? What signals are available on Windows? 3) raise() is spec'd to throw an exception if there's no handler registered. But wouldn't it make sense to allow it if the default handler is registered? 4) In an earlier message you said that the Signal object is a capability, so the security check is on getting a reference. It seems to me that setting a handler is in a different category from raising a signal; this suggests to me that using the same object as a capability for both should be rethought. 5) I don't understand the asymmetry between register() and unregister(). Your earlier exchanges with Chris and with Gerard touched on this, specifically, the requirement that the caller pass unregister() a reference to the old handler in order for unregistration to work. You had said this was safer, if there are uncoordinated pieces of code attempting to set/unset signal handlers. It looks to me like this API is really about maintaining process global state consisting of a single handler -- user-specified or default -- for each supported signal. (I agree that it shouldn't try to have a stack or a chain of handlers.) There are a few other things that are global like this, such as the security manager and policy, System.setIn/Out/Err, and so forth. As such, uncoordinated access to the signal API is pretty much broken no matter what. Thus I don't think it makes sense to have a CAS-like protocol for unregistering a handler, to protect against the case where "somebody else" might have registered a handler different from yours. Something like this might make sense: void register(Consumer handler); void unregister(); The register() call would be pretty much as currently specified; the unregister() call would restore the default handler. Alternatively, register(null) could be used instead of unregister(), but this is quite minor. Thanks, s'marks On 2/1/16 8:02 AM, Roger Riggs wrote: > Please review an API addition to handle signals such as SIGINT, SIGHUP, and > SIGTERM. > This JEP 260 motivated alternative to sun.misc.Signal supports the use case for > interactive applications that need to handle Control-C and other signals. > > The new java.util.Signal class provides a settable primary signal handler and a > default > signal handler. The primary signal handler can be unregistered and handling is > restored > to the default signal handler. System initialization registers default signal > handlers > to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires > a permission if a SecurityManager is set. > > The sun.misc.Signal implementation is modified to be layered on a common > thread and dispatch mechanism. The VM handling of native signals is not affected. > The command option to reduce signal use by the runtime with -Xrs is unmodified. > > The changes to hotspot are minimal to rename the hardcoded callback to the Java > Signal dispatcher. > > Please review and comment on the API and implementation. > > javadoc: > http://cr.openjdk.java.net/~rriggs/signal-doc/ > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8087286 > > JEP 260: > https://bugs.openjdk.java.net/browse/JDK-8132928 > > Thanks, Roger > > From stuart.marks at oracle.com Tue Feb 2 22:47:39 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 2 Feb 2016 14:47:39 -0800 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56AF819C.5020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> Message-ID: <56B1320B.8040108@oracle.com> A small implementation issue. I couldn't find anything in the code or the spec about whether the thread that executes a handler is or isn't a daemon thread. My first inclination was to make sure that the handler threads are daemon threads, similar to most system-created threads. But in thinking about use cases, a handler might want to do some cleanup before saving state and exiting. The cleanup might end up causing all non-daemon threads to exit, which would pull the rug out from under the handler thread while it's in the midst of further cleanup or saving state. That would be bad. Thus I'm thinking that the handler threads should be non-daemon threads, and further that this should be specified. What do you think? s'marks On 2/1/16 8:02 AM, Roger Riggs wrote: > Please review an API addition to handle signals such as SIGINT, SIGHUP, and > SIGTERM. > This JEP 260 motivated alternative to sun.misc.Signal supports the use case for > interactive applications that need to handle Control-C and other signals. > > The new java.util.Signal class provides a settable primary signal handler and a > default > signal handler. The primary signal handler can be unregistered and handling is > restored > to the default signal handler. System initialization registers default signal > handlers > to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API requires > a permission if a SecurityManager is set. > > The sun.misc.Signal implementation is modified to be layered on a common > thread and dispatch mechanism. The VM handling of native signals is not affected. > The command option to reduce signal use by the runtime with -Xrs is unmodified. > > The changes to hotspot are minimal to rename the hardcoded callback to the Java > Signal dispatcher. > > Please review and comment on the API and implementation. > > javadoc: > http://cr.openjdk.java.net/~rriggs/signal-doc/ > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8087286 > > JEP 260: > https://bugs.openjdk.java.net/browse/JDK-8132928 > > Thanks, Roger > > From Roger.Riggs at Oracle.com Tue Feb 2 23:04:22 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 18:04:22 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B128D2.2050301@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> Message-ID: <56B135F6.9040204@Oracle.com> Hi Stuart, From your later email; all threads to invoke signal handlers are non-daemon threads and are expected to return from the handler and terminate. Yes, worth a spec. On 2/2/2016 5:08 PM, Stuart Marks wrote: > Hi Roger, > > It will be good to get this into the JDK. Lots of people have been > asking for this. > > I have a few comments on the API. > > 1) Is there a way to query the set of signals supported? This might be > a Set returned by a static method, for example. I agree that > signal strings outside this set shouldn't be supported. Remember; "as simple and small as possible"... If you want to use a signal try it; if it throws an exception; then its not available. > > 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM > explicitly. Are these required to be implemented on all platforms, or > just on "unix-like" platforms, are they just examples? What signals > are available on Windows? It also says, everything is OS and implementation dependent. So no not required; SIGHUP is not supported on windows. > > 3) raise() is spec'd to throw an exception if there's no handler > registered. But wouldn't it make sense to allow it if the default > handler is registered? The default handlers are all known to shutdown; if you want to exit the Java runtime, call exit() or shutdown. The requirement to have a registered handler is a bit of caution to keep the behavior more stable. > > 4) In an earlier message you said that the Signal object is a > capability, so the security check is on getting a reference. It seems > to me that setting a handler is in a different category from raising a > signal; this suggests to me that using the same object as a capability > for both should be rethought. That suggests there should be two permissions and there is a different security context expected for handlers that throwers. I think that makes it more complicated than necessary. In most OS, there is no difference in the protections for throwing and catching exceptions. > > 5) I don't understand the asymmetry between register() and > unregister(). Your earlier exchanges with Chris and with Gerard > touched on this, specifically, the requirement that the caller pass > unregister() a reference to the old handler in order for > unregistration to work. You had said this was safer, if there are > uncoordinated pieces of code attempting to set/unset signal handlers. > > It looks to me like this API is really about maintaining process > global state consisting of a single handler -- user-specified or > default -- for each supported signal. (I agree that it shouldn't try > to have a stack or a chain of handlers.) There are a few other things > that are global like this, such as the security manager and policy, > System.setIn/Out/Err, and so forth. As such, uncoordinated access to > the signal API is pretty much broken no matter what. Thus I don't > think it makes sense to have a CAS-like protocol for unregistering a > handler, to protect against the case where "somebody else" might have > registered a handler different from yours. > > Something like this might make sense: > > void register(Consumer handler); > void unregister(); yes, that would be simpler; if there is more than one Signal handler in the process; its pretty close to undefined about the interactions. > > The register() call would be pretty much as currently specified; the > unregister() call would restore the default handler. Alternatively, > register(null) could be used instead of unregister(), but this is > quite minor. Great, we're down to: of(), register(h), unregister(), name(), number(), equals(), hashcode(), toString(). I'll take another pass at tomorrow. Thanks, Roger > > Thanks, > > s'marks > > > > > > On 2/1/16 8:02 AM, Roger Riggs wrote: >> Please review an API addition to handle signals such as SIGINT, >> SIGHUP, and >> SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the >> use case for >> interactive applications that need to handle Control-C and other >> signals. >> >> The new java.util.Signal class provides a settable primary signal >> handler and a >> default >> signal handler. The primary signal handler can be unregistered and >> handling is >> restored >> to the default signal handler. System initialization registers >> default signal >> handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >> requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is >> not affected. >> The command option to reduce signal use by the runtime with -Xrs is >> unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback >> to the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> From Roger.Riggs at Oracle.com Tue Feb 2 23:14:12 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 2 Feb 2016 18:14:12 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B1188E.90602@gmail.com> References: <56AF819C.5020905@Oracle.com> <56B09684.6010107@gmail.com> <56B0C859.8070504@Oracle.com> <56B0D5DD.3070501@gmail.com> <56B0DD37.4090407@Oracle.com> <56B1188E.90602@gmail.com> Message-ID: <56B13844.2020602@Oracle.com> Hi Peter, I'll take a deeper look tomorrow, I spent some time today updating my prototype with the suggestions. And there is more to iterate over tomorrow. Updated webrev: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ ... On 2/2/2016 3:58 PM, Peter Levart wrote: > > > On 02/02/2016 05:45 PM, Roger Riggs wrote: >> Hi Peter, >> >> >> On 2/2/2016 11:14 AM, Peter Levart wrote: >>> Hi Roger, >>> >>> On 02/02/2016 04:16 PM, Roger Riggs wrote: >>>> Hi Peter, >>>> >>>> On 2/2/2016 6:44 AM, Peter Levart wrote: >>>>> >>>> ... >>>>> >>>>> Also if this is to become public API, There's a chance users would >>>>> want to add a handler to the chain of existing handlers or >>>>> override them. So what about an API that allows >>>>> registering/unregistering a default (non-native) handler and other >>>>> handlers above it in a uniform way, like: >>>> The problem with chaining, as in the current API, is that there is >>>> no way to know what the next >>>> handler in the chain will do. If it is the default one for INT, >>>> TERM, HUP, it will call Shutdown and exit. >>>> So without extra information and cooperation chaining is risky. >>>> If the handler knows something about the other actors in the >>>> environment, it can coordinate with them directly. >>>> For the use cases that have been raised for existing use of >>>> sun.misc.Signal, they are simple interactive >>>> environments that want to give the appearance of being able to >>>> interrupt using control-c. >>>> >>>> I've been aiming for the simplest API that would support the >>>> current use cases. >>> >>> I noticed that sun.misc.Signal[Handler] is a critical API according >>> to JEP 260, so it can't be removed in JDK9 yet. I wanted to see if >>> sun.misc.Signal[Handler] could be modified to use java.util.Signal >>> directly while making java.util.Signal support chaining and >>> restoring of previous handler on deregister. Here's what it looks >>> like in code: >>> >>> http://cr.openjdk.java.net/~plevart/jdk9-dev/Signal/webrev.01/ >>> >>> But if chaining is not desired, then at least restoring of previous >>> handler could be implemented in a uniform way and for arbitrary >>> registration depth (no need for register/registerDefault distinction). >> >> A stack based model can work but still needs backup handlers for when >> all the specified handlers to unregistered. >> I'm ok with submerging registerDefault into the implementation used >> only by the system Termination class. >> I don't think the use cases call for a stack and it makes the API >> more complex for register/unregister. >> >> I'd rather see a good simple j.u.Signal API that meets the >> requirements without considering how to >> support the legacy s.m.Signal, if it needs a bit of cruft to support >> it for a while that's ok. >> Someday we will want to rip out all that code and have a simple API. > > Here's another attempt that removes the possibility to chain handlers, > but keeps a stack of previous handlers so that unregistration returns > to exactly the same state as before registration. j.l.System can > therefore use plain register() method for the default handlers and > nesting registrations/unregistrations are possible: > > http://cr.openjdk.java.net/~plevart/jdk9-dev/Signal/webrev.02/ > > The API is practically the same as your's with the exception of the > specification of unregister() which documents the restoration of > previous state and does not mention any "default" handlers. > > The implementation is much simpler now (counting the lines of > NativeSignal interface + j.u.Singnal class vs. your SignalImpl + > j.u.Signal results in 392 vs. 513) It looks interesting, though a different style implementation. And I'd rather use mundane synchronization primitives that keep the code easier to read, even if a bit less optimized. Thanks, Roger > > What do you think? > > > Regards, Peter > >> From iris.clark at oracle.com Tue Feb 2 23:16:27 2016 From: iris.clark at oracle.com (Iris Clark) Date: Tue, 2 Feb 2016 15:16:27 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <01a501d15d8e$1b683b50$5238b1f0$@tma.com.vn> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> <04d501d14e7f$1ee19b50$5ca4d1f0$@tma.com.vn> <7b5a0fda-6312-41ba-b211-ea56ae7031f1@default> <01a501d15d8e$1b683b50$5238b1f0$@tma.com.vn> Message-ID: Hi. Thanks again for looking at this change. > The JEP & implementation allows -$OPT to be specified without +, but the > Javadoc one does not allow that. For example, "9-pre-opt" is allowed by > JEP, but disallowed by Javadoc. The JavaDoc and JEP are both slightly wrong in different ways. The implementation is correct. Note that it enforces additional constraints beyond just the regular expression. I've added a few more regression tests to verify that the behavior is as expected. This is the updated webrev and javadoc: http://cr.openjdk.java.net/~iris/verona/8072379/webrev.4/ http://cr.openjdk.java.net/~iris/verona/8072379/doc.4/jdk/Version.html (unchanged from doc.3) > Understood, but I didn't see any part of the JEP or the Javadoc explaining > that + is needed to make the parser recognize the text followed as options > instead of pre-release identifier. It would be great if that is added. I've created a new bug to cover updating both the javadoc and JEP for this case. https://bugs.openjdk.java.net/browse/JDK-8148877 JDK-8148877: (spec) Specify when an empty '+' is required in a version string This is the key sentence in the description: '+' is required for empty build when OPT is present and PRE is not. "9+-opt" and "9-pre-opt" are legal. "9+" is not. Regards, iris -----Original Message----- From: Thanh Hong Dai [mailto:hdthanh at tma.com.vn] Sent: Monday, February 01, 2016 11:48 PM To: Iris Clark; Alan Bateman; core-libs-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Dear Iris, On closer look, there seems to be some conflicting definition of version string. In JEP: http://openjdk.java.net/jeps/223 $VNUM(-$PRE)?(\+$BUILD)?(-$OPT)? In Javadoc: http://cr.openjdk.java.net/~iris/verona/8072379/doc.3/jdk/Version.html $VNUM(-$PRE)?(\+($BUILD)?(-$OPT)?)? In implementation: The regex follows JEP's definition. The JEP & implementation allows -$OPT to be specified without +, but the Javadoc one does not allow that. For example, "9-pre-opt" is allowed by JEP, but disallowed by Javadoc. > I need to capture the plus to distinguish between cases where an empty build is allowed (e.g. "9+-foo") and when it is not ("9+"). > See code in Version.java, line 226-230 and in Basic.java, line 98, > 107-109. (Note that we use the empty "+" to distinguish "9-foo" from > "9+-foo".) Understood, but I didn't see any part of the JEP or the Javadoc explaining that + is needed to make the parser recognize the text followed as options instead of pre-release identifier. It would be great if that is added. Best regards, Hong Dai Thanh. From steve.drach at oracle.com Wed Feb 3 02:24:07 2016 From: steve.drach at oracle.com (Steve Drach) Date: Tue, 2 Feb 2016 18:24:07 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <8E943A53-0D6A-4688-A3C8-B35D6EB99766@oracle.com> <448E1831-7142-4B95-B0D8-D07A34704D32@oracle.com> <563F6F99.6070000@oracle.com> <563FAEB6.4000401@oracle.com> <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> Message-ID: > I think the following is a reasonable solution. > > If the JarFile is constructed with any of the 5 constructors that do not contain a Release argument, then entries()/stream() returns the set of all entries in the jar file including those under the META-INF/versions/directory. The base entries are ?raw?, they are not aliases for versioned entries. > > If the JarFile is constructed with the 1 constructor that does include a Release argument, then entries()/stream() returns the set of appropriately versioned entries that would result from invoking getEntry()/getJarEntry() with the name of each base entry. The entries in the tree rooted at the directory META-INF/versions/ are not returned. I created a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.05/ , that implements what I outlined above. In particular I enhanced the JarEntryIterator class and I added additional commentary to the entries() and stream() methods. I also added a new test, MultiReleaseJarIterators, to test entries() and stream(). > >> On Feb 1, 2016, at 12:29 PM, Steve Drach wrote: >> >> I?m sorry, I didn?t look at the code close enough before I started talking ;-) Right now entries()/stream() returns all entries and if the JarFile is constructed with a Release object != Release.BASE, the base entries that are returned are the versioned entries. I think this behavior is a bit confusing and we should just return all entries without regard to versioning. Then create the two new methods for specific versioned entries. >> >>> On Feb 1, 2016, at 12:18 PM, Steve Drach wrote: >>> >>>>>>> Alan?s point is that traversing using entries()/stream() always returns the versioned entries (if any) rather than all entries, thus in a sense filters. >>>>>>> >>>>>>> My assumption was the traversal should by default be consistent with a calls to getEntry, thus: >>>>>>> >>>>>>> jarFile.stream().forEach(e -> { >>>>>>> JarEntry je = jarFile.getJarEntry(e.getName()); >>>>>>> assert e.equals(je); >>>>>>> }); >>>>>>> >>>>>>> There might need to be another stream method that returns all entries. >>>>>>> >>>>>> Right, I'm mostly just wondering if entries()/streams() should override the entries in the stream with versioned entries and filter out the META-INF/versions/ tree. >>>>> I don?t think so. That kind of behavior might be difficult to understand. Returning all the entries provides some flexibility. One can write code like this: >>>>> >>>>> jarfile.stream().map(JarEntry::getName).filter(s -> !s.startsWith(?META-INF?)).map(JarFile::getJarEntry).etc >>>>> >>>>> to get the versioned results for any version you specify when constructing the JarFile. >>>> >>>> The current specification treats those class files under meta-inf/releases like >>>> kind of "metadata" of those base entries. Ideally those files should not even >>>> be individual "files", but part of their corresponding entries. The consumer of >>>> the MR-Jar should not need to be aware of these version-ed entries at all to use >>>> this MR-jar file to load classes/resources. From this point of view, these entries >>>> probably should be "invisible" from entries()/stream(), when the jar file is opened >>>> with "version-enabled". And all returned entries should have all their "data" >>>> (size, csize, timestamps, crc ...) pointed to the corresponding version-ed entries, >>>> withe the only exception is the "name". >>>> >>>> On the other hand it might be desired to keep JarFile.entries()/stream() asis to >>>> match their "zip file" perspective, to return "all" raw entries. Then it might also >>>> be desired to have an alternative "versioned streamVersion()" ? >>> >>> It seems to that we have two reasonable alternatives: (1) return all entries, and (2) return all entries except those under the ?META-INF/versions/? directory and for any entries returned, return their versioned equivalent if it exists. If we choose alternative 2, we can still get alternative 1 by asking for JarFile.super.entries() and JarFile.super.stream(). >>> >>> Or we can do it both ways, leaving entries()/stream() as is and adding two new methods, versionedEntries() and versionedStream(). >>> >>>> >>>> something like >>>> >>>> public Stream stream(Release r); ? >>> >>> We should not parametrize the methods with a Release, because what does it mean if we construct the JarFile with one Release but specify a different Release for the stream argument. Parameterizing methods with a Release object feels like we?re starting to slide down a slippery slope. >>> >>> I think adding the two new methods is the ?right? solution, but I?d like some consensus here. >>> >>>> >>>> -sherman >>>> >>>> >>>> >>>> >>>>>> If I've gone to trouble of specifying the a Release then it seems the right thing to do. On the other hand, it comes at a cost and there will be use-cases like "get the names of all entries" that would be more efficient to just build on the current entries()/stream(). I'm loath to suggest this might need a new method but it might be one of the options to consider here. Minimally there is a javadoc to specify on how these methods behave when the JAR is multi-release and opened by specifying a release. >>>>> How?s this? >>>>> >>>>> diff -r 68867430065b src/java.base/share/classes/java/util/jar/JarFile.java >>>>> --- a/src/java.base/share/classes/java/util/jar/JarFile.java Fri Jan 29 12:34:44 2016 -0800 >>>>> +++ b/src/java.base/share/classes/java/util/jar/JarFile.java Mon Feb 01 09:48:05 2016 -0800 >>>>> @@ -576,9 +576,11 @@ >>>>> } >>>>> >>>>> /** >>>>> - * Returns an enumeration of the jar file entries. >>>>> + * Returns an enumeration of all the jar file entries. Constructing this >>>>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >>>>> + * constructor does not modify the behavior of this method. >>>>> * >>>>> - * @return an enumeration of the jar file entries >>>>> + * @return an enumeration of the all jar file entries >>>>> * @throws IllegalStateException >>>>> * may be thrown if the jar file has been closed >>>>> */ >>>>> @@ -587,11 +589,13 @@ >>>>> } >>>>> >>>>> /** >>>>> - * Returns an ordered {@code Stream} over the jar file entries. >>>>> + * Returns an ordered {@code Stream} over all the jar file entries. >>>>> * Entries appear in the {@code Stream} in the order they appear in >>>>> - * the central directory of the jar file. >>>>> + * the central directory of the jar file. Constructing this >>>>> + * JarFile with the {@link JarFile#JarFile(File, boolean, int, Release)} >>>>> + * constructor does not modify the behavior of this method. >>>>> * >>>>> - * @return an ordered {@code Stream} of entries in this jar file >>>>> + * @return an ordered {@code Stream} of all entries in this jar file >>>>> * @throws IllegalStateException if the jar file has been closed >>>>> * @since 1.8 >>>>> */ >> > From david.holmes at oracle.com Wed Feb 3 03:05:49 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 3 Feb 2016 13:05:49 +1000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B128D2.2050301@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> Message-ID: <56B16E8D.6060208@oracle.com> On 3/02/2016 8:08 AM, Stuart Marks wrote: > Hi Roger, > > It will be good to get this into the JDK. Lots of people have been > asking for this. I think this API is a big mistake. The primary usecase seems to be control-C interception for utilities like jshell. Adding a general purpose signal raising and handling mechanism to the JDK does not seem like a good solution to me. While you would need to use signal management under the covers I think it would be much cleaner to expose an API that actually captures what it is you want here: a mechanism to manage "interrupt" and "terminate" events at the VM level, in a clean cross-platform way. Aside: If you want to see some prior art in this area look at PosixSignalHandler API in the Real-Time Specification for Java. Which reminds me - do you propose to support the POSIX real-time signals? David ----- > I have a few comments on the API. > > 1) Is there a way to query the set of signals supported? This might be a > Set returned by a static method, for example. I agree that > signal strings outside this set shouldn't be supported. > > 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM > explicitly. Are these required to be implemented on all platforms, or > just on "unix-like" platforms, are they just examples? What signals are > available on Windows? > > 3) raise() is spec'd to throw an exception if there's no handler > registered. But wouldn't it make sense to allow it if the default > handler is registered? > > 4) In an earlier message you said that the Signal object is a > capability, so the security check is on getting a reference. It seems to > me that setting a handler is in a different category from raising a > signal; this suggests to me that using the same object as a capability > for both should be rethought. > > 5) I don't understand the asymmetry between register() and unregister(). > Your earlier exchanges with Chris and with Gerard touched on this, > specifically, the requirement that the caller pass unregister() a > reference to the old handler in order for unregistration to work. You > had said this was safer, if there are uncoordinated pieces of code > attempting to set/unset signal handlers. > > It looks to me like this API is really about maintaining process global > state consisting of a single handler -- user-specified or default -- for > each supported signal. (I agree that it shouldn't try to have a stack or > a chain of handlers.) There are a few other things that are global like > this, such as the security manager and policy, System.setIn/Out/Err, and > so forth. As such, uncoordinated access to the signal API is pretty much > broken no matter what. Thus I don't think it makes sense to have a > CAS-like protocol for unregistering a handler, to protect against the > case where "somebody else" might have registered a handler different > from yours. > > Something like this might make sense: > > void register(Consumer handler); > void unregister(); > > The register() call would be pretty much as currently specified; the > unregister() call would restore the default handler. Alternatively, > register(null) could be used instead of unregister(), but this is quite > minor. > > Thanks, > > s'marks > > > > > > On 2/1/16 8:02 AM, Roger Riggs wrote: >> Please review an API addition to handle signals such as SIGINT, >> SIGHUP, and >> SIGTERM. >> This JEP 260 motivated alternative to sun.misc.Signal supports the use >> case for >> interactive applications that need to handle Control-C and other signals. >> >> The new java.util.Signal class provides a settable primary signal >> handler and a >> default >> signal handler. The primary signal handler can be unregistered and >> handling is >> restored >> to the default signal handler. System initialization registers >> default signal >> handlers >> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >> requires >> a permission if a SecurityManager is set. >> >> The sun.misc.Signal implementation is modified to be layered on a common >> thread and dispatch mechanism. The VM handling of native signals is >> not affected. >> The command option to reduce signal use by the runtime with -Xrs is >> unmodified. >> >> The changes to hotspot are minimal to rename the hardcoded callback to >> the Java >> Signal dispatcher. >> >> Please review and comment on the API and implementation. >> >> javadoc: >> http://cr.openjdk.java.net/~rriggs/signal-doc/ >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8087286 >> >> JEP 260: >> https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> Thanks, Roger >> >> From hdthanh at tma.com.vn Wed Feb 3 03:07:40 2016 From: hdthanh at tma.com.vn (Thanh Hong Dai) Date: Wed, 3 Feb 2016 10:07:40 +0700 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> <04d501d14e7f$1ee19b50$5ca4d1f0$@tma.com.vn> <7b5a0fda-6312-41ba-b211-ea56ae7031f1@default> <01a501d15d8e$1b683b50$5238b1f0$@tma.com.vn> Message-ID: <020501d15e30$0b533a60$21f9af20$@tma.com.vn> Dear Iris, > The JavaDoc and JEP are both slightly wrong in different ways. The implementation is correct. Note that it enforces additional constraints beyond just the regular expression. It would be great if those additional constraints are also documented outside the code (JavaDoc or JEP). Also, please update the JavaDoc at line 48, 113, optionally line 130 of Version.java, plus the comment at line 160 of Version.java to match the JEP, since you confirm "9-pre-opt" is valid. Best regards, Hong Dai Thanh. -----Original Message----- From: Iris Clark [mailto:iris.clark at oracle.com] Sent: Wednesday, 3 February, 2016 6:16 AM To: Thanh Hong Dai ; Alan Bateman ; core-libs-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi. Thanks again for looking at this change. > The JEP & implementation allows -$OPT to be specified without +, but > the Javadoc one does not allow that. For example, "9-pre-opt" is > allowed by JEP, but disallowed by Javadoc. The JavaDoc and JEP are both slightly wrong in different ways. The implementation is correct. Note that it enforces additional constraints beyond just the regular expression. I've added a few more regression tests to verify that the behavior is as expected. This is the updated webrev and javadoc: http://cr.openjdk.java.net/~iris/verona/8072379/webrev.4/ http://cr.openjdk.java.net/~iris/verona/8072379/doc.4/jdk/Version.html (unchanged from doc.3) > Understood, but I didn't see any part of the JEP or the Javadoc > explaining that + is needed to make the parser recognize the text > followed as options instead of pre-release identifier. It would be great if that is added. I've created a new bug to cover updating both the javadoc and JEP for this case. https://bugs.openjdk.java.net/browse/JDK-8148877 JDK-8148877: (spec) Specify when an empty '+' is required in a version string This is the key sentence in the description: '+' is required for empty build when OPT is present and PRE is not. "9+-opt" and "9-pre-opt" are legal. "9+" is not. Regards, iris -----Original Message----- From: Thanh Hong Dai [mailto:hdthanh at tma.com.vn] Sent: Monday, February 01, 2016 11:48 PM To: Iris Clark; Alan Bateman; core-libs-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Dear Iris, On closer look, there seems to be some conflicting definition of version string. In JEP: http://openjdk.java.net/jeps/223 $VNUM(-$PRE)?(\+$BUILD)?(-$OPT)? In Javadoc: http://cr.openjdk.java.net/~iris/verona/8072379/doc.3/jdk/Version.html $VNUM(-$PRE)?(\+($BUILD)?(-$OPT)?)? In implementation: The regex follows JEP's definition. The JEP & implementation allows -$OPT to be specified without +, but the Javadoc one does not allow that. For example, "9-pre-opt" is allowed by JEP, but disallowed by Javadoc. > I need to capture the plus to distinguish between cases where an empty build is allowed (e.g. "9+-foo") and when it is not ("9+"). > See code in Version.java, line 226-230 and in Basic.java, line 98, > 107-109. (Note that we use the empty "+" to distinguish "9-foo" from > "9+-foo".) Understood, but I didn't see any part of the JEP or the Javadoc explaining that + is needed to make the parser recognize the text followed as options instead of pre-release identifier. It would be great if that is added. Best regards, Hong Dai Thanh. From iris.clark at oracle.com Wed Feb 3 03:14:09 2016 From: iris.clark at oracle.com (Iris Clark) Date: Tue, 2 Feb 2016 19:14:09 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <020501d15e30$0b533a60$21f9af20$@tma.com.vn> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <56951E9D.2040509@oracle.com> <04d501d14e7f$1ee19b50$5ca4d1f0$@tma.com.vn> <7b5a0fda-6312-41ba-b211-ea56ae7031f1@default> <01a501d15d8e$1b683b50$5238b1f0$@tma.com.vn> <020501d15e30$0b533a60$21f9af20$@tma.com.vn> Message-ID: <728ef8ca-3ad2-4797-8db7-ad3d462e8255@default> Hi. > Also, please update the JavaDoc at line 48, 113, optionally line 130 > of Version.java, plus the comment at line 160 of Version.java to > match the JEP, since you confirm "9-pre-opt" is valid. I've added the specific Version.java line numbers to 8148877. My sponsor will push the changes for 8072379 soon. Regards, Iris -----Original Message----- From: Thanh Hong Dai [mailto:hdthanh at tma.com.vn] Sent: Tuesday, February 02, 2016 7:08 PM To: Iris Clark; Alan Bateman; core-libs-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Dear Iris, > The JavaDoc and JEP are both slightly wrong in different ways. The implementation is correct. Note that it enforces additional constraints beyond just the regular expression. It would be great if those additional constraints are also documented outside the code (JavaDoc or JEP). Also, please update the JavaDoc at line 48, 113, optionally line 130 of Version.java, plus the comment at line 160 of Version.java to match the JEP, since you confirm "9-pre-opt" is valid. Best regards, Hong Dai Thanh. -----Original Message----- From: Iris Clark [mailto:iris.clark at oracle.com] Sent: Wednesday, 3 February, 2016 6:16 AM To: Thanh Hong Dai ; Alan Bateman ; core-libs-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi. Thanks again for looking at this change. > The JEP & implementation allows -$OPT to be specified without +, but > the Javadoc one does not allow that. For example, "9-pre-opt" is > allowed by JEP, but disallowed by Javadoc. The JavaDoc and JEP are both slightly wrong in different ways. The implementation is correct. Note that it enforces additional constraints beyond just the regular expression. I've added a few more regression tests to verify that the behavior is as expected. This is the updated webrev and javadoc: http://cr.openjdk.java.net/~iris/verona/8072379/webrev.4/ http://cr.openjdk.java.net/~iris/verona/8072379/doc.4/jdk/Version.html (unchanged from doc.3) > Understood, but I didn't see any part of the JEP or the Javadoc > explaining that + is needed to make the parser recognize the text > followed as options instead of pre-release identifier. It would be great if that is added. I've created a new bug to cover updating both the javadoc and JEP for this case. https://bugs.openjdk.java.net/browse/JDK-8148877 JDK-8148877: (spec) Specify when an empty '+' is required in a version string This is the key sentence in the description: '+' is required for empty build when OPT is present and PRE is not. "9+-opt" and "9-pre-opt" are legal. "9+" is not. Regards, iris -----Original Message----- From: Thanh Hong Dai [mailto:hdthanh at tma.com.vn] Sent: Monday, February 01, 2016 11:48 PM To: Iris Clark; Alan Bateman; core-libs-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Dear Iris, On closer look, there seems to be some conflicting definition of version string. In JEP: http://openjdk.java.net/jeps/223 $VNUM(-$PRE)?(\+$BUILD)?(-$OPT)? In Javadoc: http://cr.openjdk.java.net/~iris/verona/8072379/doc.3/jdk/Version.html $VNUM(-$PRE)?(\+($BUILD)?(-$OPT)?)? In implementation: The regex follows JEP's definition. The JEP & implementation allows -$OPT to be specified without +, but the Javadoc one does not allow that. For example, "9-pre-opt" is allowed by JEP, but disallowed by Javadoc. > I need to capture the plus to distinguish between cases where an empty build is allowed (e.g. "9+-foo") and when it is not ("9+"). > See code in Version.java, line 226-230 and in Basic.java, line 98, > 107-109. (Note that we use the empty "+" to distinguish "9-foo" from > "9+-foo".) Understood, but I didn't see any part of the JEP or the Javadoc explaining that + is needed to make the parser recognize the text followed as options instead of pre-release identifier. It would be great if that is added. Best regards, Hong Dai Thanh. From iris.clark at oracle.com Wed Feb 3 03:17:30 2016 From: iris.clark at oracle.com (Iris Clark) Date: Tue, 2 Feb 2016 19:17:30 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> Message-ID: <2f3ec2a8-4488-4c21-ae79-01c4599d5121@default> Hi, Mandy. Thanks so much for pushing the changeset for the initial implementation of jdk.Version! Regards, iris -----Original Message----- From: Iris Clark Sent: Monday, January 11, 2016 1:45 PM To: Iris Clark; Joe Darcy; Mandy Chung; Magnus Ihse Bursie; Roger Riggs; Alan Bateman Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Joe, Roger, Alan, Magnus, and Mandy. At the end of December (shortly before the Christmas/Winter break and my vacation), I provided responses to your messages and an updated webrev: http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ I didn't hear from anybody, so I'd like to optimistically assume that you were satisfied. Is that correct? For you convenience, here's a reference to the December and November threads: http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036904.html I'd like to wrap up this work for the initial implementation of jdk.Version soon. Regards, iris -----Original Message----- From: Iris Clark Sent: Friday, December 18, 2015 1:55 PM To: Joe Darcy; Mandy Chung Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion Hi, Joe. Thanks for the review comments. >>> http://cr.openjdk.java.net/~irisa/verona/8072379/webrev.1/ > Is the intention that downstream JDK distributions, such as IcedTea, > whether based on OpenJDK or otherwise, would provide their own > specialization of the jdk.Version class? No. Downstream users may provide their own specialization, but there is no requirement that they must do so. jdk.OracleVersion has been removed. The functionality it provided, access to the fourth element of the version number, is trivially accessed by existing methods in jdk.Version. I've filed the following bug to address the more general use case: 8145793: Provide vendor-specific interpretation of a JDK version string https://bugs.openjdk.java.net/browse/JDK-8145793 > The equals / hashCode behavior of Version and OracleVersion is bit > surprising and I think somewhat underspecified given the possibility > of defining subclasses. Given the above regarding downstream use, I've make jdk.Version final and the constructor is now private. Oracle or any other JDK distribution wishing to impose additional semantics to the version string interpretation will need to do so via encapsulation rather than inheritance. > How is this API supposed to behave if the component version strings > have a numerical value greater than Integer.MAX_VALUE? >From the specification of the only instance method, Version.parse(): * @throws NumberFormatException * If an element of the version number or the build number cannot * be represented as an {@link Integer} > Was using Longs to record numerical versions rather than Integers > considered? Yes. In an informal poll conducted during implementation, it was thought that 2^32 bits would be more than sufficient to represent the components of typical version numbers. An updated webrev is forthcoming. Thanks, iris PS: I believe that we are both out the week of 21 December, so I don't expect to hear back from you until the New Year. From mandy.chung at oracle.com Wed Feb 3 03:19:35 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 2 Feb 2016 19:19:35 -0800 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <2f3ec2a8-4488-4c21-ae79-01c4599d5121@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <2f3ec2a8-4488-4c21-ae79-01c4599d5121@default> Message-ID: Sure thing. Glad to see this going in. Mandy > On Feb 2, 2016, at 7:17 PM, Iris Clark wrote: > > Hi, Mandy. > > Thanks so much for pushing the changeset for the initial implementation > of jdk.Version! > > Regards, > iris > > -----Original Message----- > From: Iris Clark > Sent: Monday, January 11, 2016 1:45 PM > To: Iris Clark; Joe Darcy; Mandy Chung; Magnus Ihse Bursie; Roger Riggs; Alan Bateman > Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net > Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion > > Hi, Joe, Roger, Alan, Magnus, and Mandy. > > At the end of December (shortly before the Christmas/Winter break and my vacation), I provided responses to your messages and an updated webrev: > > http://cr.openjdk.java.net/~iris/verona/8072379/webrev.2/ > > I didn't hear from anybody, so I'd like to optimistically assume that you were satisfied. Is that correct? > > For you convenience, here's a reference to the December and November > threads: > > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-December/037062.html > http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036904.html > > I'd like to wrap up this work for the initial implementation of jdk.Version soon. > > Regards, > iris > > -----Original Message----- > From: Iris Clark > Sent: Friday, December 18, 2015 1:55 PM > To: Joe Darcy; Mandy Chung > Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net > Subject: RE: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion > > Hi, Joe. > > Thanks for the review comments. > >>>> http://cr.openjdk.java.net/~irisa/verona/8072379/webrev.1/ > >> Is the intention that downstream JDK distributions, such as IcedTea, >> whether based on OpenJDK or otherwise, would provide their own >> specialization of the jdk.Version class? > > No. Downstream users may provide their own specialization, but there is no requirement that they must do so. > > jdk.OracleVersion has been removed. The functionality it provided, access to the fourth element of the version number, is trivially accessed by existing methods in jdk.Version. > > I've filed the following bug to address the more general use case: > > 8145793: Provide vendor-specific interpretation of a JDK version string > https://bugs.openjdk.java.net/browse/JDK-8145793 > >> The equals / hashCode behavior of Version and OracleVersion is bit >> surprising and I think somewhat underspecified given the possibility >> of defining subclasses. > > Given the above regarding downstream use, I've make jdk.Version final and the constructor is now private. Oracle or any other JDK distribution wishing to impose additional semantics to the version string interpretation will need to do so via encapsulation rather than inheritance. > >> How is this API supposed to behave if the component version strings >> have a numerical value greater than Integer.MAX_VALUE? > > From the specification of the only instance method, Version.parse(): > > * @throws NumberFormatException > * If an element of the version number or the build number cannot > * be represented as an {@link Integer} > >> Was using Longs to record numerical versions rather than Integers >> considered? > > Yes. In an informal poll conducted during implementation, it was thought that 2^32 bits would be more than sufficient to represent the components of typical version numbers. > > An updated webrev is forthcoming. > > Thanks, > iris > > PS: I believe that we are both out the week of 21 December, so I don't expect to hear back from you until the New Year. From Alan.Bateman at oracle.com Wed Feb 3 08:52:51 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Feb 2016 08:52:51 +0000 Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <2f3ec2a8-4488-4c21-ae79-01c4599d5121@default> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <2f3ec2a8-4488-4c21-ae79-01c4599d5121@default> Message-ID: <56B1BFE3.1010201@oracle.com> On 03/02/2016 03:17, Iris Clark wrote: > Hi, Mandy. > > Thanks so much for pushing the changeset for the initial implementation > of jdk.Version! > Good to have this in but now we need to decide on where it should live. It's JDK-specific so we'll need it exported by a JDK module rather than java.base. -Alan From Alan.Bateman at oracle.com Wed Feb 3 09:05:16 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Feb 2016 09:05:16 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> Message-ID: <56B1C2CC.7080102@oracle.com> On 03/02/2016 02:24, Steve Drach wrote: > I created a new webrev, > http://cr.openjdk.java.net/~sdrach/8132734/webrev.05/ > , that > implements what I outlined above. In particular I enhanced the > JarEntryIterator class and I added additional commentary to the > entries() and stream() methods. I also added a new test, > MultiReleaseJarIterators, to test entries() and stream(). > I think having stream and entries do this is right although I would like to see some performance data if possible. Also I would expect that if a JAR file is not mult-release but the library opens it with Release.RUNTIME to perform the same as opening the JAR file with the Release-less constructors. I think the javadoc will need to also need to make it clear whether entries with names starting with META-INF/versions/ are returned. I see you've added @since 9 to the existing methods, I assume you didn't mean to do this. At some point then we need to discuss how RUNTIME_VERSION is computed. Iris (via Mandy) has pushed jdk.Version to jdk9/dev but having it exported by java.base conflicts with the design principles in JEP 200. Moving it to another module means that code in java.base cannot use it and thus the JAR file can't use it. -Alan. From aph at redhat.com Wed Feb 3 09:43:03 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 3 Feb 2016 09:43:03 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B102AD.7020800@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> Message-ID: <56B1CBA7.4050902@redhat.com> On 02/02/16 19:25, Mikael Vidstedt wrote: > Please review this change which introduces a Copy::conjoint_swap and an > Unsafe.copySwapMemory method to call it from Java, along with the > necessary changes to have java.nio.Bits call it instead of the Bits.c code. There doesn't seem to be any way to use a byte-swap instruction in the swapping code. This will make it unnecessarily slow. Andrew. From scolebourne at joda.org Wed Feb 3 10:52:13 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 3 Feb 2016 10:52:13 +0000 Subject: RFR:JDK-8146747:java.time.Duration.toNanos() and toMillis() exception on negative durations In-Reply-To: <56AEF8B8.4070703@oracle.com> References: <56AEF8B8.4070703@oracle.com> Message-ID: Yes, looks fine thanks Stephen On 1 February 2016 at 06:18, nadeesh tv wrote: > Hi all, > > Please review following > > Bug Id : https://bugs.openjdk.java.net/browse/JDK-8146747 > > Solution: Handled the negative duration separately > > webrev : http://cr.openjdk.java.net/~ntv/8146747/webrev.00/ > > -- > Thanks and Regards, > Nadeesh TV > From peter.levart at gmail.com Wed Feb 3 11:07:15 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 3 Feb 2016 12:07:15 +0100 Subject: RFR JDK-7071819: To support Extended Grapheme Clusters in Regex In-Reply-To: <56B11EC3.3010505@oracle.com> References: <569DEB35.7020006@oracle.com> <56B11EC3.3010505@oracle.com> Message-ID: <56B1DF63.2040708@gmail.com> Hi Sherman, I don't currently have any idea how to squeeze the hashtable any more further. It is already very compact in my opinion. But I noticed a data race that could result in navigating the half-initialized data structure. It is a classical unsafe publication bug. It has been present before in get(int cp) and it is present now in both getName(int cp) and getCodePoint(String name). For example: 151 static int getCodePoint(String name) { 152 byte[] strPool = null; 153 if (refStrPool == null || (strPool = refStrPool.get()) == null) { 154 strPool = initNamePool(); 155 } vs. 111 refStrPool = new SoftReference<>(strPool); ...the static refStrPool field is not marked volatile. One way to fix this is to mark field volatile and then rearrange code in getName/getCodePoint to only read from it once by introducing a local var. The other would be to change the line 111 into something like: SoftReference rsp = new SoftReference<>(strPool); unsafe.storeFence(); refStrPool = rsp; ...*and* also rearrange code in getName/getCodePoint to only read from field once by introducing a local var. Regards, Peter On 02/02/2016 10:25 PM, Xueming Shen wrote: > Hi, > > Have not heard any feedback on this one so far. I'm adding > a little more to make it attractive for reviewers :-) > > On top of the \N now the webrev includes the proposal to add > two more matchers, \X for unicode extended grapheme cluster > and \b{g} for the corresponding boundary. > > Issue: https://bugs.openjdk.java.net/browse/JDK-7071819 > Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 > webrev: http://cr.openjdk.java.net/~sherman/8147531_7071819/webrev/ > > Thanks! > Sherman > > On 01/18/2016 11:52 PM, Xueming Shen wrote: >> Hi, >> >> Please help review the change to add \N support in regex. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 >> webrev: http://cr.openjdk.java.net/~sherman/8147531/webrev >> >> This is one of the items we were planning to address via JEP111 >> http://openjdk.java.net/jeps/111 >> https://bugs.openjdk.java.net/browse/JDK-8046101 >> >> Some of the constructs had been added already in early release. I'm >> planning to address the rest as individual rfe separately. >> >> Thanks, >> Sherman > From peter.levart at gmail.com Wed Feb 3 11:26:36 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 3 Feb 2016 12:26:36 +0100 Subject: RFR JDK-7071819: To support Extended Grapheme Clusters in Regex In-Reply-To: <56B1DF63.2040708@gmail.com> References: <569DEB35.7020006@oracle.com> <56B11EC3.3010505@oracle.com> <56B1DF63.2040708@gmail.com> Message-ID: <56B1E3EC.2060509@gmail.com> Hi Again, I also have a comment on the implementation of the hash table and it's GC-ness. You keep the string pool under a SoftReference because it is the biggest object so it makes most sense to clear it when heap becomes full. Other arrays are kept strongly reachable, but you re-generate them nevertheless when string pool is cleared by GC. Would it make sense to: - define all int[] arrays (including strPool) as final instance variables of CharacterName - have one static field: private static SoftReference refInstance; - convert initNamePool() into a private constructor. - convert public static getName/getCodePoint into public instance methods - introduce public static method: public static CharacterName getInstance() { SoftReference ref = refInstance; CharacterName instance; if (ref != null && ((instance = ref.get) != null) { return instance; } instance = new CharacterName(); refInstance = new SoftReference<>(instance); return instance; } ...in this arrangement, you don't need volatile fields or explicit fences, as all instance fields can be final and JMM guarantees safe publication in this case. Regards, Peter On 02/03/2016 12:07 PM, Peter Levart wrote: > Hi Sherman, > > I don't currently have any idea how to squeeze the hashtable any more > further. It is already very compact in my opinion. But I noticed a > data race that could result in navigating the half-initialized data > structure. It is a classical unsafe publication bug. It has been > present before in get(int cp) and it is present now in both > getName(int cp) and getCodePoint(String name). For example: > > 151 static int getCodePoint(String name) { > 152 byte[] strPool = null; > 153 if (refStrPool == null || (strPool = refStrPool.get()) == > null) { > 154 strPool = initNamePool(); > 155 } > > vs. > > 111 refStrPool = new SoftReference<>(strPool); > > ...the static refStrPool field is not marked volatile. > > One way to fix this is to mark field volatile and then rearrange code > in getName/getCodePoint to only read from it once by introducing a > local var. The other would be to change the line 111 into something like: > > SoftReference rsp = new SoftReference<>(strPool); > unsafe.storeFence(); > refStrPool = rsp; > > ...*and* also rearrange code in getName/getCodePoint to only read from > field once by introducing a local var. > > > Regards, Peter > > On 02/02/2016 10:25 PM, Xueming Shen wrote: >> Hi, >> >> Have not heard any feedback on this one so far. I'm adding >> a little more to make it attractive for reviewers :-) >> >> On top of the \N now the webrev includes the proposal to add >> two more matchers, \X for unicode extended grapheme cluster >> and \b{g} for the corresponding boundary. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-7071819 >> Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 >> webrev: http://cr.openjdk.java.net/~sherman/8147531_7071819/webrev/ >> >> Thanks! >> Sherman >> >> On 01/18/2016 11:52 PM, Xueming Shen wrote: >>> Hi, >>> >>> Please help review the change to add \N support in regex. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 >>> webrev: http://cr.openjdk.java.net/~sherman/8147531/webrev >>> >>> This is one of the items we were planning to address via JEP111 >>> http://openjdk.java.net/jeps/111 >>> https://bugs.openjdk.java.net/browse/JDK-8046101 >>> >>> Some of the constructs had been added already in early release. I'm >>> planning to address the rest as individual rfe separately. >>> >>> Thanks, >>> Sherman >> > From bitterfoxc at gmail.com Wed Feb 3 12:11:11 2016 From: bitterfoxc at gmail.com (ShinyaYoshida) Date: Wed, 3 Feb 2016 21:11:11 +0900 Subject: RFR:JDK-8146747:java.time.Duration.toNanos() and toMillis() exception on negative durations In-Reply-To: <56AEF8B8.4070703@oracle.com> References: <56AEF8B8.4070703@oracle.com> Message-ID: Hi Nadeesh, Almost LGTM!(But I'm not a reviewer;) ) However I've noticed that you don't use NANOS_PER_SECOND at L1223 and L1246. Is there some reason not to use it? Regards, shinyafox(Shinya Yoshida) 2016-02-01 15:18 GMT+09:00 nadeesh tv : > Hi all, > > Please review following > > Bug Id : https://bugs.openjdk.java.net/browse/JDK-8146747 < > https://bugs.openjdk.java.net/browse/JDK-8146747> > > Solution: Handled the negative duration separately > > webrev : http://cr.openjdk.java.net/~ntv/8146747/webrev.00/ < > http://cr.openjdk.java.net/%7Entv/8146747/webrev.00/> > > -- > Thanks and Regards, > Nadeesh TV > > From nadeesh.tv at oracle.com Wed Feb 3 12:18:16 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Wed, 03 Feb 2016 17:48:16 +0530 Subject: RFR:JDK-8146747:java.time.Duration.toNanos() and toMillis() exception on negative durations In-Reply-To: References: <56AEF8B8.4070703@oracle.com> Message-ID: <56B1F008.5030800@oracle.com> Hi Shinya, Thnx. I will update it. Regards, Nadeesh On 2/3/2016 5:41 PM, ShinyaYoshida wrote: > Hi Nadeesh, > Almost LGTM!(But I'm not a reviewer;) ) > However I've noticed that you don't use NANOS_PER_SECOND at L1223 and > L1246. > Is there some reason not to use it? > > Regards, > shinyafox(Shinya Yoshida) > > 2016-02-01 15:18 GMT+09:00 nadeesh tv >: > > Hi all, > > Please review following > > Bug Id : https://bugs.openjdk.java.net/browse/JDK-8146747 > > > Solution: Handled the negative duration separately > > webrev : http://cr.openjdk.java.net/~ntv/8146747/webrev.00/ > > > > -- > Thanks and Regards, > Nadeesh TV > > -- Thanks and Regards, Nadeesh TV From amaembo at gmail.com Wed Feb 3 12:19:45 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Wed, 3 Feb 2016 18:19:45 +0600 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> Message-ID: <179345382.20160203181946@gmail.com> Hello! Here's updated webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r2/ PS> My inclination is to keep this simple and not support splitting after partial traversal. PS> Sometimes splitting after partial traversal might be more complex PS> not support (e.g. ArrayList). In other cases it?s more complex to PS> support and in such cases i would argue it is not worth it since PS> this kind of functionality is an edge case. I would still like that this problem is fixed (i.e. support splitting after advance, not just return null). This is probably an edge case for JDK, but might be crucial for library writers. As a library writer I actually had bad times working around this issue. While returning null instead of incorrect splitting would indeed be helpful, it will unnecessarily remove parallelization capabilities in some cases. For example, sometimes it's desired to extract the first element from the stream and create the stream from the tail. Returning null here would mean that the resulting stream will never split after that. To my opinion this fix is not very complex. It just adds several lines into single method (trySplit()). I added a comment which clarifies the intention. Other changes like extraction of initPusher() do not add complexity. Actually they reduce the complexity as four identical lambdas are merged into one. This patch actually reduces the size of created class files. With javac 9-ea+99 StreamSpliterators.java is compiled into 79004 bytes after my patch and 79472 bytes before my patch. Also this patch does not require primitive specializations and adds no overhead for common case when traversal is not started. PS> Testing wise you are right to be concerned about the increase in PS> test execution time. The lack of flatMap is definitely an PS> omission. In this case I think it is ok to replace map with PS> flatMap, as both result in stuff going into the holding buffer, PS> and we can use the smaller data sets, e.g. PS> "StreamTestData.small?, that were recently added. I replaced all the datasets with .small alternatives (I think it's ok here as WrappingSpliterator behavior does not differ much for various sources) and removed all .map tests. Now the test works faster (like 25 sec -> 17 sec on by box). Please check. With best regards, Tagir Valeev. PS. PS> Paul. >> On 2 Feb 2016, at 09:24, Tagir F. Valeev wrote: >> >> Please review and sponsor this fix: >> >> https://bugs.openjdk.java.net/browse/JDK-8148838 >> http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r1/ >> >> When buffer traversal is already started, and split is requested, then >> the existing buffer should be carefully transferred to the prefix >> part. >> >> The only problem I see here is the testing time. Due to >> permuteFunctions() call it increases significantly what I added the >> flatMap() test. If this becomes an issue, I can write new simple test >> which tests flatMap() only (without permutations with other >> intermediate operations). >> >> With best regards, >> Tagir Valeev. >> From amaembo at gmail.com Wed Feb 3 12:20:24 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Wed, 3 Feb 2016 18:20:24 +0600 Subject: RFR-8148748: ArrayList.subList().spliterator() is not late-binding In-Reply-To: <564410418.20160202102813@gmail.com> References: <27476765.20160129103224@gmail.com> <564410418.20160202102813@gmail.com> Message-ID: <1084445298.20160203182024@gmail.com> Hello! Here's the webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8148748/r1/ I noticed that list == null checks are never true, thus redundant, so I removed them. The list field is final and always assigned to the outer ArrayList instance. So it seems quite natural to make ArrayListSpliterator non-static instead. Also I added one more field into ArrayListSpliterator to hold the SubList reference. Hopefully such footprint increase is acceptable. The offset field is initialized eagerly as it does not change during the SubList lifetime. Also I replaced "if (action == null)" checks with "Objects.requireNonNull". Some more thoughts about forEachRemaining: To me it seems that if ((a = lst.elementData) != null) is a redundant check as well as in current ArrayList implementation elementData is never null. So it can be replaced with simple assignment. Also this check: (i = index) >= 0 I don't see the case when i could be negative. Should it be removed as well? Or probably it's added to aid JIT to elimitate bounds-checking? With best regards, Tagir Valeev. PS> Hi Tagir, PS> Last-binding while adding some complexity to the implementation PS> is an important property, as it is to fail-fast on a best effort basis. PS> When one has mutable collections that can be structurally PS> modified and lazy views over those collections (Streams/Iterators) PS> then those views really need to act on the most up to date PS> structure and not a snapshot (structural or otherwise) when the PS> view was created. Otherwise, it is a potential source of bugs PS> producing incorrect results or exceptions, or another form of PS> complexity for the implementation to maintain integrity under such conditions. PS> We took a short-cut here to reuse the existing top-level PS> spliterator in a non-late-binding manner by specifying the PS> absolute fence. We should fix it. PS> Glancing at the code i believe there is a simple fix. When -1 is PS> passed as the fence to the ArrayList.ArrayListSpliterator then the PS> fence is created lazily and should be the result of offset + PS> list.size e.g. in the constructor: PS> hi = fence = offset + lst.size; PS> The forEach impl also needs updating. PS> ? PS> Sublists are also tricky beasts. See documentation on List.subList: PS> * The semantics of the list returned by this method become undefined if PS> * the backing list (i.e., this list) is structurally modified in PS> * any way other than via the returned list. (Structural modifications are PS> * those that change the size of this list, or otherwise perturb it in such PS> * a fashion that iterations in progress may yield incorrect results.) PS> Paul. >> On 29 Jan 2016, at 05:32, Tagir F. Valeev wrote: >> >> Hello! >> >> When reviewing 8079136 I noticed that >> ArrayList.subList().spliterator() is not late-binding: >> >> List list = new ArrayList<>(Arrays.asList(1,2,3,4)).subList(0, 3); >> Stream s = list.stream(); >> list.add(5); >> s.findFirst(); >> --> Exception in thread "main" java.util.ConcurrentModificationException >> >> This works correctly for other list implementation (for example, >> replacing ArrayList with LinkedList makes this code working). As >> Collection.spliterator() spec says: >> >>> In order to preserve expected laziness behavior for the stream() and >>> parallelStream() methods, spliterators should either have the >>> characteristic of IMMUTABLE or CONCURRENT, or be late-binding. If >>> none of these is practical, the overriding class should describe the >>> spliterator's documented policy of binding and structural >>> interference, and should override the stream() and parallelStream() >>> methods to create streams using a Supplier of the spliterator >> >> None of these is true for ArrayList.subList().spliterator(): it's not >> IMMUTABLE, not CONCURRENT, not late-binding, the binding policy is not >> documented and stream()/parallelStream() methods are not overridden. >> >> Is this an issue? >> >> Actually to my opinion late-binding spliterator behavior is a mistake. >> Nobody cares about it as nobody modifies the source between stream >> creation and terminal operation call. On the other hand, this >> requirement adds implementation complexity and I already found the >> second late-binding problem in existing code (the first one was with >> Pattern.splitAsStream). Also it's not documented that Stream.concat >> may bind the late-binding source, but nobody cares anyways. Life would >> be easier without late-binding spliterators. >> >> With best regards, >> Tagir Valeev. >> ===8<===========End of original message text=========== -- Best regards, Tagir mailto:amaembo at gmail.com From paul.sandoz at oracle.com Wed Feb 3 12:59:24 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Feb 2016 12:59:24 +0000 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <179345382.20160203181946@gmail.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> Message-ID: <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> Hi Tagir, Test updates look good, thanks, that should reduce the test times on some of our test machines. I still disagree and pushing back on the support for splitting after partial traversal. It?s not a pattern i think we should encourage and propagate so such behaviour can be generally relied upon, and predominantly for edge cases. That?s where the complexity string is being pulled on. While your fix in isolation is not terribly complex, it is more complex than the alternative (which was actually the intent of the current impl, we just forget to include the check). Paul. > On 3 Feb 2016, at 12:19, Tagir F. Valeev wrote: > > Hello! > > Here's updated webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r2/ > > PS> My inclination is to keep this simple and not support splitting after partial traversal. > > PS> Sometimes splitting after partial traversal might be more complex > PS> not support (e.g. ArrayList). In other cases it?s more complex to > PS> support and in such cases i would argue it is not worth it since > PS> this kind of functionality is an edge case. > > I would still like that this problem is fixed (i.e. support splitting > after advance, not just return null). This is probably an edge case > for JDK, but might be crucial for library writers. As a library writer > I actually had bad times working around this issue. While returning > null instead of incorrect splitting would indeed be helpful, it will > unnecessarily remove parallelization capabilities in some cases. For > example, sometimes it's desired to extract the first element from the > stream and create the stream from the tail. Returning null here would > mean that the resulting stream will never split after that. > > To my opinion this fix is not very complex. It just adds several lines > into single method (trySplit()). I added a comment which clarifies the > intention. Other changes like extraction of initPusher() do not add > complexity. Actually they reduce the complexity as four identical > lambdas are merged into one. This patch actually reduces the size of > created class files. With javac 9-ea+99 StreamSpliterators.java is > compiled into 79004 bytes after my patch and 79472 bytes before my > patch. Also this patch does not require primitive specializations and > adds no overhead for common case when traversal is not started. > > PS> Testing wise you are right to be concerned about the increase in > PS> test execution time. The lack of flatMap is definitely an > PS> omission. In this case I think it is ok to replace map with > PS> flatMap, as both result in stuff going into the holding buffer, > PS> and we can use the smaller data sets, e.g. > PS> "StreamTestData.small?, that were recently added. > > I replaced all the datasets with .small alternatives (I think it's ok > here as WrappingSpliterator behavior does not differ much for various > sources) and removed all .map tests. Now the test works faster > (like 25 sec -> 17 sec on by box). Please check. > > With best regards, > Tagir Valeev. > > PS. > > PS> Paul. > >>> On 2 Feb 2016, at 09:24, Tagir F. Valeev wrote: >>> >>> Please review and sponsor this fix: >>> >>> https://bugs.openjdk.java.net/browse/JDK-8148838 >>> http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r1/ >>> >>> When buffer traversal is already started, and split is requested, then >>> the existing buffer should be carefully transferred to the prefix >>> part. >>> >>> The only problem I see here is the testing time. Due to >>> permuteFunctions() call it increases significantly what I added the >>> flatMap() test. If this becomes an issue, I can write new simple test >>> which tests flatMap() only (without permutations with other >>> intermediate operations). >>> >>> With best regards, >>> Tagir Valeev. >>> > From harold.seigel at oracle.com Wed Feb 3 13:16:41 2016 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 Feb 2016 08:16:41 -0500 Subject: RFR(XS) 8148785: Update class file version to 53 for JDK-9 Message-ID: <56B1FDB9.20009@oracle.com> Hi, Please review this small change to fix bug 8148785. The fix bumps the class file version to 53 for JDK-9. Open webrevs: http://cr.openjdk.java.net/~hseigel/bug_8148785.jdk/ http://cr.openjdk.java.net/~hseigel/bug_8148785.hs/ JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8148785 The fix was tested with JCK Lang, VM, and API tests, UTE quick and split verifier tests, and the hotspot, JDK vm, java/io, java/lang, and java/util JTreg tests, and the new test. Thanks, Harold From aleksey.shipilev at oracle.com Wed Feb 3 13:36:07 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 3 Feb 2016 16:36:07 +0300 Subject: RFR(XS) 8148785: Update class file version to 53 for JDK-9 In-Reply-To: <56B1FDB9.20009@oracle.com> References: <56B1FDB9.20009@oracle.com> Message-ID: <56B20247.9060504@oracle.com> On 02/03/2016 04:16 PM, harold seigel wrote: > Open webrevs: > http://cr.openjdk.java.net/~hseigel/bug_8148785.jdk/ > http://cr.openjdk.java.net/~hseigel/bug_8148785.hs/ +1 > JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8148785 Description seems to imply we change the compiled class file versions to 53.0. But this change only makes VM to accept 53.0, it does not actually produce 53.0 files yet. Are we sure the patches do what 8148785 intended? Anyhow, there seems to be little risk with this particular change. -Aleksey From erik.joelsson at oracle.com Wed Feb 3 13:50:27 2016 From: erik.joelsson at oracle.com (Erik Joelsson) Date: Wed, 3 Feb 2016 14:50:27 +0100 Subject: RFR: JDK-8148955: libjimage.so compiled with wrong flags Message-ID: <56B205A3.1070205@oracle.com> The library libjimage.so consists of C++ source files, but is currently setup to compile using CFLAGS_JDKLIB instead of CXXFLAGS_JDKLIB. On Solaris, these variables contain quite a few differences due to the compilers taking very different arguments. This makes the build process very noisy in the build log. When correcting the flags, the proper warnings were also enabled. I took the liberty of fixing the warning complaining about the variable "i" hiding a variable in the outer scope. Bug: https://bugs.openjdk.java.net/browse/JDK-8148955 Webrev: http://cr.openjdk.java.net/~erikj/8148955/webrev.jdk.01/ /Erik From Roger.Riggs at oracle.com Wed Feb 3 14:10:06 2016 From: Roger.Riggs at oracle.com (Roger Riggs) Date: Wed, 3 Feb 2016 09:10:06 -0500 Subject: RFR: JDK-8148955: libjimage.so compiled with wrong flags In-Reply-To: <56B205A3.1070205@oracle.com> References: <56B205A3.1070205@oracle.com> Message-ID: <56B20A3E.7050301@oracle.com> +1 (at least for the jdk code) Thanks Erik On 02/03/2016 08:50 AM, Erik Joelsson wrote: > The library libjimage.so consists of C++ source files, but is > currently setup to compile using CFLAGS_JDKLIB instead of > CXXFLAGS_JDKLIB. On Solaris, these variables contain quite a few > differences due to the compilers taking very different arguments. This > makes the build process very noisy in the build log. > > When correcting the flags, the proper warnings were also enabled. I > took the liberty of fixing the warning complaining about the variable > "i" hiding a variable in the outer scope. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8148955 > Webrev: http://cr.openjdk.java.net/~erikj/8148955/webrev.jdk.01/ > > /Erik From harold.seigel at oracle.com Wed Feb 3 14:12:46 2016 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 Feb 2016 09:12:46 -0500 Subject: RFR(XS) 8148785: Update class file version to 53 for JDK-9 In-Reply-To: <56B20247.9060504@oracle.com> References: <56B1FDB9.20009@oracle.com> <56B20247.9060504@oracle.com> Message-ID: <56B20ADE.3040509@oracle.com> Hi Aleksey, Thanks for the review. The plan is to first change the runtime to accept version 53 files and then change the tools to generate them. Hopefully, this will reduce incompatibility problems. See JDK-8148651 for details. Thanks, Harold On 2/3/2016 8:36 AM, Aleksey Shipilev wrote: > On 02/03/2016 04:16 PM, harold seigel wrote: >> Open webrevs: >> http://cr.openjdk.java.net/~hseigel/bug_8148785.jdk/ >> http://cr.openjdk.java.net/~hseigel/bug_8148785.hs/ > +1 > >> JBS Bug: https://bugs.openjdk.java.net/browse/JDK-8148785 > Description seems to imply we change the compiled class file versions to > 53.0. But this change only makes VM to accept 53.0, it does not actually > produce 53.0 files yet. Are we sure the patches do what 8148785 intended? > > Anyhow, there seems to be little risk with this particular change. > > -Aleksey > > From paul.sandoz at oracle.com Wed Feb 3 14:29:02 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 3 Feb 2016 14:29:02 +0000 Subject: Stream.foldLeft, one more time (8133680) In-Reply-To: <1957867051.20160129203759@gmail.com> References: <1957867051.20160129203759@gmail.com> Message-ID: Hi Tagir, I think this is reasonable, no objections. Naming-wise reduceLeft might fit better. Those default methods look good. We might be able to do a little better tweaking stuff in ForEachOps to avoid the array box, for some increase in code i.e. a clone of ForEachOps.ForEachOp. Testing-wise see that for take/dropWhile for testing default methods. Paul. > On 29 Jan 2016, at 14:37, Tagir F. Valeev wrote: > > Hello! > > What do you think, is it still reasonable to add foldLeft operation to > JDK-9 Stream API (foldLeft only, no foldRight)? > > I propose the following signatures: > > interface Stream { > public U foldLeft(U seed, BiFunction accumulator); > public Optional foldLeft(BinaryOperator accumulator); > } > > interface IntStream { > public int foldLeft(int seed, IntBinaryOperator accumulator); > public OptionalInt foldLeft(IntBinaryOperator accumulator); > } > > interface LongStream { > public OptionalLong foldLeft(LongBinaryOperator accumulator); > public long foldLeft(long seed, LongBinaryOperator accumulator); > } > > interface DoubleStream { > public OptionalDouble foldLeft(DoubleBinaryOperator accumulator); > public double foldLeft(double seed, DoubleBinaryOperator accumulator); > } > > They are similar to the corresponding reduce operations, but it's not > required for accumulator to be associative and it's not required for > seed to have identity properties. It should be clearly stated in > JavaDoc that for associative accumulator reduce should be used > instead. > > All methods can be easily implemented as default methods in the > interfaces. For example, DoubleStream::foldLeft: > > public double foldLeft(double seed, DoubleBinaryOperator accumulator) { > double[] box = new double[] { seed }; > forEachOrdered(t -> box[0] = accumulator.applyAsDouble(box[0], t)); > return box[0]; > } > > I can take this issue (if nobody works on it already), but in order > not to do the useless work I want to be sure that there are no strong > objections against such feature. > > With best regards, > Tagir Valeev. > From Alan.Bateman at oracle.com Wed Feb 3 14:32:55 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Feb 2016 14:32:55 +0000 Subject: RFR(XS) 8148785: Update class file version to 53 for JDK-9 In-Reply-To: <56B20ADE.3040509@oracle.com> References: <56B1FDB9.20009@oracle.com> <56B20247.9060504@oracle.com> <56B20ADE.3040509@oracle.com> Message-ID: <56B20F97.8020300@oracle.com> On 03/02/2016 14:12, harold seigel wrote: > Hi Aleksey, > > Thanks for the review. The plan is to first change the runtime to > accept version 53 files and then change the tools to generate them. > Hopefully, this will reduce incompatibility problems. See JDK-8148651 > for details. Right, there are several things that need to happen and I think we need hotspot accepting 53.0 before javac starts to generate them. The webrevs look okay to me, except it might be better to drop "New Module attribute" from the comment in classfileParser.cpp until JSR 376 is further along and we line up the trucks to bring the module system into JDK 9. -Alan From Alan.Bateman at oracle.com Wed Feb 3 14:34:17 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Feb 2016 14:34:17 +0000 Subject: RFR: JDK-8148955: libjimage.so compiled with wrong flags In-Reply-To: <56B205A3.1070205@oracle.com> References: <56B205A3.1070205@oracle.com> Message-ID: <56B20FE9.8000301@oracle.com> On 03/02/2016 13:50, Erik Joelsson wrote: > The library libjimage.so consists of C++ source files, but is > currently setup to compile using CFLAGS_JDKLIB instead of > CXXFLAGS_JDKLIB. On Solaris, these variables contain quite a few > differences due to the compilers taking very different arguments. This > makes the build process very noisy in the build log. > > When correcting the flags, the proper warnings were also enabled. I > took the liberty of fixing the warning complaining about the variable > "i" hiding a variable in the outer scope. > > Bug: https://bugs.openjdk.java.net/browse/JDK-8148955 > Webrev: http://cr.openjdk.java.net/~erikj/8148955/webrev.jdk.01/ > This looks okay to me. -Alan. From harold.seigel at oracle.com Wed Feb 3 14:36:47 2016 From: harold.seigel at oracle.com (harold seigel) Date: Wed, 3 Feb 2016 09:36:47 -0500 Subject: RFR(XS) 8148785: Update class file version to 53 for JDK-9 In-Reply-To: <56B20F97.8020300@oracle.com> References: <56B1FDB9.20009@oracle.com> <56B20247.9060504@oracle.com> <56B20ADE.3040509@oracle.com> <56B20F97.8020300@oracle.com> Message-ID: <56B2107F.1010102@oracle.com> Hi Alan, Thanks for looking at the change. I'll drop the comment before checking it in. Harold On 2/3/2016 9:32 AM, Alan Bateman wrote: > > > On 03/02/2016 14:12, harold seigel wrote: >> Hi Aleksey, >> >> Thanks for the review. The plan is to first change the runtime to >> accept version 53 files and then change the tools to generate them. >> Hopefully, this will reduce incompatibility problems. See >> JDK-8148651 for >> details. > Right, there are several things that need to happen and I think we > need hotspot accepting 53.0 before javac starts to generate them. > > The webrevs look okay to me, except it might be better to drop "New > Module attribute" from the comment in classfileParser.cpp until JSR > 376 is further along and we line up the trucks to bring the module > system into JDK 9. > > -Alan From martinrb at google.com Wed Feb 3 15:16:10 2016 From: martinrb at google.com (Martin Buchholz) Date: Wed, 3 Feb 2016 07:16:10 -0800 Subject: RFR-8148748: ArrayList.subList().spliterator() is not late-binding In-Reply-To: <1084445298.20160203182024@gmail.com> References: <27476765.20160129103224@gmail.com> <564410418.20160202102813@gmail.com> <1084445298.20160203182024@gmail.com> Message-ID: On Wed, Feb 3, 2016 at 4:20 AM, Tagir F. Valeev wrote: > Some more thoughts about forEachRemaining: > > To me it seems that > if ((a = lst.elementData) != null) > is a redundant check as well as in current ArrayList implementation > elementData is never null. So it can be replaced with simple > assignment. The null check for something that is provably non-null is a sign of the hand of Doug Lea. I'm often tempted to try to remove them from j.u.concurrent as well, but I resist. They may help hotspot generate better code. From thomas.stuefe at gmail.com Wed Feb 3 15:20:13 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 3 Feb 2016 16:20:13 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B16E8D.6060208@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> Message-ID: On Wed, Feb 3, 2016 at 4:05 AM, David Holmes wrote: > On 3/02/2016 8:08 AM, Stuart Marks wrote: > >> Hi Roger, >> >> It will be good to get this into the JDK. Lots of people have been >> asking for this. >> > > I think this API is a big mistake. The primary usecase seems to be > control-C interception for utilities like jshell. Adding a general purpose > signal raising and handling mechanism to the JDK does not seem like a good > solution to me. While you would need to use signal management under the > covers I think it would be much cleaner to expose an API that actually > captures what it is you want here: a mechanism to manage "interrupt" and > "terminate" events at the VM level, in a clean cross-platform way. > > I agree with David. One problem I see is that it is difficult to write portable java applications with this API. Not only are WIndows and Posix are very different, but also there are also sublte differences between Posix platforms. For instance, in the jbs SIGTRAP was mentioned as a possible signal someone wanted to raise, but SIGTRAP is used by the JIT in the powerpc port. So applications using Signal.of("SIGTRAP") would run fine on x86, but cause a crash on powerpc. Kind Regards, Thomas Aside: If you want to see some prior art in this area look at > PosixSignalHandler API in the Real-Time Specification for Java. > > Which reminds me - do you propose to support the POSIX real-time signals? > > David > ----- > > > I have a few comments on the API. >> >> 1) Is there a way to query the set of signals supported? This might be a >> Set returned by a static method, for example. I agree that >> signal strings outside this set shouldn't be supported. >> >> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >> explicitly. Are these required to be implemented on all platforms, or >> just on "unix-like" platforms, are they just examples? What signals are >> available on Windows? >> >> 3) raise() is spec'd to throw an exception if there's no handler >> registered. But wouldn't it make sense to allow it if the default >> handler is registered? >> >> 4) In an earlier message you said that the Signal object is a >> capability, so the security check is on getting a reference. It seems to >> me that setting a handler is in a different category from raising a >> signal; this suggests to me that using the same object as a capability >> for both should be rethought. >> >> 5) I don't understand the asymmetry between register() and unregister(). >> Your earlier exchanges with Chris and with Gerard touched on this, >> specifically, the requirement that the caller pass unregister() a >> reference to the old handler in order for unregistration to work. You >> had said this was safer, if there are uncoordinated pieces of code >> attempting to set/unset signal handlers. >> >> It looks to me like this API is really about maintaining process global >> state consisting of a single handler -- user-specified or default -- for >> each supported signal. (I agree that it shouldn't try to have a stack or >> a chain of handlers.) There are a few other things that are global like >> this, such as the security manager and policy, System.setIn/Out/Err, and >> so forth. As such, uncoordinated access to the signal API is pretty much >> broken no matter what. Thus I don't think it makes sense to have a >> CAS-like protocol for unregistering a handler, to protect against the >> case where "somebody else" might have registered a handler different >> from yours. >> >> Something like this might make sense: >> >> void register(Consumer handler); >> void unregister(); >> >> The register() call would be pretty much as currently specified; the >> unregister() call would restore the default handler. Alternatively, >> register(null) could be used instead of unregister(), but this is quite >> minor. >> >> Thanks, >> >> s'marks >> >> >> >> >> >> On 2/1/16 8:02 AM, Roger Riggs wrote: >> >>> Please review an API addition to handle signals such as SIGINT, >>> SIGHUP, and >>> SIGTERM. >>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>> case for >>> interactive applications that need to handle Control-C and other signals. >>> >>> The new java.util.Signal class provides a settable primary signal >>> handler and a >>> default >>> signal handler. The primary signal handler can be unregistered and >>> handling is >>> restored >>> to the default signal handler. System initialization registers >>> default signal >>> handlers >>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>> requires >>> a permission if a SecurityManager is set. >>> >>> The sun.misc.Signal implementation is modified to be layered on a common >>> thread and dispatch mechanism. The VM handling of native signals is >>> not affected. >>> The command option to reduce signal use by the runtime with -Xrs is >>> unmodified. >>> >>> The changes to hotspot are minimal to rename the hardcoded callback to >>> the Java >>> Signal dispatcher. >>> >>> Please review and comment on the API and implementation. >>> >>> javadoc: >>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>> >>> Webrev: >>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>> >>> JEP 260: >>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> Thanks, Roger >>> >>> >>> From chris.hegarty at oracle.com Wed Feb 3 15:22:35 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 3 Feb 2016 15:22:35 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> Message-ID: <56B21B3B.7040008@oracle.com> On 03/02/16 15:20, Thomas St?fe wrote: > On Wed, Feb 3, 2016 at 4:05 AM, David Holmes > wrote: > >> On 3/02/2016 8:08 AM, Stuart Marks wrote: >> >>> Hi Roger, >>> >>> It will be good to get this into the JDK. Lots of people have been >>> asking for this. >>> >> >> I think this API is a big mistake. The primary usecase seems to be >> control-C interception for utilities like jshell. Adding a general purpose >> signal raising and handling mechanism to the JDK does not seem like a good >> solution to me. While you would need to use signal management under the >> covers I think it would be much cleaner to expose an API that actually >> captures what it is you want here: a mechanism to manage "interrupt" and >> "terminate" events at the VM level, in a clean cross-platform way. >> >> > I agree with David. One problem I see is that it is difficult to write > portable java applications with this API. Not only are WIndows and Posix > are very different, but also there are also sublte differences between > Posix platforms. For instance, in the jbs SIGTRAP was mentioned as a > possible signal someone wanted to raise, but SIGTRAP is used by the JIT in > the powerpc port. So applications using Signal.of("SIGTRAP") would run fine > on x86, but cause a crash on powerpc. I accept the sentiment of your mail, but I suspect that Signal.of("SIGTRAP") would throw UOE with this API, and not crash the VM, otherwise it is a bug. -Chris. > Kind Regards, Thomas > > Aside: If you want to see some prior art in this area look at >> PosixSignalHandler API in the Real-Time Specification for Java. >> >> Which reminds me - do you propose to support the POSIX real-time signals? >> >> David >> ----- >> >> >> I have a few comments on the API. >>> >>> 1) Is there a way to query the set of signals supported? This might be a >>> Set returned by a static method, for example. I agree that >>> signal strings outside this set shouldn't be supported. >>> >>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>> explicitly. Are these required to be implemented on all platforms, or >>> just on "unix-like" platforms, are they just examples? What signals are >>> available on Windows? >>> >>> 3) raise() is spec'd to throw an exception if there's no handler >>> registered. But wouldn't it make sense to allow it if the default >>> handler is registered? >>> >>> 4) In an earlier message you said that the Signal object is a >>> capability, so the security check is on getting a reference. It seems to >>> me that setting a handler is in a different category from raising a >>> signal; this suggests to me that using the same object as a capability >>> for both should be rethought. >>> >>> 5) I don't understand the asymmetry between register() and unregister(). >>> Your earlier exchanges with Chris and with Gerard touched on this, >>> specifically, the requirement that the caller pass unregister() a >>> reference to the old handler in order for unregistration to work. You >>> had said this was safer, if there are uncoordinated pieces of code >>> attempting to set/unset signal handlers. >>> >>> It looks to me like this API is really about maintaining process global >>> state consisting of a single handler -- user-specified or default -- for >>> each supported signal. (I agree that it shouldn't try to have a stack or >>> a chain of handlers.) There are a few other things that are global like >>> this, such as the security manager and policy, System.setIn/Out/Err, and >>> so forth. As such, uncoordinated access to the signal API is pretty much >>> broken no matter what. Thus I don't think it makes sense to have a >>> CAS-like protocol for unregistering a handler, to protect against the >>> case where "somebody else" might have registered a handler different >>> from yours. >>> >>> Something like this might make sense: >>> >>> void register(Consumer handler); >>> void unregister(); >>> >>> The register() call would be pretty much as currently specified; the >>> unregister() call would restore the default handler. Alternatively, >>> register(null) could be used instead of unregister(), but this is quite >>> minor. >>> >>> Thanks, >>> >>> s'marks >>> >>> >>> >>> >>> >>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>> >>>> Please review an API addition to handle signals such as SIGINT, >>>> SIGHUP, and >>>> SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>>> case for >>>> interactive applications that need to handle Control-C and other signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal >>>> handler and a >>>> default >>>> signal handler. The primary signal handler can be unregistered and >>>> handling is >>>> restored >>>> to the default signal handler. System initialization registers >>>> default signal >>>> handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>> requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a common >>>> thread and dispatch mechanism. The VM handling of native signals is >>>> not affected. >>>> The command option to reduce signal use by the runtime with -Xrs is >>>> unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback to >>>> the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> Webrev: >>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> JEP 260: >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> Thanks, Roger >>>> >>>> >>>> From thomas.stuefe at gmail.com Wed Feb 3 15:37:07 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 3 Feb 2016 16:37:07 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B21B3B.7040008@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B21B3B.7040008@oracle.com> Message-ID: Hi Chris, On Wed, Feb 3, 2016 at 4:22 PM, Chris Hegarty wrote: > On 03/02/16 15:20, Thomas St?fe wrote: > >> On Wed, Feb 3, 2016 at 4:05 AM, David Holmes >> wrote: >> >> On 3/02/2016 8:08 AM, Stuart Marks wrote: >>> >>> Hi Roger, >>>> >>>> It will be good to get this into the JDK. Lots of people have been >>>> asking for this. >>>> >>>> >>> I think this API is a big mistake. The primary usecase seems to be >>> control-C interception for utilities like jshell. Adding a general >>> purpose >>> signal raising and handling mechanism to the JDK does not seem like a >>> good >>> solution to me. While you would need to use signal management under the >>> covers I think it would be much cleaner to expose an API that actually >>> captures what it is you want here: a mechanism to manage "interrupt" and >>> "terminate" events at the VM level, in a clean cross-platform way. >>> >>> >>> I agree with David. One problem I see is that it is difficult to write >> portable java applications with this API. Not only are WIndows and Posix >> are very different, but also there are also sublte differences between >> Posix platforms. For instance, in the jbs SIGTRAP was mentioned as a >> possible signal someone wanted to raise, but SIGTRAP is used by the JIT in >> the powerpc port. So applications using Signal.of("SIGTRAP") would run >> fine >> on x86, but cause a crash on powerpc. >> > > I accept the sentiment of your mail, but I suspect that > Signal.of("SIGTRAP") would throw UOE with this API, and not crash > the VM, otherwise it is a bug. > > Of course, you are right. From a user standpoint the result is the same, though. ..Thomas > -Chris. > > > Kind Regards, Thomas >> >> Aside: If you want to see some prior art in this area look at >> >>> PosixSignalHandler API in the Real-Time Specification for Java. >>> >>> Which reminds me - do you propose to support the POSIX real-time signals? >>> >>> David >>> ----- >>> >>> >>> I have a few comments on the API. >>> >>>> >>>> 1) Is there a way to query the set of signals supported? This might be a >>>> Set returned by a static method, for example. I agree that >>>> signal strings outside this set shouldn't be supported. >>>> >>>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>>> explicitly. Are these required to be implemented on all platforms, or >>>> just on "unix-like" platforms, are they just examples? What signals are >>>> available on Windows? >>>> >>>> 3) raise() is spec'd to throw an exception if there's no handler >>>> registered. But wouldn't it make sense to allow it if the default >>>> handler is registered? >>>> >>>> 4) In an earlier message you said that the Signal object is a >>>> capability, so the security check is on getting a reference. It seems to >>>> me that setting a handler is in a different category from raising a >>>> signal; this suggests to me that using the same object as a capability >>>> for both should be rethought. >>>> >>>> 5) I don't understand the asymmetry between register() and unregister(). >>>> Your earlier exchanges with Chris and with Gerard touched on this, >>>> specifically, the requirement that the caller pass unregister() a >>>> reference to the old handler in order for unregistration to work. You >>>> had said this was safer, if there are uncoordinated pieces of code >>>> attempting to set/unset signal handlers. >>>> >>>> It looks to me like this API is really about maintaining process global >>>> state consisting of a single handler -- user-specified or default -- for >>>> each supported signal. (I agree that it shouldn't try to have a stack or >>>> a chain of handlers.) There are a few other things that are global like >>>> this, such as the security manager and policy, System.setIn/Out/Err, and >>>> so forth. As such, uncoordinated access to the signal API is pretty much >>>> broken no matter what. Thus I don't think it makes sense to have a >>>> CAS-like protocol for unregistering a handler, to protect against the >>>> case where "somebody else" might have registered a handler different >>>> from yours. >>>> >>>> Something like this might make sense: >>>> >>>> void register(Consumer handler); >>>> void unregister(); >>>> >>>> The register() call would be pretty much as currently specified; the >>>> unregister() call would restore the default handler. Alternatively, >>>> register(null) could be used instead of unregister(), but this is quite >>>> minor. >>>> >>>> Thanks, >>>> >>>> s'marks >>>> >>>> >>>> >>>> >>>> >>>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>>> >>>> Please review an API addition to handle signals such as SIGINT, >>>>> SIGHUP, and >>>>> SIGTERM. >>>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>>>> case for >>>>> interactive applications that need to handle Control-C and other >>>>> signals. >>>>> >>>>> The new java.util.Signal class provides a settable primary signal >>>>> handler and a >>>>> default >>>>> signal handler. The primary signal handler can be unregistered and >>>>> handling is >>>>> restored >>>>> to the default signal handler. System initialization registers >>>>> default signal >>>>> handlers >>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>>> requires >>>>> a permission if a SecurityManager is set. >>>>> >>>>> The sun.misc.Signal implementation is modified to be layered on a >>>>> common >>>>> thread and dispatch mechanism. The VM handling of native signals is >>>>> not affected. >>>>> The command option to reduce signal use by the runtime with -Xrs is >>>>> unmodified. >>>>> >>>>> The changes to hotspot are minimal to rename the hardcoded callback to >>>>> the Java >>>>> Signal dispatcher. >>>>> >>>>> Please review and comment on the API and implementation. >>>>> >>>>> javadoc: >>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>> >>>>> Webrev: >>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>> >>>>> JEP 260: >>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> >>>>> From thomas.stuefe at gmail.com Wed Feb 3 15:43:50 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Wed, 3 Feb 2016 16:43:50 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> Message-ID: On Wed, Feb 3, 2016 at 4:20 PM, Thomas St?fe wrote: > > > On Wed, Feb 3, 2016 at 4:05 AM, David Holmes > wrote: > >> On 3/02/2016 8:08 AM, Stuart Marks wrote: >> >>> Hi Roger, >>> >>> It will be good to get this into the JDK. Lots of people have been >>> asking for this. >>> >> >> I think this API is a big mistake. The primary usecase seems to be >> control-C interception for utilities like jshell. Adding a general purpose >> signal raising and handling mechanism to the JDK does not seem like a good >> solution to me. While you would need to use signal management under the >> covers I think it would be much cleaner to expose an API that actually >> captures what it is you want here: a mechanism to manage "interrupt" and >> "terminate" events at the VM level, in a clean cross-platform way. >> >> > I agree with David. One problem I see is that it is difficult to write > portable java applications with this API. Not only are WIndows and Posix > are very different, but also there are also sublte differences between > Posix platforms. For instance, in the jbs SIGTRAP was mentioned as a > possible signal someone wanted to raise, but SIGTRAP is used by the JIT in > the powerpc port. So applications using Signal.of("SIGTRAP") would run fine > on x86, but cause a crash on powerpc. > > Kind Regards, Thomas > > Just occurred to me that a second, subtle problem may be that once java developers start using signals for their own application, we are barred from using the same signals in the future for our own purposes, even if we do not use them now. At least not without breaking those java applications. ..Thomas > Aside: If you want to see some prior art in this area look at >> PosixSignalHandler API in the Real-Time Specification for Java. >> >> Which reminds me - do you propose to support the POSIX real-time signals? >> >> David >> ----- >> >> >> I have a few comments on the API. >>> >>> 1) Is there a way to query the set of signals supported? This might be a >>> Set returned by a static method, for example. I agree that >>> signal strings outside this set shouldn't be supported. >>> >>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>> explicitly. Are these required to be implemented on all platforms, or >>> just on "unix-like" platforms, are they just examples? What signals are >>> available on Windows? >>> >>> 3) raise() is spec'd to throw an exception if there's no handler >>> registered. But wouldn't it make sense to allow it if the default >>> handler is registered? >>> >>> 4) In an earlier message you said that the Signal object is a >>> capability, so the security check is on getting a reference. It seems to >>> me that setting a handler is in a different category from raising a >>> signal; this suggests to me that using the same object as a capability >>> for both should be rethought. >>> >>> 5) I don't understand the asymmetry between register() and unregister(). >>> Your earlier exchanges with Chris and with Gerard touched on this, >>> specifically, the requirement that the caller pass unregister() a >>> reference to the old handler in order for unregistration to work. You >>> had said this was safer, if there are uncoordinated pieces of code >>> attempting to set/unset signal handlers. >>> >>> It looks to me like this API is really about maintaining process global >>> state consisting of a single handler -- user-specified or default -- for >>> each supported signal. (I agree that it shouldn't try to have a stack or >>> a chain of handlers.) There are a few other things that are global like >>> this, such as the security manager and policy, System.setIn/Out/Err, and >>> so forth. As such, uncoordinated access to the signal API is pretty much >>> broken no matter what. Thus I don't think it makes sense to have a >>> CAS-like protocol for unregistering a handler, to protect against the >>> case where "somebody else" might have registered a handler different >>> from yours. >>> >>> Something like this might make sense: >>> >>> void register(Consumer handler); >>> void unregister(); >>> >>> The register() call would be pretty much as currently specified; the >>> unregister() call would restore the default handler. Alternatively, >>> register(null) could be used instead of unregister(), but this is quite >>> minor. >>> >>> Thanks, >>> >>> s'marks >>> >>> >>> >>> >>> >>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>> >>>> Please review an API addition to handle signals such as SIGINT, >>>> SIGHUP, and >>>> SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>>> case for >>>> interactive applications that need to handle Control-C and other >>>> signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal >>>> handler and a >>>> default >>>> signal handler. The primary signal handler can be unregistered and >>>> handling is >>>> restored >>>> to the default signal handler. System initialization registers >>>> default signal >>>> handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>> requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a common >>>> thread and dispatch mechanism. The VM handling of native signals is >>>> not affected. >>>> The command option to reduce signal use by the runtime with -Xrs is >>>> unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback to >>>> the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> Webrev: >>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> JEP 260: >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> Thanks, Roger >>>> >>>> >>>> > From chris.hegarty at oracle.com Wed Feb 3 16:00:43 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 3 Feb 2016 16:00:43 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> Message-ID: <56B2242B.3030105@oracle.com> On 03/02/16 15:43, Thomas St?fe wrote: > On Wed, Feb 3, 2016 at 4:20 PM, Thomas St?fe > wrote: > >> On Wed, Feb 3, 2016 at 4:05 AM, David Holmes >> wrote: >> >>> On 3/02/2016 8:08 AM, Stuart Marks wrote: >>> >>>> Hi Roger, >>>> >>>> It will be good to get this into the JDK. Lots of people have been >>>> asking for this. >>>> >>> >>> I think this API is a big mistake. The primary usecase seems to be >>> control-C interception for utilities like jshell. Adding a general purpose >>> signal raising and handling mechanism to the JDK does not seem like a good >>> solution to me. While you would need to use signal management under the >>> covers I think it would be much cleaner to expose an API that actually >>> captures what it is you want here: a mechanism to manage "interrupt" and >>> "terminate" events at the VM level, in a clean cross-platform way. >>> >>> >> I agree with David. One problem I see is that it is difficult to write >> portable java applications with this API. Not only are WIndows and Posix >> are very different, but also there are also sublte differences between >> Posix platforms. For instance, in the jbs SIGTRAP was mentioned as a >> possible signal someone wanted to raise, but SIGTRAP is used by the JIT in >> the powerpc port. So applications using Signal.of("SIGTRAP") would run fine >> on x86, but cause a crash on powerpc. >> >> Kind Regards, Thomas >> >> > Just occurred to me that a second, subtle problem may be that once java > developers start using signals for their own application, we are barred > from using the same signals in the future for our own purposes, even if we > do not use them now. At least not without breaking those java applications. This is an excellent point. The API should include appropriate wording to indicate that there is no guarantee that an invocation of Signal.of(...) that succeeds in one release, is guaranteed to succeed in another, future, release. -Chris. > ..Thomas > > > >> Aside: If you want to see some prior art in this area look at >>> PosixSignalHandler API in the Real-Time Specification for Java. >>> >>> Which reminds me - do you propose to support the POSIX real-time signals? >>> >>> David >>> ----- >>> >>> >>> I have a few comments on the API. >>>> >>>> 1) Is there a way to query the set of signals supported? This might be a >>>> Set returned by a static method, for example. I agree that >>>> signal strings outside this set shouldn't be supported. >>>> >>>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>>> explicitly. Are these required to be implemented on all platforms, or >>>> just on "unix-like" platforms, are they just examples? What signals are >>>> available on Windows? >>>> >>>> 3) raise() is spec'd to throw an exception if there's no handler >>>> registered. But wouldn't it make sense to allow it if the default >>>> handler is registered? >>>> >>>> 4) In an earlier message you said that the Signal object is a >>>> capability, so the security check is on getting a reference. It seems to >>>> me that setting a handler is in a different category from raising a >>>> signal; this suggests to me that using the same object as a capability >>>> for both should be rethought. >>>> >>>> 5) I don't understand the asymmetry between register() and unregister(). >>>> Your earlier exchanges with Chris and with Gerard touched on this, >>>> specifically, the requirement that the caller pass unregister() a >>>> reference to the old handler in order for unregistration to work. You >>>> had said this was safer, if there are uncoordinated pieces of code >>>> attempting to set/unset signal handlers. >>>> >>>> It looks to me like this API is really about maintaining process global >>>> state consisting of a single handler -- user-specified or default -- for >>>> each supported signal. (I agree that it shouldn't try to have a stack or >>>> a chain of handlers.) There are a few other things that are global like >>>> this, such as the security manager and policy, System.setIn/Out/Err, and >>>> so forth. As such, uncoordinated access to the signal API is pretty much >>>> broken no matter what. Thus I don't think it makes sense to have a >>>> CAS-like protocol for unregistering a handler, to protect against the >>>> case where "somebody else" might have registered a handler different >>>> from yours. >>>> >>>> Something like this might make sense: >>>> >>>> void register(Consumer handler); >>>> void unregister(); >>>> >>>> The register() call would be pretty much as currently specified; the >>>> unregister() call would restore the default handler. Alternatively, >>>> register(null) could be used instead of unregister(), but this is quite >>>> minor. >>>> >>>> Thanks, >>>> >>>> s'marks >>>> >>>> >>>> >>>> >>>> >>>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>>> >>>>> Please review an API addition to handle signals such as SIGINT, >>>>> SIGHUP, and >>>>> SIGTERM. >>>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>>>> case for >>>>> interactive applications that need to handle Control-C and other >>>>> signals. >>>>> >>>>> The new java.util.Signal class provides a settable primary signal >>>>> handler and a >>>>> default >>>>> signal handler. The primary signal handler can be unregistered and >>>>> handling is >>>>> restored >>>>> to the default signal handler. System initialization registers >>>>> default signal >>>>> handlers >>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>>> requires >>>>> a permission if a SecurityManager is set. >>>>> >>>>> The sun.misc.Signal implementation is modified to be layered on a common >>>>> thread and dispatch mechanism. The VM handling of native signals is >>>>> not affected. >>>>> The command option to reduce signal use by the runtime with -Xrs is >>>>> unmodified. >>>>> >>>>> The changes to hotspot are minimal to rename the hardcoded callback to >>>>> the Java >>>>> Signal dispatcher. >>>>> >>>>> Please review and comment on the API and implementation. >>>>> >>>>> javadoc: >>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>> >>>>> Webrev: >>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>> >>>>> JEP 260: >>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> >> From Roger.Riggs at Oracle.com Wed Feb 3 16:07:33 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 3 Feb 2016 11:07:33 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B2242B.3030105@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2242B.3030105@oracle.com> Message-ID: <56B225C5.60907@Oracle.com> Hi, The current wording is explicit about signal support being implementation and os dependent, it can say its release dependent too. Roger On 2/3/2016 11:00 AM, Chris Hegarty wrote: > On 03/02/16 15:43, Thomas St?fe wrote: >> On Wed, Feb 3, 2016 at 4:20 PM, Thomas St?fe >> wrote: >> >>> On Wed, Feb 3, 2016 at 4:05 AM, David Holmes >>> wrote: >>> >>>> On 3/02/2016 8:08 AM, Stuart Marks wrote: >>>> >>>>> Hi Roger, >>>>> >>>>> It will be good to get this into the JDK. Lots of people have been >>>>> asking for this. >>>>> >>>> >>>> I think this API is a big mistake. The primary usecase seems to be >>>> control-C interception for utilities like jshell. Adding a general >>>> purpose >>>> signal raising and handling mechanism to the JDK does not seem like >>>> a good >>>> solution to me. While you would need to use signal management under >>>> the >>>> covers I think it would be much cleaner to expose an API that actually >>>> captures what it is you want here: a mechanism to manage >>>> "interrupt" and >>>> "terminate" events at the VM level, in a clean cross-platform way. >>>> >>>> >>> I agree with David. One problem I see is that it is difficult to write >>> portable java applications with this API. Not only are WIndows and >>> Posix >>> are very different, but also there are also sublte differences between >>> Posix platforms. For instance, in the jbs SIGTRAP was mentioned as a >>> possible signal someone wanted to raise, but SIGTRAP is used by the >>> JIT in >>> the powerpc port. So applications using Signal.of("SIGTRAP") would >>> run fine >>> on x86, but cause a crash on powerpc. >>> >>> Kind Regards, Thomas >>> >>> >> Just occurred to me that a second, subtle problem may be that once java >> developers start using signals for their own application, we are barred >> from using the same signals in the future for our own purposes, even >> if we >> do not use them now. At least not without breaking those java >> applications. > > This is an excellent point. The API should include appropriate wording > to indicate that there is no guarantee that an invocation of > Signal.of(...) that succeeds in one release, is guaranteed to succeed > in another, future, release. > > -Chris. > >> ..Thomas >> >> >> >>> Aside: If you want to see some prior art in this area look at >>>> PosixSignalHandler API in the Real-Time Specification for Java. >>>> >>>> Which reminds me - do you propose to support the POSIX real-time >>>> signals? >>>> >>>> David >>>> ----- >>>> >>>> >>>> I have a few comments on the API. >>>>> >>>>> 1) Is there a way to query the set of signals supported? This >>>>> might be a >>>>> Set returned by a static method, for example. I agree that >>>>> signal strings outside this set shouldn't be supported. >>>>> >>>>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>>>> explicitly. Are these required to be implemented on all platforms, or >>>>> just on "unix-like" platforms, are they just examples? What >>>>> signals are >>>>> available on Windows? >>>>> >>>>> 3) raise() is spec'd to throw an exception if there's no handler >>>>> registered. But wouldn't it make sense to allow it if the default >>>>> handler is registered? >>>>> >>>>> 4) In an earlier message you said that the Signal object is a >>>>> capability, so the security check is on getting a reference. It >>>>> seems to >>>>> me that setting a handler is in a different category from raising a >>>>> signal; this suggests to me that using the same object as a >>>>> capability >>>>> for both should be rethought. >>>>> >>>>> 5) I don't understand the asymmetry between register() and >>>>> unregister(). >>>>> Your earlier exchanges with Chris and with Gerard touched on this, >>>>> specifically, the requirement that the caller pass unregister() a >>>>> reference to the old handler in order for unregistration to work. You >>>>> had said this was safer, if there are uncoordinated pieces of code >>>>> attempting to set/unset signal handlers. >>>>> >>>>> It looks to me like this API is really about maintaining process >>>>> global >>>>> state consisting of a single handler -- user-specified or default >>>>> -- for >>>>> each supported signal. (I agree that it shouldn't try to have a >>>>> stack or >>>>> a chain of handlers.) There are a few other things that are global >>>>> like >>>>> this, such as the security manager and policy, >>>>> System.setIn/Out/Err, and >>>>> so forth. As such, uncoordinated access to the signal API is >>>>> pretty much >>>>> broken no matter what. Thus I don't think it makes sense to have a >>>>> CAS-like protocol for unregistering a handler, to protect against the >>>>> case where "somebody else" might have registered a handler different >>>>> from yours. >>>>> >>>>> Something like this might make sense: >>>>> >>>>> void register(Consumer handler); >>>>> void unregister(); >>>>> >>>>> The register() call would be pretty much as currently specified; the >>>>> unregister() call would restore the default handler. Alternatively, >>>>> register(null) could be used instead of unregister(), but this is >>>>> quite >>>>> minor. >>>>> >>>>> Thanks, >>>>> >>>>> s'marks >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>>>> >>>>>> Please review an API addition to handle signals such as SIGINT, >>>>>> SIGHUP, and >>>>>> SIGTERM. >>>>>> This JEP 260 motivated alternative to sun.misc.Signal supports >>>>>> the use >>>>>> case for >>>>>> interactive applications that need to handle Control-C and other >>>>>> signals. >>>>>> >>>>>> The new java.util.Signal class provides a settable primary signal >>>>>> handler and a >>>>>> default >>>>>> signal handler. The primary signal handler can be unregistered and >>>>>> handling is >>>>>> restored >>>>>> to the default signal handler. System initialization registers >>>>>> default signal >>>>>> handlers >>>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>>>> requires >>>>>> a permission if a SecurityManager is set. >>>>>> >>>>>> The sun.misc.Signal implementation is modified to be layered on a >>>>>> common >>>>>> thread and dispatch mechanism. The VM handling of native signals is >>>>>> not affected. >>>>>> The command option to reduce signal use by the runtime with -Xrs is >>>>>> unmodified. >>>>>> >>>>>> The changes to hotspot are minimal to rename the hardcoded >>>>>> callback to >>>>>> the Java >>>>>> Signal dispatcher. >>>>>> >>>>>> Please review and comment on the API and implementation. >>>>>> >>>>>> javadoc: >>>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>>> >>>>>> Webrev: >>>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>>> hotspot: >>>>>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>>> >>>>>> Issue: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>>> >>>>>> JEP 260: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>>> >>>>>> >>> From mikael.vidstedt at oracle.com Wed Feb 3 16:13:38 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 3 Feb 2016 08:13:38 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B1CBA7.4050902@redhat.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <56B1CBA7.4050902@redhat.com> Message-ID: <56B22732.6010609@oracle.com> On 2016-02-03 01:43, Andrew Haley wrote: > On 02/02/16 19:25, Mikael Vidstedt wrote: >> Please review this change which introduces a Copy::conjoint_swap and an >> Unsafe.copySwapMemory method to call it from Java, along with the >> necessary changes to have java.nio.Bits call it instead of the Bits.c code. > There doesn't seem to be any way to use a byte-swap instruction > in the swapping code. This will make it unnecessarily slow. To be clear, this isn't trying to provide the absolutely most optimal copy+swap implementation. It's trying to fix the Bits.c unaligned bug and pave the way for further improvements. Further performance improvements here are certainly possible, but at this point I'm happy as long as the performance is on par (or better) with the Bits.c implementation it's replacing. That said, at least gcc seems to recognize the byte swapping pattern and does emit a bswap on linux-x64. I'm not sure about the other platforms though. Cheers, Mikael From aph at redhat.com Wed Feb 3 16:15:44 2016 From: aph at redhat.com (Andrew Haley) Date: Wed, 3 Feb 2016 16:15:44 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B22732.6010609@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <56B1CBA7.4050902@redhat.com> <56B22732.6010609@oracle.com> Message-ID: <56B227B0.6080705@redhat.com> On 02/03/2016 04:13 PM, Mikael Vidstedt wrote: > > On 2016-02-03 01:43, Andrew Haley wrote: >> On 02/02/16 19:25, Mikael Vidstedt wrote: >>> Please review this change which introduces a Copy::conjoint_swap and an >>> Unsafe.copySwapMemory method to call it from Java, along with the >>> necessary changes to have java.nio.Bits call it instead of the Bits.c code. >> There doesn't seem to be any way to use a byte-swap instruction >> in the swapping code. This will make it unnecessarily slow. > > To be clear, this isn't trying to provide the absolutely most optimal > copy+swap implementation. It's trying to fix the Bits.c unaligned bug > and pave the way for further improvements. Further performance > improvements here are certainly possible, but at this point I'm happy as > long as the performance is on par (or better) with the Bits.c > implementation it's replacing. Got it, sure. It's just nice to be able to replace low-level routines with platform ones. > That said, at least gcc seems to recognize the byte swapping pattern and > does emit a bswap on linux-x64. I'm not sure about the other platforms > though. Oh, very nice. Right, I'll check that once your patch does in. Andrew. From xueming.shen at oracle.com Wed Feb 3 17:09:39 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 3 Feb 2016 09:09:39 -0800 Subject: RFR JDK-7071819: To support Extended Grapheme Clusters in Regex In-Reply-To: <56B1E3EC.2060509@gmail.com> References: <569DEB35.7020006@oracle.com> <56B11EC3.3010505@oracle.com> <56B1DF63.2040708@gmail.com> <56B1E3EC.2060509@gmail.com> Message-ID: <56B23453.8020403@oracle.com> Hi Peter, Thanks for the review and suggestion. This appears to be a better approach. I was wondering if I should go this way to cache those lookup tables as well, but ... Webrev has been updated as suggested. Thanks! Sherman On 2/3/16 3:26 AM, Peter Levart wrote: > Hi Again, > > I also have a comment on the implementation of the hash table and it's > GC-ness. You keep the string pool under a SoftReference because it is > the biggest object so it makes most sense to clear it when heap > becomes full. Other arrays are kept strongly reachable, but you > re-generate them nevertheless when string pool is cleared by GC. Would > it make sense to: > > - define all int[] arrays (including strPool) as final instance > variables of CharacterName > - have one static field: > > private static SoftReference refInstance; > > - convert initNamePool() into a private constructor. > > - convert public static getName/getCodePoint into public instance methods > > - introduce public static method: > > public static CharacterName getInstance() { > SoftReference ref = refInstance; > CharacterName instance; > if (ref != null && ((instance = ref.get) != null) { > return instance; > } > instance = new CharacterName(); > refInstance = new SoftReference<>(instance); > return instance; > } > > ...in this arrangement, you don't need volatile fields or explicit > fences, as all instance fields can be final and JMM guarantees safe > publication in this case. > > > Regards, Peter > > On 02/03/2016 12:07 PM, Peter Levart wrote: >> Hi Sherman, >> >> I don't currently have any idea how to squeeze the hashtable any more >> further. It is already very compact in my opinion. But I noticed a >> data race that could result in navigating the half-initialized data >> structure. It is a classical unsafe publication bug. It has been >> present before in get(int cp) and it is present now in both >> getName(int cp) and getCodePoint(String name). For example: >> >> 151 static int getCodePoint(String name) { >> 152 byte[] strPool = null; >> 153 if (refStrPool == null || (strPool = refStrPool.get()) >> == null) { >> 154 strPool = initNamePool(); >> 155 } >> >> vs. >> >> 111 refStrPool = new SoftReference<>(strPool); >> >> ...the static refStrPool field is not marked volatile. >> >> One way to fix this is to mark field volatile and then rearrange code >> in getName/getCodePoint to only read from it once by introducing a >> local var. The other would be to change the line 111 into something like: >> >> SoftReference rsp = new SoftReference<>(strPool); >> unsafe.storeFence(); >> refStrPool = rsp; >> >> ...*and* also rearrange code in getName/getCodePoint to only read >> from field once by introducing a local var. >> >> >> Regards, Peter >> >> On 02/02/2016 10:25 PM, Xueming Shen wrote: >>> Hi, >>> >>> Have not heard any feedback on this one so far. I'm adding >>> a little more to make it attractive for reviewers :-) >>> >>> On top of the \N now the webrev includes the proposal to add >>> two more matchers, \X for unicode extended grapheme cluster >>> and \b{g} for the corresponding boundary. >>> >>> Issue: https://bugs.openjdk.java.net/browse/JDK-7071819 >>> Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 >>> webrev: http://cr.openjdk.java.net/~sherman/8147531_7071819/webrev/ >>> >>> Thanks! >>> Sherman >>> >>> On 01/18/2016 11:52 PM, Xueming Shen wrote: >>>> Hi, >>>> >>>> Please help review the change to add \N support in regex. >>>> >>>> Issue: https://bugs.openjdk.java.net/browse/JDK-8147531 >>>> webrev: http://cr.openjdk.java.net/~sherman/8147531/webrev >>>> >>>> This is one of the items we were planning to address via JEP111 >>>> http://openjdk.java.net/jeps/111 >>>> https://bugs.openjdk.java.net/browse/JDK-8046101 >>>> >>>> Some of the constructs had been added already in early release. I'm >>>> planning to address the rest as individual rfe separately. >>>> >>>> Thanks, >>>> Sherman >>> >> > From steve.drach at oracle.com Wed Feb 3 17:24:16 2016 From: steve.drach at oracle.com (Steve Drach) Date: Wed, 3 Feb 2016 09:24:16 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56B1C2CC.7080102@oracle.com> References: <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC! .7080102@oracle.com> Message-ID: Thanks for the comments Alan. Responses in-line. >> I created a new webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.05/ , that implements what I outlined above. In particular I enhanced the JarEntryIterator class and I added additional commentary to the entries() and stream() methods. I also added a new test, MultiReleaseJarIterators, to test entries() and stream(). >> > I think having stream and entries do this is right although I would like to see some performance data if possible. I?ll see what I can do. I suspect the non-multi-release jar will be very comparable since there?s just a couple boolean tests that were added for this case. > Also I would expect that if a JAR file is not mult-release but the library opens it with Release.RUNTIME to perform the same as opening the JAR file with the Release-less constructors. Perhaps. There is a slightly different path with an additional method call and boolean test in this case, but I?ll try to get some metrics here too. > > I think the javadoc will need to also need to make it clear whether entries with names starting with META-INF/versions/ are returned. It was a bit difficult to explain in a succinct way, but the careful reader should be able to infer that the META-INF/versions/ entries are not returned when the constructor with the Release argument is invoked. I?ll try to add some additional detail. > > I see you've added @since 9 to the existing methods, I assume you didn't mean to do this. I did mean to do it, but now that you mention it, I see it was a mistake. I?ll fix that. > > At some point then we need to discuss how RUNTIME_VERSION is computed. Iris (via Mandy) has pushed jdk.Version to jdk9/dev but having it exported by java.base conflicts with the design principles in JEP 200. Moving it to another module means that code in java.base cannot use it and thus the JAR file can't use it. I guess I need to wait until that settles down a bit. From kubota.yuji at gmail.com Wed Feb 3 17:27:09 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Thu, 4 Feb 2016 02:27:09 +0900 Subject: [DING] Re: [PING] Potential infinite waiting at JMXConnection#createConnection Message-ID: Hi all, Could someone please review and sponsor this fix ? I write the details of this issue again. Please review it. =Problem= Potential infinite waiting at TCPChannel#createConnection. This method flushes the DataOutputStream without the socket timeout settings when choose stream protocol [1]. If connection lost or the destination server do not return response during the flush, this method wait forever because the timeout settings is set the default value of SO_TIMEOUT, i.e., infinite. [1]: http://hg.openjdk.java.net/jdk9/dev/jdk/file/7adef1c3afd5/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java#l227 I think this issue is rarely, however serious. =Reproduce= I write a test program to reproduce. You can reproduce by the below. * hg clone http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/ * cd fixLoopAtJMXConnectorFactory; mvn package * setting "stop_time" at debugcontrol.properties if you need. * java -cp .:target/debugcontrol-1.0-SNAPSHOT.jar debugcontrol.DebugController This program keep to wait at TCPChannel#createConnection due to this issue. After "debugcontroltest.stop_time" ms, this program release the waiting by sending quit to jdb which is stopping the destination server. Finally, return 2. =Solution= Set timeout by using property-configured value: sun.rmi.transport.tcp.responseTimeout. My patch is below. http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch If you run the test program with modified JDK9 by my patch, the test program will get java.net.SocketTimeoutException after the connection timeout happen, then return 0. Thanks, Yuji. 2016-01-13 23:31 GMT+09:00 KUBOTA Yuji : > Hi all, > > Can somebody please review and sponsor this fix ? > > Thanks, > Yuji > > 2016-01-05 17:56 GMT+09:00 KUBOTA Yuji : >> Hi Jaroslav and core-libs-dev, >> >> Thank Jaroslav for your kindness! >> >> For core-libs-dev members, links the information about this issue. >> >> * details of problem >> http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002152.html >> >> * patch >> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch >> >> * testcase for reproduce >> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/testProgram >> http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-December/018415.html >> >> Could you please review these reports? >> Hope this patch helps to community. >> >> Thanks, >> Yuji >> >> 2016-01-04 23:51 GMT+09:00 Jaroslav Bachorik : >>> Hi Yuji, >>> >>> On 4.1.2016 15:14, KUBOTA Yuji wrote: >>>> >>>> Hi all, >>>> >>>> Could you please review this patch? >>> >>> >>> Sorry for the long delay. Shanliang has not been present for some time and >>> probably this slipped the attention of the others. >>> >>> However, core-libs mailing list might be more appropriate place to review >>> this change since you are dealing with s.r.t.t.TCPChannel >>> (http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch) >>> >>> Regards, >>> >>> -JB- From iris.clark at oracle.com Wed Feb 3 17:48:57 2016 From: iris.clark at oracle.com (Iris Clark) Date: Wed, 3 Feb 2016 09:48:57 -0800 (PST) Subject: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion In-Reply-To: <56B1BFE3.1010201@oracle.com> References: <50765a19-34e4-4115-a32d-cea84c25394e@default> <38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com> <565628C5.1040905@oracle.com> <92b9301d-2418-4618-b877-b13c88216520@default> <7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default> <2f3ec2a8-4488-4c21-ae79-01c4599d5121@default> <56B1BFE3.1010201@oracle.com> Message-ID: Hi, Alan. > Good to have this in but now we need to decide on where it > should live. It's JDK-specific so we'll need it exported > by a JDK module rather than java.base. 8144062 was next on my list. Do you have any suggestions for the module? Of the existing modules in the jdk repository, jdk.dev seems to have the most appropriate name. It appears to only contain jimage so I'm not sure about that module's charter. Thanks, iris -----Original Message----- From: Alan Bateman Sent: Wednesday, February 03, 2016 12:53 AM To: Iris Clark; Mandy Chung Cc: verona-dev at openjdk.java.net; core-libs-dev at openjdk.java.net Subject: Re: RFR: 8072379: Implement jdk.Version and jdk.OracleVersion On 03/02/2016 03:17, Iris Clark wrote: > Hi, Mandy. > > Thanks so much for pushing the changeset for the initial > implementation of jdk.Version! > Good to have this in but now we need to decide on where it should live. It's JDK-specific so we'll need it exported by a JDK module rather than java.base. -Alan From aleksej.efimov at oracle.com Wed Feb 3 18:19:37 2016 From: aleksej.efimov at oracle.com (Aleksej Efimov) Date: Wed, 3 Feb 2016 21:19:37 +0300 Subject: [9] RFR: 8144593: Suppress not recognized property/feature warning messages from SAXParser Message-ID: <56B244B9.2020404@oracle.com> Hi, Please, help to review the fix for JDK-8144593 bug [1] in jaxp area: http://cr.openjdk.java.net/~aefimov/8144593/9/00/ Problem: JDK produces warnings when non-default parser implementation is used and this parser doesn't support jaxp features/properties. The quantity of such messages can be enormous for long-running application that performs frequent xml transformation or validation operations. The proposed fix suppresses such messages and prints them only once per one JVM invocation for each unique pair of: parser class in use AND property name that is not supported by this parser. Testing results: JDK regression tests (with new regression tests for transformation and validation operations), JCK and JPRT shows no xml tests failures. Best Regards, Aleksej [1] https://bugs.openjdk.java.net/browse/JDK-8144593 From aleksey.shipilev at oracle.com Wed Feb 3 18:25:30 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 3 Feb 2016 21:25:30 +0300 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings Message-ID: <56B2461A.1030901@oracle.com> Hi, JDK-8006627 did the JavaLangAccess hack to improve UUID.toString() performance: public String toString() { char[] chars = new char[36]; jla.formatUnsignedLong(mostSigBits >> 32, 4, chars, 0, 8); chars[8] = '-'; jla.formatUnsignedLong(mostSigBits >> 16, 4, chars, 9, 4); chars[13] = '-'; jla.formatUnsignedLong(mostSigBits, 4, chars, 14, 4); chars[18] = '-'; jla.formatUnsignedLong(leastSigBits >> 48, 4, chars, 19, 4); chars[23] = '-'; jla.formatUnsignedLong(leastSigBits, 4, chars, 24, 12); return jla.newStringUnsafe(chars); } This is a good performance improvement, but it clashes with Compact Strings which now have to re-compress the resulting char[] array into byte[]. And we know that UUID would always produce Latin1 String. In fact, the entire JavaLangAccess.newStringUnsafe is now not doing what power users would expect: it *does* allocate now! So, we need to phase out that internal gateway to avoid confusion. UUID is one of these users (StringJoiner is another). This is the proposed fix: http://cr.openjdk.java.net/~shade/8148936/webrev.01/ My attempts in exposing the entire String coder business to UUID proved to be rather ugly, so I opted to just all into a single method, and let java/lang internals to sort this out. The patch does restore the post-Compact Strings performance, and even improves it further. See: http://cr.openjdk.java.net/~shade/8148936/notes.txt Cheers, -Aleksey From huizhe.wang at oracle.com Wed Feb 3 19:02:51 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 03 Feb 2016 11:02:51 -0800 Subject: [9] RFR: 8144593: Suppress not recognized property/feature warning messages from SAXParser In-Reply-To: <56B244B9.2020404@oracle.com> References: <56B244B9.2020404@oracle.com> Message-ID: <56B24EDB.1040206@oracle.com> Looks good Aleksej! -Joe On 2/3/2016 10:19 AM, Aleksej Efimov wrote: > Hi, > Please, help to review the fix for JDK-8144593 bug [1] in jaxp area: > http://cr.openjdk.java.net/~aefimov/8144593/9/00/ > Problem: > JDK produces warnings when non-default parser implementation is used > and this parser doesn't support jaxp features/properties. The quantity > of such messages can be enormous for long-running application that > performs frequent xml transformation or validation operations. > The proposed fix suppresses such messages and prints them only once > per one JVM invocation for each unique pair of: parser class in use > AND property name that is not supported by this parser. > > Testing results: > JDK regression tests (with new regression tests for transformation and > validation operations), JCK and JPRT shows no xml tests failures. > > Best Regards, > Aleksej > > [1] https://bugs.openjdk.java.net/browse/JDK-8144593 From ivan.gerasimov at oracle.com Wed Feb 3 19:16:52 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 3 Feb 2016 22:16:52 +0300 Subject: RFR: 8079136: Accessing a nested sublist leads to StackOverflowError In-Reply-To: References: <55486EA5.8000009@oracle.com> <56A9D4D7.5040506@oracle.com> <85988769.20160129101608@gmail.com> <56AB8FE0.5080301@oracle.com> <35864771.20160202121910@gmail.com> Message-ID: <56B25224.6030409@oracle.com> Hello! Here's the updated webrev with rangeCheckForAdd() restored in both AbstractList and ArrayList. http://cr.openjdk.java.net/~igerasim/8079136/07/webrev/ The fix was built/tested successfully on all supported platforms. Sincerely yours, Ivan On 02.02.2016 9:55, Martin Buchholz wrote: > On Mon, Feb 1, 2016 at 10:19 PM, Tagir F. Valeev wrote: > >> I have a doubt about replacing rangeCheckForAdd. What if size() == >> Integer.MAX_VALUE? This is not an issue for ArrayList as it's limited >> by MAX_ARRAY_SIZE which is Integer.MAX_VALUE - 8. > Actually, the limit is Integer.MAX_VALUE. But it only grows beyond > MAX_ARRAY_SIZE if there's no choice. > From Roger.Riggs at Oracle.com Wed Feb 3 21:15:28 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 3 Feb 2016 16:15:28 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B16E8D.6060208@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> Message-ID: <56B26DF0.9020905@Oracle.com> Hi David, On 2/2/2016 10:05 PM, David Holmes wrote: > On 3/02/2016 8:08 AM, Stuart Marks wrote: >> Hi Roger, >> >> It will be good to get this into the JDK. Lots of people have been >> asking for this. > > I think this API is a big mistake. The primary usecase seems to be > control-C interception for utilities like jshell. Adding a general > purpose signal raising and handling mechanism to the JDK does not seem > like a good solution to me. While you would need to use signal > management under the covers I think it would be much cleaner to expose > an API that actually captures what it is you want here: a mechanism to > manage "interrupt" and "terminate" events at the VM level, in a clean > cross-platform way. > > Aside: If you want to see some prior art in this area look at > PosixSignalHandler API in the Real-Time Specification for Java. > Looked at it; it provides access to Posix signals with async delivery. Is there some specific point of interest there? > Which reminds me - do you propose to support the POSIX real-time signals? No, I don't believe the VM supports them; it is an implementation limitation. Roger > > David > ----- > >> I have a few comments on the API. >> >> 1) Is there a way to query the set of signals supported? This might be a >> Set returned by a static method, for example. I agree that >> signal strings outside this set shouldn't be supported. >> >> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >> explicitly. Are these required to be implemented on all platforms, or >> just on "unix-like" platforms, are they just examples? What signals are >> available on Windows? >> >> 3) raise() is spec'd to throw an exception if there's no handler >> registered. But wouldn't it make sense to allow it if the default >> handler is registered? >> >> 4) In an earlier message you said that the Signal object is a >> capability, so the security check is on getting a reference. It seems to >> me that setting a handler is in a different category from raising a >> signal; this suggests to me that using the same object as a capability >> for both should be rethought. >> >> 5) I don't understand the asymmetry between register() and unregister(). >> Your earlier exchanges with Chris and with Gerard touched on this, >> specifically, the requirement that the caller pass unregister() a >> reference to the old handler in order for unregistration to work. You >> had said this was safer, if there are uncoordinated pieces of code >> attempting to set/unset signal handlers. >> >> It looks to me like this API is really about maintaining process global >> state consisting of a single handler -- user-specified or default -- for >> each supported signal. (I agree that it shouldn't try to have a stack or >> a chain of handlers.) There are a few other things that are global like >> this, such as the security manager and policy, System.setIn/Out/Err, and >> so forth. As such, uncoordinated access to the signal API is pretty much >> broken no matter what. Thus I don't think it makes sense to have a >> CAS-like protocol for unregistering a handler, to protect against the >> case where "somebody else" might have registered a handler different >> from yours. >> >> Something like this might make sense: >> >> void register(Consumer handler); >> void unregister(); >> >> The register() call would be pretty much as currently specified; the >> unregister() call would restore the default handler. Alternatively, >> register(null) could be used instead of unregister(), but this is quite >> minor. >> >> Thanks, >> >> s'marks >> >> >> >> >> >> On 2/1/16 8:02 AM, Roger Riggs wrote: >>> Please review an API addition to handle signals such as SIGINT, >>> SIGHUP, and >>> SIGTERM. >>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>> case for >>> interactive applications that need to handle Control-C and other >>> signals. >>> >>> The new java.util.Signal class provides a settable primary signal >>> handler and a >>> default >>> signal handler. The primary signal handler can be unregistered and >>> handling is >>> restored >>> to the default signal handler. System initialization registers >>> default signal >>> handlers >>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>> requires >>> a permission if a SecurityManager is set. >>> >>> The sun.misc.Signal implementation is modified to be layered on a >>> common >>> thread and dispatch mechanism. The VM handling of native signals is >>> not affected. >>> The command option to reduce signal use by the runtime with -Xrs is >>> unmodified. >>> >>> The changes to hotspot are minimal to rename the hardcoded callback to >>> the Java >>> Signal dispatcher. >>> >>> Please review and comment on the API and implementation. >>> >>> javadoc: >>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>> >>> Webrev: >>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>> >>> JEP 260: >>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> Thanks, Roger >>> >>> From stuart.marks at oracle.com Wed Feb 3 22:32:58 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Wed, 3 Feb 2016 14:32:58 -0800 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B16E8D.6060208@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> Message-ID: <56B2801A.7060508@oracle.com> On 2/2/16 7:05 PM, David Holmes wrote: > On 3/02/2016 8:08 AM, Stuart Marks wrote: >> It will be good to get this into the JDK. Lots of people have been >> asking for this. > > I think this API is a big mistake. The primary usecase seems to be control-C > interception for utilities like jshell. Adding a general purpose signal raising > and handling mechanism to the JDK does not seem like a good solution to me. > While you would need to use signal management under the covers I think it would > be much cleaner to expose an API that actually captures what it is you want > here: a mechanism to manage "interrupt" and "terminate" events at the VM level, > in a clean cross-platform way. OK, I've looked some at the implementation, and there's more going on here than meets the eye. I was under the impression (or at least I was hoping) that the API would expose a carefully curated set of signals that are (a) known to be useful to applications, and (b) are safe for the JVM to allow applications to handle. Examples of this would include SIGWINCH and SIGTSTP, which are common for Unix applications to want to handle, as well as the Control-C (SIGINT) case that jshell among others want to handle. But I tried out the patch and looked through the Hotspot signal handling code, and the set of signals exposed is much broader than I would have expected. On Mac OS X, the signals for which a handler can be registered include the following: SIGABRT SIGALRM SIGBUS SIGCHLD SIGCONT SIGEMT SIGHUP SIGINT SIGIO SIGPIPE SIGPROF SIGSYS SIGTERM SIGTRAP SIGTSTP SIGTTIN SIGTTOU SIGURG SIGUSR1 SIGUSR2 SIGVTALRM SIGWINCH SIGXCPU SIGXFSZ I'm quite surprised by this. It seems quite unwise to expose all of these. Perhaps this is what David is concerned about. If so, I'm starting to share his concern. In addition, the signals for which a Signal instance can be gotten via Signal.of(), but which cannot be handled (throws UOE), include: SIGFPE SIGILL SIGKILL SIGQUIT SIGSEGV SIGSTOP It's very strange to expose Signal instances representing these signals when they can't be handled. They can't be raised either (at least in Roger's first patch) since raising a signal is prohibited if there's no handler installed. I still think a signal-handling API, even a system-specific one, can be useful for a well curated set of signals. But this implementation seems to bring an internal interface directly out to the API. That doesn't seem like the right approach. s'marks From peter.firmstone at zeus.net.au Thu Feb 4 00:40:22 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Thu, 4 Feb 2016 10:40:22 +1000 (AEST) Subject: ObjectInputStream SPI Message-ID: <2a7859285494d908effbc200502cbd52@org.tizen.email> In light of recent examples of gadget deserialization attacks, I believe we need an OIS SPI. While OIS functionality can be overridden, there's no way to ensure this can be done for all uses of OIS. I believe this is necessary for security reasons, to allow Serialization to be completely disabled or restricted to only those classes in use by an application or reimplemented to allow input validation. An OIS SPI would be a very simple straightforward solution. Regards, Peter Firmstone. Sent from my Samsung device. ? From david.holmes at oracle.com Thu Feb 4 00:59:21 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Feb 2016 10:59:21 +1000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B225C5.60907@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2242B.3030105@oracle.com> <56B225C5.60907@Oracle.com> Message-ID: <56B2A269.4090102@oracle.com> Adding back hotspot-runtime-dev On 4/02/2016 2:07 AM, Roger Riggs wrote: > Hi, > > The current wording is explicit about signal support being > implementation and os dependent, > it can say its release dependent too. So you think an API that is implementation dependent, OS dependent and release dependent, is a good candidate for the Write-Once-Run-Anywhere Java platform as a core API addition ??? At best this should be in some side add-on (anyone remember javax.*). David ----- > > Roger > > > > On 2/3/2016 11:00 AM, Chris Hegarty wrote: >> On 03/02/16 15:43, Thomas St?fe wrote: >>> On Wed, Feb 3, 2016 at 4:20 PM, Thomas St?fe >>> wrote: >>> >>>> On Wed, Feb 3, 2016 at 4:05 AM, David Holmes >>>> wrote: >>>> >>>>> On 3/02/2016 8:08 AM, Stuart Marks wrote: >>>>> >>>>>> Hi Roger, >>>>>> >>>>>> It will be good to get this into the JDK. Lots of people have been >>>>>> asking for this. >>>>>> >>>>> >>>>> I think this API is a big mistake. The primary usecase seems to be >>>>> control-C interception for utilities like jshell. Adding a general >>>>> purpose >>>>> signal raising and handling mechanism to the JDK does not seem like >>>>> a good >>>>> solution to me. While you would need to use signal management under >>>>> the >>>>> covers I think it would be much cleaner to expose an API that actually >>>>> captures what it is you want here: a mechanism to manage >>>>> "interrupt" and >>>>> "terminate" events at the VM level, in a clean cross-platform way. >>>>> >>>>> >>>> I agree with David. One problem I see is that it is difficult to write >>>> portable java applications with this API. Not only are WIndows and >>>> Posix >>>> are very different, but also there are also sublte differences between >>>> Posix platforms. For instance, in the jbs SIGTRAP was mentioned as a >>>> possible signal someone wanted to raise, but SIGTRAP is used by the >>>> JIT in >>>> the powerpc port. So applications using Signal.of("SIGTRAP") would >>>> run fine >>>> on x86, but cause a crash on powerpc. >>>> >>>> Kind Regards, Thomas >>>> >>>> >>> Just occurred to me that a second, subtle problem may be that once java >>> developers start using signals for their own application, we are barred >>> from using the same signals in the future for our own purposes, even >>> if we >>> do not use them now. At least not without breaking those java >>> applications. >> >> This is an excellent point. The API should include appropriate wording >> to indicate that there is no guarantee that an invocation of >> Signal.of(...) that succeeds in one release, is guaranteed to succeed >> in another, future, release. >> >> -Chris. >> >>> ..Thomas >>> >>> >>> >>>> Aside: If you want to see some prior art in this area look at >>>>> PosixSignalHandler API in the Real-Time Specification for Java. >>>>> >>>>> Which reminds me - do you propose to support the POSIX real-time >>>>> signals? >>>>> >>>>> David >>>>> ----- >>>>> >>>>> >>>>> I have a few comments on the API. >>>>>> >>>>>> 1) Is there a way to query the set of signals supported? This >>>>>> might be a >>>>>> Set returned by a static method, for example. I agree that >>>>>> signal strings outside this set shouldn't be supported. >>>>>> >>>>>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>>>>> explicitly. Are these required to be implemented on all platforms, or >>>>>> just on "unix-like" platforms, are they just examples? What >>>>>> signals are >>>>>> available on Windows? >>>>>> >>>>>> 3) raise() is spec'd to throw an exception if there's no handler >>>>>> registered. But wouldn't it make sense to allow it if the default >>>>>> handler is registered? >>>>>> >>>>>> 4) In an earlier message you said that the Signal object is a >>>>>> capability, so the security check is on getting a reference. It >>>>>> seems to >>>>>> me that setting a handler is in a different category from raising a >>>>>> signal; this suggests to me that using the same object as a >>>>>> capability >>>>>> for both should be rethought. >>>>>> >>>>>> 5) I don't understand the asymmetry between register() and >>>>>> unregister(). >>>>>> Your earlier exchanges with Chris and with Gerard touched on this, >>>>>> specifically, the requirement that the caller pass unregister() a >>>>>> reference to the old handler in order for unregistration to work. You >>>>>> had said this was safer, if there are uncoordinated pieces of code >>>>>> attempting to set/unset signal handlers. >>>>>> >>>>>> It looks to me like this API is really about maintaining process >>>>>> global >>>>>> state consisting of a single handler -- user-specified or default >>>>>> -- for >>>>>> each supported signal. (I agree that it shouldn't try to have a >>>>>> stack or >>>>>> a chain of handlers.) There are a few other things that are global >>>>>> like >>>>>> this, such as the security manager and policy, >>>>>> System.setIn/Out/Err, and >>>>>> so forth. As such, uncoordinated access to the signal API is >>>>>> pretty much >>>>>> broken no matter what. Thus I don't think it makes sense to have a >>>>>> CAS-like protocol for unregistering a handler, to protect against the >>>>>> case where "somebody else" might have registered a handler different >>>>>> from yours. >>>>>> >>>>>> Something like this might make sense: >>>>>> >>>>>> void register(Consumer handler); >>>>>> void unregister(); >>>>>> >>>>>> The register() call would be pretty much as currently specified; the >>>>>> unregister() call would restore the default handler. Alternatively, >>>>>> register(null) could be used instead of unregister(), but this is >>>>>> quite >>>>>> minor. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> s'marks >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>>>>> >>>>>>> Please review an API addition to handle signals such as SIGINT, >>>>>>> SIGHUP, and >>>>>>> SIGTERM. >>>>>>> This JEP 260 motivated alternative to sun.misc.Signal supports >>>>>>> the use >>>>>>> case for >>>>>>> interactive applications that need to handle Control-C and other >>>>>>> signals. >>>>>>> >>>>>>> The new java.util.Signal class provides a settable primary signal >>>>>>> handler and a >>>>>>> default >>>>>>> signal handler. The primary signal handler can be unregistered and >>>>>>> handling is >>>>>>> restored >>>>>>> to the default signal handler. System initialization registers >>>>>>> default signal >>>>>>> handlers >>>>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>>>>> requires >>>>>>> a permission if a SecurityManager is set. >>>>>>> >>>>>>> The sun.misc.Signal implementation is modified to be layered on a >>>>>>> common >>>>>>> thread and dispatch mechanism. The VM handling of native signals is >>>>>>> not affected. >>>>>>> The command option to reduce signal use by the runtime with -Xrs is >>>>>>> unmodified. >>>>>>> >>>>>>> The changes to hotspot are minimal to rename the hardcoded >>>>>>> callback to >>>>>>> the Java >>>>>>> Signal dispatcher. >>>>>>> >>>>>>> Please review and comment on the API and implementation. >>>>>>> >>>>>>> javadoc: >>>>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>>>> >>>>>>> Webrev: >>>>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>>>> hotspot: >>>>>>> http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>>>> >>>>>>> Issue: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>>>> >>>>>>> JEP 260: >>>>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>>>> >>>>>>> Thanks, Roger >>>>>>> >>>>>>> >>>>>>> >>>> > From david.holmes at oracle.com Thu Feb 4 01:03:43 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Feb 2016 11:03:43 +1000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B26DF0.9020905@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B26DF0.9020905@Oracle.com> Message-ID: <56B2A36F.2030204@oracle.com> On 4/02/2016 7:15 AM, Roger Riggs wrote: > Hi David, > > On 2/2/2016 10:05 PM, David Holmes wrote: >> On 3/02/2016 8:08 AM, Stuart Marks wrote: >>> Hi Roger, >>> >>> It will be good to get this into the JDK. Lots of people have been >>> asking for this. >> >> I think this API is a big mistake. The primary usecase seems to be >> control-C interception for utilities like jshell. Adding a general >> purpose signal raising and handling mechanism to the JDK does not seem >> like a good solution to me. While you would need to use signal >> management under the covers I think it would be much cleaner to expose >> an API that actually captures what it is you want here: a mechanism to >> manage "interrupt" and "terminate" events at the VM level, in a clean >> cross-platform way. >> >> Aside: If you want to see some prior art in this area look at >> PosixSignalHandler API in the Real-Time Specification for Java. >> > Looked at it; it provides access to Posix signals with async delivery. > Is there some specific point of interest there? Just an example of how signal interaction in Java has been handled in the past. > >> Which reminds me - do you propose to support the POSIX real-time signals? > No, I don't believe the VM supports them; it is an implementation > limitation. Not sure what you mean here. The VM doesn't use them - which kind-of makes them ideal for application code use as they won't interfere with the "normal" signals which have very specific purposes and mostly are not intended for general use. David > Roger > >> >> David >> ----- >> >>> I have a few comments on the API. >>> >>> 1) Is there a way to query the set of signals supported? This might be a >>> Set returned by a static method, for example. I agree that >>> signal strings outside this set shouldn't be supported. >>> >>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>> explicitly. Are these required to be implemented on all platforms, or >>> just on "unix-like" platforms, are they just examples? What signals are >>> available on Windows? >>> >>> 3) raise() is spec'd to throw an exception if there's no handler >>> registered. But wouldn't it make sense to allow it if the default >>> handler is registered? >>> >>> 4) In an earlier message you said that the Signal object is a >>> capability, so the security check is on getting a reference. It seems to >>> me that setting a handler is in a different category from raising a >>> signal; this suggests to me that using the same object as a capability >>> for both should be rethought. >>> >>> 5) I don't understand the asymmetry between register() and unregister(). >>> Your earlier exchanges with Chris and with Gerard touched on this, >>> specifically, the requirement that the caller pass unregister() a >>> reference to the old handler in order for unregistration to work. You >>> had said this was safer, if there are uncoordinated pieces of code >>> attempting to set/unset signal handlers. >>> >>> It looks to me like this API is really about maintaining process global >>> state consisting of a single handler -- user-specified or default -- for >>> each supported signal. (I agree that it shouldn't try to have a stack or >>> a chain of handlers.) There are a few other things that are global like >>> this, such as the security manager and policy, System.setIn/Out/Err, and >>> so forth. As such, uncoordinated access to the signal API is pretty much >>> broken no matter what. Thus I don't think it makes sense to have a >>> CAS-like protocol for unregistering a handler, to protect against the >>> case where "somebody else" might have registered a handler different >>> from yours. >>> >>> Something like this might make sense: >>> >>> void register(Consumer handler); >>> void unregister(); >>> >>> The register() call would be pretty much as currently specified; the >>> unregister() call would restore the default handler. Alternatively, >>> register(null) could be used instead of unregister(), but this is quite >>> minor. >>> >>> Thanks, >>> >>> s'marks >>> >>> >>> >>> >>> >>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>>> Please review an API addition to handle signals such as SIGINT, >>>> SIGHUP, and >>>> SIGTERM. >>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>>> case for >>>> interactive applications that need to handle Control-C and other >>>> signals. >>>> >>>> The new java.util.Signal class provides a settable primary signal >>>> handler and a >>>> default >>>> signal handler. The primary signal handler can be unregistered and >>>> handling is >>>> restored >>>> to the default signal handler. System initialization registers >>>> default signal >>>> handlers >>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>> requires >>>> a permission if a SecurityManager is set. >>>> >>>> The sun.misc.Signal implementation is modified to be layered on a >>>> common >>>> thread and dispatch mechanism. The VM handling of native signals is >>>> not affected. >>>> The command option to reduce signal use by the runtime with -Xrs is >>>> unmodified. >>>> >>>> The changes to hotspot are minimal to rename the hardcoded callback to >>>> the Java >>>> Signal dispatcher. >>>> >>>> Please review and comment on the API and implementation. >>>> >>>> javadoc: >>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>> >>>> Webrev: >>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>> >>>> JEP 260: >>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> Thanks, Roger >>>> >>>> > From david.holmes at oracle.com Thu Feb 4 02:14:47 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Feb 2016 12:14:47 +1000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B102AD.7020800@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> Message-ID: <56B2B417.3050803@oracle.com> Hi Mikael, Can't really comment on the bit-twiddling details. A couple of minor style nits: - don't put "return" on a line by itself, include the first part of the return expression - spaces after commas in template definitions/instantiation The JVM_ENTRY_FROM_LEAF etc was a little mind twisting but seems okay. Otherwise hotspot and JDK code appear okay. Thanks, David On 3/02/2016 5:25 AM, Mikael Vidstedt wrote: > > Please review this change which introduces a Copy::conjoint_swap and an > Unsafe.copySwapMemory method to call it from Java, along with the > necessary changes to have java.nio.Bits call it instead of the Bits.c code. > > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ > > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ > > On the jdk/ side I don't think there should be a lot of surprises. > Bits.c is gone and that required a mapfile-vers to be changed > accordingly. I also added a relatively extensive > jdk/internal/misc/Unsafe/CopySwap.java test which exercises all the > various copySwap configurations and verifies that the resulting data is > correct. There are also a handful of negative tests in there. > > On the hotspot/ side: > > * the copy logic in copy.cpp is leveraging templates to help the C++ > compiler produce tight copy loops for the various configurations > {element type, copy direction, src aligned, dst aligned}. > * Unsafe_CopySwapMemory is a leaf to not stall safe points more than > necessary. Only if needed (THROW, copy involves heap objects) will it > enter VM using a new JVM_ENTRY_FROM_LEAF macro. > * JVM_ENTRY_FROM_LEAF calls a new VM_ENTRY_BASE_FROM_LEAF helper macro, > which mimics what VM_ENTRY_BASE does, but also does a > debug_only(ResetNoHandleMark __rnhm;) - this is because > JVM_LEAF/VM_LEAF_BASE does debug_only(NoHandleMark __hm;). > > I'm in the process of getting the last performance numbers, but from > what I've seen so far this will outperform the earlier implementation. > > Cheers, > Mikeal > > On 2016-01-27 17:13, Mikael Vidstedt wrote: >> >> Just an FYI: >> >> I'm working on moving all of this to the Hotspot Copy class and >> bridging to it via jdk.internal.misc.Unsafe, removing Bits.c >> altogether. The implementation is working, and the preliminary >> performance numbers beat the pants off of any of the suggested Bits.c >> implementations (yay!). >> >> I'm currently in the progress of getting some unit tests in place for >> it all to make sure it covers all the corner cases and then I'll run >> some real benchmarks to see if it actually lives up to the expectations. >> >> Cheers, >> Mikael >> >> On 2016-01-26 11:13, John Rose wrote: >>> On Jan 26, 2016, at 11:08 AM, Andrew Haley wrote: >>>> On 01/26/2016 07:04 PM, John Rose wrote: >>>>> Unsafe.copyMemory bottoms out to Copy::conjoint_memory_atomic. >>>>> IMO that's a better starting point than memcpy. Perhaps it can be >>>>> given an additional parameter (or overloading) to specify a swap size. >>>> OK, but conjoint_memory_atomic doesn't guarantee that destination >>>> words won't be torn if their source is misaligned: in fact it >>>> guarantees that they will will be. >>> That's a good point, and argues for a new function with the >>> stronger guarantee. Actually, it would be perfectly reasonable >>> to strengthen the guarantee on the existing function. I don't >>> think anyone will care about the slight performance change, >>> especially since it is probably favorable. Since it's Unsafe, >>> they are not supposed to care, either. >>> >>> ? John >> > From huaming.li at oracle.com Thu Feb 4 03:18:43 2016 From: huaming.li at oracle.com (Hamlin Li) Date: Thu, 4 Feb 2016 11:18:43 +0800 Subject: RFR: 8148928: java/util/stream/test/**/SequentialOpTest.java timed out intermittently Message-ID: <56B2C313.6020402@oracle.com> Hi everyone, Would you please help to review the fix for bug https://bugs.openjdk.java.net/browse/JDK-8148928, java/util/stream/test/**/SequentialOpTest.java timed out intermittently. webrev: http://cr.openjdk.java.net/~mli/8148928/webrev.00/ Thank you -Hamlin From nadeesh.tv at oracle.com Thu Feb 4 05:27:31 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Thu, 04 Feb 2016 10:57:31 +0530 Subject: RFR:JDK-8146747:java.time.Duration.toNanos() and toMillis() exception on negative durations In-Reply-To: <56B1F008.5030800@oracle.com> References: <56AEF8B8.4070703@oracle.com> <56B1F008.5030800@oracle.com> Message-ID: <56B2E143.7050308@oracle.com> Hi all, Please see the updated webrev http://cr.openjdk.java.net/~ntv/8146747/webrev.01/ Regards, Nadeesh On 2/3/2016 5:48 PM, nadeesh tv wrote: > Hi Shinya, > Thnx. I will update it. > Regards, > Nadeesh > On 2/3/2016 5:41 PM, ShinyaYoshida wrote: >> Hi Nadeesh, >> Almost LGTM!(But I'm not a reviewer;) ) >> However I've noticed that you don't use NANOS_PER_SECOND at L1223 and >> L1246. >> Is there some reason not to use it? >> >> Regards, >> shinyafox(Shinya Yoshida) >> >> 2016-02-01 15:18 GMT+09:00 nadeesh tv > >: >> >> Hi all, >> >> Please review following >> >> Bug Id : https://bugs.openjdk.java.net/browse/JDK-8146747 >> >> >> Solution: Handled the negative duration separately >> >> webrev : http://cr.openjdk.java.net/~ntv/8146747/webrev.00/ >> >> >> >> -- Thanks and Regards, >> Nadeesh TV >> >> > -- Thanks and Regards, Nadeesh TV From amy.lu at oracle.com Thu Feb 4 06:22:02 2016 From: amy.lu at oracle.com (Amy Lu) Date: Thu, 4 Feb 2016 14:22:02 +0800 Subject: JDK 9 RFR of JDK-8149003: Mark more jdk_core tests as intermittently failing Message-ID: <56B2EE0A.6050901@oracle.com> java/net/NetworkInterface/NetworkInterfaceStreamTest.java java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java java/rmi/Naming/DefaultRegistryPort.java java/rmi/transport/closeServerSocket/CloseServerSocket.java Above tests are known to fail intermittently, this patch is to mark the test accordingly with keyword 'intermittent'. bug: https://bugs.openjdk.java.net/browse/JDK-8149003 webrev: http://cr.openjdk.java.net/~amlu/8149003/webrev.00/ Thanks, Amy --- old/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 +++ new/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2015, 2016, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -28,6 +28,7 @@ * @build java.util.stream.OpTestCase * @run testng/othervm NetworkInterfaceStreamTest * @run testng/othervm -Djava.net.preferIPv4Stack=true NetworkInterfaceStreamTest + * @key intermittent */ import org.testng.annotations.Test; --- old/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 +++ new/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -33,6 +33,7 @@ * java.rmi/sun.rmi.transport.tcp * @build TestLibrary * @run main/othervm DefaultRegistryPort + * @key intermittent */ /* --- old/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 +++ new/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -33,6 +33,7 @@ * java.rmi/sun.rmi.transport.tcp * @build TestLibrary Legal LegalRegistryNames_Stub * @run main/othervm LegalRegistryNames + * @key intermittent */ import java.net.InetAddress; --- old/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 +++ new/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2005, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2005, 2016, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -37,6 +37,7 @@ * java.rmi/sun.rmi.transport.tcp * @build TestLibrary * @run main/othervm CloseServerSocket + * @key intermittent */ import java.io.IOException; From chris.hegarty at oracle.com Thu Feb 4 06:23:10 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 4 Feb 2016 06:23:10 +0000 Subject: JDK 9 RFR of JDK-8149003: Mark more jdk_core tests as intermittently failing In-Reply-To: <56B2EE0A.6050901@oracle.com> References: <56B2EE0A.6050901@oracle.com> Message-ID: Seems reasonable to me May. -Chris. On 4 Feb 2016, at 06:22, Amy Lu wrote: > java/net/NetworkInterface/NetworkInterfaceStreamTest.java > java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java > java/rmi/Naming/DefaultRegistryPort.java > java/rmi/transport/closeServerSocket/CloseServerSocket.java > > Above tests are known to fail intermittently, this patch is to mark the test accordingly with keyword 'intermittent'. > > bug: https://bugs.openjdk.java.net/browse/JDK-8149003 > webrev: http://cr.openjdk.java.net/~amlu/8149003/webrev.00/ > > Thanks, > Amy > > --- old/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 > +++ new/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2015, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -28,6 +28,7 @@ > * @build java.util.stream.OpTestCase > * @run testng/othervm NetworkInterfaceStreamTest > * @run testng/othervm -Djava.net.preferIPv4Stack=true NetworkInterfaceStreamTest > + * @key intermittent > */ > import org.testng.annotations.Test; > --- old/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 > +++ new/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -33,6 +33,7 @@ > * java.rmi/sun.rmi.transport.tcp > * @build TestLibrary > * @run main/othervm DefaultRegistryPort > + * @key intermittent > */ > /* > --- old/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 > +++ new/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -33,6 +33,7 @@ > * java.rmi/sun.rmi.transport.tcp > * @build TestLibrary Legal LegalRegistryNames_Stub > * @run main/othervm LegalRegistryNames > + * @key intermittent > */ > import java.net.InetAddress; > --- old/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 > +++ new/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2005, 2013, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2005, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -37,6 +37,7 @@ > * java.rmi/sun.rmi.transport.tcp > * @build TestLibrary > * @run main/othervm CloseServerSocket > + * @key intermittent > */ > import java.io.IOException; > > From chris.hegarty at oracle.com Thu Feb 4 06:24:36 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 4 Feb 2016 06:24:36 +0000 Subject: JDK 9 RFR of JDK-8149003: Mark more jdk_core tests as intermittently failing In-Reply-To: References: <56B2EE0A.6050901@oracle.com> Message-ID: <706A73A9-FBDF-4692-9EBB-679997F7E7AA@oracle.com> s/May/Amy ;-) -Chris. On 4 Feb 2016, at 06:23, Chris Hegarty wrote: > Seems reasonable to me May. > > -Chris. > > On 4 Feb 2016, at 06:22, Amy Lu wrote: > >> java/net/NetworkInterface/NetworkInterfaceStreamTest.java >> java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java >> java/rmi/Naming/DefaultRegistryPort.java >> java/rmi/transport/closeServerSocket/CloseServerSocket.java >> >> Above tests are known to fail intermittently, this patch is to mark the test accordingly with keyword 'intermittent'. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8149003 >> webrev: http://cr.openjdk.java.net/~amlu/8149003/webrev.00/ >> >> Thanks, >> Amy >> >> --- old/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 >> +++ new/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. >> + * Copyright (c) 2015, 2016, Oracle and/or its affiliates. All rights reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -28,6 +28,7 @@ >> * @build java.util.stream.OpTestCase >> * @run testng/othervm NetworkInterfaceStreamTest >> * @run testng/othervm -Djava.net.preferIPv4Stack=true NetworkInterfaceStreamTest >> + * @key intermittent >> */ >> import org.testng.annotations.Test; >> --- old/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 >> +++ new/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. >> + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -33,6 +33,7 @@ >> * java.rmi/sun.rmi.transport.tcp >> * @build TestLibrary >> * @run main/othervm DefaultRegistryPort >> + * @key intermittent >> */ >> /* >> --- old/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 >> +++ new/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. >> + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -33,6 +33,7 @@ >> * java.rmi/sun.rmi.transport.tcp >> * @build TestLibrary Legal LegalRegistryNames_Stub >> * @run main/othervm LegalRegistryNames >> + * @key intermittent >> */ >> import java.net.InetAddress; >> --- old/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 >> +++ new/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 >> @@ -1,5 +1,5 @@ >> /* >> - * Copyright (c) 2005, 2013, Oracle and/or its affiliates. All rights reserved. >> + * Copyright (c) 2005, 2016, Oracle and/or its affiliates. All rights reserved. >> * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. >> * >> * This code is free software; you can redistribute it and/or modify it >> @@ -37,6 +37,7 @@ >> * java.rmi/sun.rmi.transport.tcp >> * @build TestLibrary >> * @run main/othervm CloseServerSocket >> + * @key intermittent >> */ >> import java.io.IOException; >> >> > From joe.darcy at oracle.com Thu Feb 4 06:24:54 2016 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 3 Feb 2016 22:24:54 -0800 Subject: JDK 9 RFR of JDK-8149003: Mark more jdk_core tests as intermittently failing In-Reply-To: <56B2EE0A.6050901@oracle.com> References: <56B2EE0A.6050901@oracle.com> Message-ID: <56B2EEB6.1090906@oracle.com> Hi Amy, Looks fine; thanks, -Joe On 2/3/2016 10:22 PM, Amy Lu wrote: > java/net/NetworkInterface/NetworkInterfaceStreamTest.java > java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java > java/rmi/Naming/DefaultRegistryPort.java > java/rmi/transport/closeServerSocket/CloseServerSocket.java > > Above tests are known to fail intermittently, this patch is to mark > the test accordingly with keyword 'intermittent'. > > bug: https://bugs.openjdk.java.net/browse/JDK-8149003 > webrev: http://cr.openjdk.java.net/~amlu/8149003/webrev.00/ > > Thanks, > Amy > > --- old/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 > +++ new/test/java/net/NetworkInterface/NetworkInterfaceStreamTest.java 2016-02-04 14:13:21.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2015, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2015, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -28,6 +28,7 @@ > * @build java.util.stream.OpTestCase > * @run testng/othervm NetworkInterfaceStreamTest > * @run testng/othervm -Djava.net.preferIPv4Stack=true NetworkInterfaceStreamTest > + * @key intermittent > */ > > import org.testng.annotations.Test; > --- old/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 > +++ new/test/java/rmi/Naming/DefaultRegistryPort.java 2016-02-04 14:13:22.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -33,6 +33,7 @@ > * java.rmi/sun.rmi.transport.tcp > * @build TestLibrary > * @run main/othervm DefaultRegistryPort > + * @key intermittent > */ > > /* > --- old/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 > +++ new/test/java/rmi/Naming/legalRegistryNames/LegalRegistryNames.java 2016-02-04 14:13:23.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 1999, 2012, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 1999, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -33,6 +33,7 @@ > * java.rmi/sun.rmi.transport.tcp > * @build TestLibrary Legal LegalRegistryNames_Stub > * @run main/othervm LegalRegistryNames > + * @key intermittent > */ > > import java.net.InetAddress; > --- old/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 > +++ new/test/java/rmi/transport/closeServerSocket/CloseServerSocket.java 2016-02-04 14:13:23.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2005, 2013, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2005, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -37,6 +37,7 @@ > * java.rmi/sun.rmi.transport.tcp > * @build TestLibrary > * @run main/othervm CloseServerSocket > + * @key intermittent > */ > > import java.io.IOException; > From john.r.rose at oracle.com Thu Feb 4 06:27:06 2016 From: john.r.rose at oracle.com (John Rose) Date: Wed, 3 Feb 2016 22:27:06 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B102AD.7020800@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> Message-ID: <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> On Feb 2, 2016, at 11:25 AM, Mikael Vidstedt wrote: > Please review this change which introduces a Copy::conjoint_swap and an Unsafe.copySwapMemory method to call it from Java, along with the necessary changes to have java.nio.Bits call it instead of the Bits.c code. > > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ This is very good. I have some nit-picks: These days, when we introduce a new intrinsic (@HSIntrCand), we write the argument checking code separately in a non-intrinsic bytecode method. In this case, we don't (yet) have an intrinsic binding for U.copy*, but we might in the future. (C intrinsifies memcpy, which is a precedent.) In any case, I would prefer if we could structure the argument checking code in a similar way, with Unsafe.java containing both copySwapMemory and a private copySwapMemory0. Then we can JIT-optimize the safety checks. You might as well extend the same treatment to the pre-existing copyMemory call. The most important check (and the only one in U.copyMemory) is to ensure that the size_t operand has not wrapped around from a Java negative value to a crazy-large size_t value. That's the low-hanging fruit. Checking the pointers (for null or oob) is more problematic, of course. Checking consistency around elemSize is cheap and easy, so I agree that the U.copySM should do that work also. Basically, Unsafe can do very basic checks if there is a tricky user model to enforce, but it mustn't "sign up" to guard the user against all errors. Rule of thumb: Unsafe calls don't throw NPEs, they just SEGV. And the rare bit that *does* throw (IAE usually) should be placed into Unsafe.java, not unsafe.cpp. (The best-practice rule for putting argument checking code outside of the intrinsic is a newer one, so Unsafe code might not always do this.) The comment "Generalizing it would be reasonable, but requires card marking" is bogus, since we never byte-swap managed pointers. The test logic will flow a little smoother if your GenericPointer guy, the onHeap version, stores the appropriate array base offset in his offset field. You won't have to mention p.isOnHeap nearly so much, and the code will set a slightly better example. The VM_ENTRY_BASE_FROM_LEAF macro is really cool. The C++ template code is cool also. It reminds me of the kind of work Gosling's "Ace" processor could do, but now it's mainstreamed for all to use in C++. We're going to get some of that goodness in Project Valhalla with specialization logic. I find it amazing that the right way to code this in C is to use memcpy for unaligned accesses and byte peek/poke into registers for byte-swapping operators. I'm glad we can write this code *once* for the JVM and JDK. Possible future work: If we can get a better handle on writing vectorizable loops from Java, including Unsafe-based ones, we can move some of the C code back up to Java. Perhaps U.copy* calls for very short lengths deserved to be broken out into small loops of U.get/put* (with alignment). I think you experimented with this, and there were problems with the JIT putting fail-safe memory barriers between U.get/put* calls. Paul's work on Array.mismatch ran into similar issues, with the right answer being to write manual vector code in assembly. Anyway, you can count me as a reviewer. Thanks, ? John From masayoshi.okutsu at oracle.com Thu Feb 4 06:47:01 2016 From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu) Date: Thu, 4 Feb 2016 15:47:01 +0900 Subject: RFR: 8148446: (tz) Support tzdata2016a In-Reply-To: <7050b3f7-d566-4097-bdcc-9cbf76c272e4@default> References: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> <7050b3f7-d566-4097-bdcc-9cbf76c272e4@default> Message-ID: <56B2F3E5.1070703@oracle.com> Hi Ramanand, I noticed that the zones in Yakutsk Time [1] seem to have their own names, such as "Khandyga Time" for Asia/Khandyga, and you seem to follow that convention for Asia/Chita. That causes some mismatch between the long names and abbreviations. I dag out some past tzdata fixes to see how that happened. What I found out is that the 2013b fix [2] used "Yakutsk Time", but that the 2013c [3] fix used "Khandyga Time". 2013b went to JDK 7 updates and earlier ones and 2013c went to 8. JDK 9 inherits the 8 fix. I prefer to restore the 2013b convention for Asia/Yakutsk, Asia/Chita, and Asia/Khandyga to have: {"Yakutsk Time", "YAKT", "Yakutsk Summer Time", "YAKST", "Yakutsk Time", "YAKT"} Thanks, Masayoshi [1] https://en.wikipedia.org/wiki/Yakutsk_Time [2] http://hg.openjdk.java.net/jdk7u/jdk7u-dev/jdk/rev/b8009df64dc8 [3] http://hg.openjdk.java.net/jdk8/jdk8/jdk/rev/ae35fdbab949 On 2/2/2016 8:00 PM, Ramanand Patil wrote: > HI all, > > Please review the latest TZDATA integration (tzdata2016a) to JDK9. > > Bug - https://bugs.openjdk.java.net/browse/JDK-8148446 > > Webrev - http://cr.openjdk.java.net/~rpatil/8148446/webrev.00/ > > > > All the TimeZone related tests are passed after integration. > > > > Regards, > > Ramanand. From thomas.stuefe at gmail.com Thu Feb 4 07:16:46 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Thu, 4 Feb 2016 08:16:46 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B2A36F.2030204@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B26DF0.9020905@Oracle.com> <56B2A36F.2030204@oracle.com> Message-ID: > > > >> Which reminds me - do you propose to support the POSIX real-time signals? >>> >> No, I don't believe the VM supports them; it is an implementation >> limitation. >> > > Not sure what you mean here. The VM doesn't use them - which kind-of makes > them ideal for application code use as they won't interfere with the > "normal" signals which have very specific purposes and mostly are not > intended for general use. > > Note that this is also platform dependend, e.g. Mac OS does not support them. Even for platforms which support them, the semantics differ from platform to platform, see e.g. http://zinascii.com/2014/crossed-signals.html . .. Thomas > David > > > Roger >> >> >>> David >>> ----- >>> >>> I have a few comments on the API. >>>> >>>> 1) Is there a way to query the set of signals supported? This might be a >>>> Set returned by a static method, for example. I agree that >>>> signal strings outside this set shouldn't be supported. >>>> >>>> 2) The Signal class spec mentions SIGINT, SIGHUP, and SIGTERM >>>> explicitly. Are these required to be implemented on all platforms, or >>>> just on "unix-like" platforms, are they just examples? What signals are >>>> available on Windows? >>>> >>>> 3) raise() is spec'd to throw an exception if there's no handler >>>> registered. But wouldn't it make sense to allow it if the default >>>> handler is registered? >>>> >>>> 4) In an earlier message you said that the Signal object is a >>>> capability, so the security check is on getting a reference. It seems to >>>> me that setting a handler is in a different category from raising a >>>> signal; this suggests to me that using the same object as a capability >>>> for both should be rethought. >>>> >>>> 5) I don't understand the asymmetry between register() and unregister(). >>>> Your earlier exchanges with Chris and with Gerard touched on this, >>>> specifically, the requirement that the caller pass unregister() a >>>> reference to the old handler in order for unregistration to work. You >>>> had said this was safer, if there are uncoordinated pieces of code >>>> attempting to set/unset signal handlers. >>>> >>>> It looks to me like this API is really about maintaining process global >>>> state consisting of a single handler -- user-specified or default -- for >>>> each supported signal. (I agree that it shouldn't try to have a stack or >>>> a chain of handlers.) There are a few other things that are global like >>>> this, such as the security manager and policy, System.setIn/Out/Err, and >>>> so forth. As such, uncoordinated access to the signal API is pretty much >>>> broken no matter what. Thus I don't think it makes sense to have a >>>> CAS-like protocol for unregistering a handler, to protect against the >>>> case where "somebody else" might have registered a handler different >>>> from yours. >>>> >>>> Something like this might make sense: >>>> >>>> void register(Consumer handler); >>>> void unregister(); >>>> >>>> The register() call would be pretty much as currently specified; the >>>> unregister() call would restore the default handler. Alternatively, >>>> register(null) could be used instead of unregister(), but this is quite >>>> minor. >>>> >>>> Thanks, >>>> >>>> s'marks >>>> >>>> >>>> >>>> >>>> >>>> On 2/1/16 8:02 AM, Roger Riggs wrote: >>>> >>>>> Please review an API addition to handle signals such as SIGINT, >>>>> SIGHUP, and >>>>> SIGTERM. >>>>> This JEP 260 motivated alternative to sun.misc.Signal supports the use >>>>> case for >>>>> interactive applications that need to handle Control-C and other >>>>> signals. >>>>> >>>>> The new java.util.Signal class provides a settable primary signal >>>>> handler and a >>>>> default >>>>> signal handler. The primary signal handler can be unregistered and >>>>> handling is >>>>> restored >>>>> to the default signal handler. System initialization registers >>>>> default signal >>>>> handlers >>>>> to terminate on SIGINT, SIGHUP, and SIGTERM. Use of the Signal API >>>>> requires >>>>> a permission if a SecurityManager is set. >>>>> >>>>> The sun.misc.Signal implementation is modified to be layered on a >>>>> common >>>>> thread and dispatch mechanism. The VM handling of native signals is >>>>> not affected. >>>>> The command option to reduce signal use by the runtime with -Xrs is >>>>> unmodified. >>>>> >>>>> The changes to hotspot are minimal to rename the hardcoded callback to >>>>> the Java >>>>> Signal dispatcher. >>>>> >>>>> Please review and comment on the API and implementation. >>>>> >>>>> javadoc: >>>>> http://cr.openjdk.java.net/~rriggs/signal-doc/ >>>>> >>>>> Webrev: >>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-8087286/ >>>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-8087286/ >>>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8087286 >>>>> >>>>> JEP 260: >>>>> https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> >> From ramanand.patil at oracle.com Thu Feb 4 08:55:06 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Thu, 4 Feb 2016 00:55:06 -0800 (PST) Subject: RFR: 8148446: (tz) Support tzdata2016a In-Reply-To: <56B2F3E5.1070703@oracle.com> References: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> <7050b3f7-d566-4097-bdcc-9cbf76c272e4@default> <56B2F3E5.1070703@oracle.com> Message-ID: Hi Masayoshi/all, Please review the updated Webrev at: http://cr.openjdk.java.net/~rpatil/8148446/webrev.01/ Regards, Ramanand. -----Original Message----- From: Masayoshi Okutsu Sent: Thursday, February 04, 2016 12:17 PM To: Ramanand Patil; i18n-dev at openjdk.java.net Cc: core-libs-dev at openjdk.java.net Subject: Re: RFR: 8148446: (tz) Support tzdata2016a Hi Ramanand, I noticed that the zones in Yakutsk Time [1] seem to have their own names, such as "Khandyga Time" for Asia/Khandyga, and you seem to follow that convention for Asia/Chita. That causes some mismatch between the long names and abbreviations. I dag out some past tzdata fixes to see how that happened. What I found out is that the 2013b fix [2] used "Yakutsk Time", but that the 2013c [3] fix used "Khandyga Time". 2013b went to JDK 7 updates and earlier ones and 2013c went to 8. JDK 9 inherits the 8 fix. I prefer to restore the 2013b convention for Asia/Yakutsk, Asia/Chita, and Asia/Khandyga to have: {"Yakutsk Time", "YAKT", "Yakutsk Summer Time", "YAKST", "Yakutsk Time", "YAKT"} Thanks, Masayoshi [1] https://en.wikipedia.org/wiki/Yakutsk_Time [2] http://hg.openjdk.java.net/jdk7u/jdk7u-dev/jdk/rev/b8009df64dc8 [3] http://hg.openjdk.java.net/jdk8/jdk8/jdk/rev/ae35fdbab949 On 2/2/2016 8:00 PM, Ramanand Patil wrote: > HI all, > > Please review the latest TZDATA integration (tzdata2016a) to JDK9. > > Bug - https://bugs.openjdk.java.net/browse/JDK-8148446 > > Webrev - http://cr.openjdk.java.net/~rpatil/8148446/webrev.00/ > > > > All the TimeZone related tests are passed after integration. > > > > Regards, > > Ramanand. From ivan.gerasimov at oracle.com Thu Feb 4 11:34:52 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 4 Feb 2016 14:34:52 +0300 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings In-Reply-To: <56B2461A.1030901@oracle.com> References: <56B2461A.1030901@oracle.com> Message-ID: <56B3375C.1000207@oracle.com> Hi Aleksey! Looks good to me! Two minorish comments 1) 431 formatUnsignedLong0(lsb >> 48, 4, buf, 19, 4); I would suggest using unsigned shift >>> here just to avoid thinking about whether the sign matters or not. Not a big deal, of course. 2) Do you want to add the bug id 8148936 to the @bug list in the regression test? Sincerely yours, Ivan On 03.02.2016 21:25, Aleksey Shipilev wrote: > Hi, > > JDK-8006627 did the JavaLangAccess hack to improve UUID.toString() > performance: > > public String toString() { > char[] chars = new char[36]; > jla.formatUnsignedLong(mostSigBits >> 32, 4, chars, 0, 8); > chars[8] = '-'; > jla.formatUnsignedLong(mostSigBits >> 16, 4, chars, 9, 4); > chars[13] = '-'; > jla.formatUnsignedLong(mostSigBits, 4, chars, 14, 4); > chars[18] = '-'; > jla.formatUnsignedLong(leastSigBits >> 48, 4, chars, 19, 4); > chars[23] = '-'; > jla.formatUnsignedLong(leastSigBits, 4, chars, 24, 12); > return jla.newStringUnsafe(chars); > } > > This is a good performance improvement, but it clashes with Compact > Strings which now have to re-compress the resulting char[] array into > byte[]. And we know that UUID would always produce Latin1 String. > > In fact, the entire JavaLangAccess.newStringUnsafe is now not doing what > power users would expect: it *does* allocate now! So, we need to phase > out that internal gateway to avoid confusion. UUID is one of these users > (StringJoiner is another). > > This is the proposed fix: > http://cr.openjdk.java.net/~shade/8148936/webrev.01/ > > My attempts in exposing the entire String coder business to UUID proved > to be rather ugly, so I opted to just all into a single method, and let > java/lang internals to sort this out. > > The patch does restore the post-Compact Strings performance, and even > improves it further. See: > http://cr.openjdk.java.net/~shade/8148936/notes.txt > > Cheers, > -Aleksey > > From aleksey.shipilev at oracle.com Thu Feb 4 11:54:57 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 4 Feb 2016 14:54:57 +0300 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings In-Reply-To: <56B3375C.1000207@oracle.com> References: <56B2461A.1030901@oracle.com> <56B3375C.1000207@oracle.com> Message-ID: <56B33C11.2000907@oracle.com> Sure, here's a new webrev: http://cr.openjdk.java.net/~shade/8148936/webrev.02/ Cheers, -Aleksey On 02/04/2016 02:34 PM, Ivan Gerasimov wrote: > Hi Aleksey! > > Looks good to me! > > Two minorish comments > > 1) > 431 formatUnsignedLong0(lsb >> 48, 4, buf, 19, 4); > I would suggest using unsigned shift >>> here just to avoid thinking > about whether the sign matters or not. > Not a big deal, of course. > > 2) > Do you want to add the bug id 8148936 to the @bug list in the regression > test? > > Sincerely yours, > Ivan > > > On 03.02.2016 21:25, Aleksey Shipilev wrote: >> Hi, >> >> JDK-8006627 did the JavaLangAccess hack to improve UUID.toString() >> performance: >> >> public String toString() { >> char[] chars = new char[36]; >> jla.formatUnsignedLong(mostSigBits >> 32, 4, chars, 0, 8); >> chars[8] = '-'; >> jla.formatUnsignedLong(mostSigBits >> 16, 4, chars, 9, 4); >> chars[13] = '-'; >> jla.formatUnsignedLong(mostSigBits, 4, chars, 14, 4); >> chars[18] = '-'; >> jla.formatUnsignedLong(leastSigBits >> 48, 4, chars, 19, 4); >> chars[23] = '-'; >> jla.formatUnsignedLong(leastSigBits, 4, chars, 24, 12); >> return jla.newStringUnsafe(chars); >> } >> >> This is a good performance improvement, but it clashes with Compact >> Strings which now have to re-compress the resulting char[] array into >> byte[]. And we know that UUID would always produce Latin1 String. >> >> In fact, the entire JavaLangAccess.newStringUnsafe is now not doing what >> power users would expect: it *does* allocate now! So, we need to phase >> out that internal gateway to avoid confusion. UUID is one of these users >> (StringJoiner is another). >> >> This is the proposed fix: >> http://cr.openjdk.java.net/~shade/8148936/webrev.01/ >> >> My attempts in exposing the entire String coder business to UUID proved >> to be rather ugly, so I opted to just all into a single method, and let >> java/lang internals to sort this out. >> >> The patch does restore the post-Compact Strings performance, and even >> improves it further. See: >> http://cr.openjdk.java.net/~shade/8148936/notes.txt >> >> Cheers, >> -Aleksey >> >> > From ivan.gerasimov at oracle.com Thu Feb 4 12:10:36 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Thu, 4 Feb 2016 15:10:36 +0300 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings In-Reply-To: <56B33C11.2000907@oracle.com> References: <56B2461A.1030901@oracle.com> <56B3375C.1000207@oracle.com> <56B33C11.2000907@oracle.com> Message-ID: <56B33FBC.8080507@oracle.com> Looks even better, thanks! Sincerely yours, Ivan On 04.02.2016 14:54, Aleksey Shipilev wrote: > Sure, here's a new webrev: > http://cr.openjdk.java.net/~shade/8148936/webrev.02/ > > Cheers, > -Aleksey > > On 02/04/2016 02:34 PM, Ivan Gerasimov wrote: >> Hi Aleksey! >> >> Looks good to me! >> >> Two minorish comments >> >> 1) >> 431 formatUnsignedLong0(lsb >> 48, 4, buf, 19, 4); >> I would suggest using unsigned shift >>> here just to avoid thinking >> about whether the sign matters or not. >> Not a big deal, of course. >> >> 2) >> Do you want to add the bug id 8148936 to the @bug list in the regression >> test? >> >> Sincerely yours, >> Ivan >> >> >> On 03.02.2016 21:25, Aleksey Shipilev wrote: >>> Hi, >>> >>> JDK-8006627 did the JavaLangAccess hack to improve UUID.toString() >>> performance: >>> >>> public String toString() { >>> char[] chars = new char[36]; >>> jla.formatUnsignedLong(mostSigBits >> 32, 4, chars, 0, 8); >>> chars[8] = '-'; >>> jla.formatUnsignedLong(mostSigBits >> 16, 4, chars, 9, 4); >>> chars[13] = '-'; >>> jla.formatUnsignedLong(mostSigBits, 4, chars, 14, 4); >>> chars[18] = '-'; >>> jla.formatUnsignedLong(leastSigBits >> 48, 4, chars, 19, 4); >>> chars[23] = '-'; >>> jla.formatUnsignedLong(leastSigBits, 4, chars, 24, 12); >>> return jla.newStringUnsafe(chars); >>> } >>> >>> This is a good performance improvement, but it clashes with Compact >>> Strings which now have to re-compress the resulting char[] array into >>> byte[]. And we know that UUID would always produce Latin1 String. >>> >>> In fact, the entire JavaLangAccess.newStringUnsafe is now not doing what >>> power users would expect: it *does* allocate now! So, we need to phase >>> out that internal gateway to avoid confusion. UUID is one of these users >>> (StringJoiner is another). >>> >>> This is the proposed fix: >>> http://cr.openjdk.java.net/~shade/8148936/webrev.01/ >>> >>> My attempts in exposing the entire String coder business to UUID proved >>> to be rather ugly, so I opted to just all into a single method, and let >>> java/lang internals to sort this out. >>> >>> The patch does restore the post-Compact Strings performance, and even >>> improves it further. See: >>> http://cr.openjdk.java.net/~shade/8148936/notes.txt >>> >>> Cheers, >>> -Aleksey >>> >>> > From claes.redestad at oracle.com Thu Feb 4 12:19:19 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 4 Feb 2016 13:19:19 +0100 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings In-Reply-To: <56B33C11.2000907@oracle.com> References: <56B2461A.1030901@oracle.com> <56B3375C.1000207@oracle.com> <56B33C11.2000907@oracle.com> Message-ID: <56B341C7.9040601@oracle.com> Hi Aleksey, On 2016-02-04 12:54, Aleksey Shipilev wrote: > Sure, here's a new webrev: > http://cr.openjdk.java.net/~shade/8148936/webrev.02/ +1 Long::fastUUID feels odd, but I see why you ended up with that. Perhaps add a javadoc with a reference to the method in UUID that uses it? I can't help to wonder if there are other places where we want to similarly prepare either a byte[] or char[] for efficient, copy-free String allocation, and that exposing the package-private String constructors together with the COMPACT_STRINGS boolean in String might end up with a cleaner solution which would have more reusability. Thanks! /Claes > > Cheers, > -Aleksey > > On 02/04/2016 02:34 PM, Ivan Gerasimov wrote: >> Hi Aleksey! >> >> Looks good to me! >> >> Two minorish comments >> >> 1) >> 431 formatUnsignedLong0(lsb >> 48, 4, buf, 19, 4); >> I would suggest using unsigned shift >>> here just to avoid thinking >> about whether the sign matters or not. >> Not a big deal, of course. >> >> 2) >> Do you want to add the bug id 8148936 to the @bug list in the regression >> test? >> >> Sincerely yours, >> Ivan >> >> >> On 03.02.2016 21:25, Aleksey Shipilev wrote: >>> Hi, >>> >>> JDK-8006627 did the JavaLangAccess hack to improve UUID.toString() >>> performance: >>> >>> public String toString() { >>> char[] chars = new char[36]; >>> jla.formatUnsignedLong(mostSigBits >> 32, 4, chars, 0, 8); >>> chars[8] = '-'; >>> jla.formatUnsignedLong(mostSigBits >> 16, 4, chars, 9, 4); >>> chars[13] = '-'; >>> jla.formatUnsignedLong(mostSigBits, 4, chars, 14, 4); >>> chars[18] = '-'; >>> jla.formatUnsignedLong(leastSigBits >> 48, 4, chars, 19, 4); >>> chars[23] = '-'; >>> jla.formatUnsignedLong(leastSigBits, 4, chars, 24, 12); >>> return jla.newStringUnsafe(chars); >>> } >>> >>> This is a good performance improvement, but it clashes with Compact >>> Strings which now have to re-compress the resulting char[] array into >>> byte[]. And we know that UUID would always produce Latin1 String. >>> >>> In fact, the entire JavaLangAccess.newStringUnsafe is now not doing what >>> power users would expect: it *does* allocate now! So, we need to phase >>> out that internal gateway to avoid confusion. UUID is one of these users >>> (StringJoiner is another). >>> >>> This is the proposed fix: >>> http://cr.openjdk.java.net/~shade/8148936/webrev.01/ >>> >>> My attempts in exposing the entire String coder business to UUID proved >>> to be rather ugly, so I opted to just all into a single method, and let >>> java/lang internals to sort this out. >>> >>> The patch does restore the post-Compact Strings performance, and even >>> improves it further. See: >>> http://cr.openjdk.java.net/~shade/8148936/notes.txt >>> >>> Cheers, >>> -Aleksey >>> >>> > From aph at redhat.com Thu Feb 4 12:22:34 2016 From: aph at redhat.com (Andrew Haley) Date: Thu, 4 Feb 2016 12:22:34 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B102AD.7020800@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> Message-ID: <56B3428A.3010704@redhat.com> On 02/02/2016 07:25 PM, Mikael Vidstedt wrote: > Please review this change which introduces a Copy::conjoint_swap and an > Unsafe.copySwapMemory method to call it from Java, along with the > necessary changes to have java.nio.Bits call it instead of the Bits.c code. > > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ One other little thing: why are the byte-swapping methods in class nio.Bits not called copySwapSomething? e.g.: 826 /** 827 * Copy and byte swap 16 bit elements from off-heap memory to a heap array 828 * 829 * @param srcAddr 830 * source address 831 * @param dst 832 * destination array, must be a 16-bit primitive array type 833 * @param dstPos 834 * byte offset within the destination array of the first element to write 835 * @param length 836 * number of bytes to copy 837 */ 838 static void copyToCharArray(long srcAddr, Object dst, long dstPos, long length) { 839 unsafe.copySwapMemory(null, srcAddr, dst, unsafe.arrayBaseOffset(dst.getClass()) + dstPos, length, 2); 840 } Andrew. From aleksey.shipilev at oracle.com Thu Feb 4 13:04:38 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 4 Feb 2016 16:04:38 +0300 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings In-Reply-To: <56B33FBC.8080507@oracle.com> References: <56B2461A.1030901@oracle.com> <56B3375C.1000207@oracle.com> <56B33C11.2000907@oracle.com> <56B33FBC.8080507@oracle.com> Message-ID: <56B34C66.9060409@oracle.com> Thanks guys, pushed. Now, on to StringJoiner... -Aleksey On 02/04/2016 03:10 PM, Ivan Gerasimov wrote: > Looks even better, thanks! > > Sincerely yours, > Ivan > > On 04.02.2016 14:54, Aleksey Shipilev wrote: >> Sure, here's a new webrev: >> http://cr.openjdk.java.net/~shade/8148936/webrev.02/ From shilpi.rastogi at oracle.com Thu Feb 4 13:16:48 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Thu, 04 Feb 2016 18:46:48 +0530 Subject: RFR: 8071368 Use more concrete types for NamedFunction constants in the code Message-ID: <56B34F40.1010800@oracle.com> Hi All, Please review the following- https://bugs.openjdk.java.net/browse/JDK-8071368 http://cr.openjdk.java.net/~srastogi/8071368/webrev.02/ Thanks, Shilpi From shilpi.rastogi at oracle.com Thu Feb 4 13:17:26 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Thu, 04 Feb 2016 18:47:26 +0530 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <56AB5E6C.1010201@oracle.com> References: <56A21316.1070007@oracle.com> <56AB5E6C.1010201@oracle.com> Message-ID: <56B34F66.2090304@oracle.com> Gentle reminder! On 1/29/2016 6:13 PM, shilpi rastogi wrote: > Hi All, > > Please review the updated patch- > > http://cr.openjdk.java.net/~srastogi/8138578/webrev.01/ > > I verified Lookup.unreflectSpecial() also throws > java.lang.IllegalAccessException so updated the JavaDoc. > > Thanks, > Shilpi > >> >> -------- Original Message -------- >> Subject: Re: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc >> fails to consider static methods >> Date: Fri, 22 Jan 2016 10:30:29 +0100 >> From: Paul Sandoz >> CC: core-libs-dev at openjdk.java.net >> >> >> >> Hi Shilpi, >> >> Can you also double check Lookup.unreflectSpecial, it?s documentation might require updating too. >> >> Separating the ?or? statements with a comma would help readability in the JavaDoc (same applies to the the patch for findVirtual). >> >> Paul. >> >> > On 22 Jan 2016, at 10:22, shilpi rastogi wrote: >> > >> > >> > Gentle Reminder! >> > >> > -------- Original Message -------- >> > Subject: [9u-dev] 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods >> > Date: Wed, 20 Jan 2016 18:12:30 +0530 >> > From: shilpi rastogi >> > Organization: Oracle Corporation >> > To: core-libs-dev at openjdk.java.net >> > >> > >> > >> > Hi All, >> > >> > Please review my fix related to java doc. >> > >> > MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods- >> > >> > MethodHandles.Lookup.findSpecial() throws IllegalAccessException if the target method is static. >> > >> > Bug link-https://bugs.openjdk.java.net/browse/JDK-8138578 >> > Webrev link-http://cr.openjdk.java.net/~srastogi/8138578/webrev.00/ >> > >> > Thnaks, >> > Shilpi >> > >> > >> >> >> > From fguillaume at nuxeo.com Thu Feb 4 13:29:14 2016 From: fguillaume at nuxeo.com (Florent Guillaume) Date: Thu, 4 Feb 2016 14:29:14 +0100 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings In-Reply-To: <56B34C66.9060409@oracle.com> References: <56B2461A.1030901@oracle.com> <56B3375C.1000207@oracle.com> <56B33C11.2000907@oracle.com> <56B33FBC.8080507@oracle.com> <56B34C66.9060409@oracle.com> Message-ID: Hi, Looking at the webrev it seems that the Javadoc for Long.formatUnsignedLong has not been removed. Regards, Florent On Thu, Feb 4, 2016 at 2:04 PM, Aleksey Shipilev < aleksey.shipilev at oracle.com> wrote: > Thanks guys, pushed. > Now, on to StringJoiner... > > -Aleksey > > On 02/04/2016 03:10 PM, Ivan Gerasimov wrote: > > Looks even better, thanks! > > > > Sincerely yours, > > Ivan > > > > On 04.02.2016 14:54, Aleksey Shipilev wrote: > >> Sure, here's a new webrev: > >> http://cr.openjdk.java.net/~shade/8148936/webrev.02/ > > > -- Florent Guillaume, Director of R&D, Nuxeo Open Source, Java EE based, Enterprise Content Management (ECM) http://www.nuxeo.com http://www.nuxeo.org +33 1 40 33 79 87 From paul.sandoz at oracle.com Thu Feb 4 14:24:26 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 4 Feb 2016 14:24:26 +0000 Subject: RFR: 8148928: java/util/stream/test/**/SequentialOpTest.java timed out intermittently In-Reply-To: <56B2C313.6020402@oracle.com> References: <56B2C313.6020402@oracle.com> Message-ID: > On 4 Feb 2016, at 03:18, Hamlin Li wrote: > > Hi everyone, > > Would you please help to review the fix for bug https://bugs.openjdk.java.net/browse/JDK-8148928, java/util/stream/test/**/SequentialOpTest.java timed out intermittently. > webrev: http://cr.openjdk.java.net/~mli/8148928/webrev.00/ > +1 Paul. From aleksey.shipilev at oracle.com Thu Feb 4 14:50:16 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 4 Feb 2016 17:50:16 +0300 Subject: RFR (S) 8148936: Adapt UUID.toString() to Compact Strings In-Reply-To: References: <56B2461A.1030901@oracle.com> <56B3375C.1000207@oracle.com> <56B33C11.2000907@oracle.com> <56B33FBC.8080507@oracle.com> <56B34C66.9060409@oracle.com> Message-ID: <56B36528.1000900@oracle.com> On 02/04/2016 04:29 PM, Florent Guillaume wrote: > Looking at the webrev it seems that the Javadoc for > Long.formatUnsignedLong has not been removed. There are two following methods that are related to that Javadoc. But I agree, these should be cleaned up along with other *Latin1/*UTF16 methods in Long class. Any takers? Cheers, -Aleksey From amaembo at gmail.com Thu Feb 4 15:55:27 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Thu, 4 Feb 2016 21:55:27 +0600 Subject: RFR-8148748: ArrayList.subList().spliterator() is not late-binding In-Reply-To: References: <27476765.20160129103224@gmail.com> <564410418.20160202102813@gmail.com> <1084445298.20160203182024@gmail.com> Message-ID: <534936148.20160204215527@gmail.com> Hello! Thank you for clarification. As far as I know currently JIT-compiled null checks are almost always implicit (via trapped page-fault). Well, ok, I'm not a Hotspot expert (probably for C1 this still matters). I will keep this code as is. Webrev was not changed: http://cr.openjdk.java.net/~tvaleev/webrev/8148748/r1/ Any more reviews, please? With best regards, Tagir Valeev. MB> On Wed, Feb 3, 2016 at 4:20 AM, Tagir F. Valeev wrote: >> Some more thoughts about forEachRemaining: >> >> To me it seems that >> if ((a = lst.elementData) != null) >> is a redundant check as well as in current ArrayList implementation >> elementData is never null. So it can be replaced with simple >> assignment. MB> The null check for something that is provably non-null is a sign of MB> the hand of Doug Lea. MB> I'm often tempted to try to remove them from j.u.concurrent as well, MB> but I resist. MB> They may help hotspot generate better code. From aleksey.shipilev at oracle.com Thu Feb 4 18:29:14 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 4 Feb 2016 21:29:14 +0300 Subject: RFR (XS) (urgent) 8149044: jdk/internal/misc/JavaLangAccess/FormatUnsigned.java fails all platforms Message-ID: <56B3987A.9090009@oracle.com> Hi, jdk/internal/misc/JavaLangAccess/FormatUnsigned.java started failing after UUID.toString() optimizations purged the methods from JavaLangAccess: https://bugs.openjdk.java.net/browse/JDK-8149044 My bad, I should have ran JavaLangAccess tests after the original change. The fix is to remove the test, because all tested methods are gone now: http://cr.openjdk.java.net/~shade/8149044/webrev.01/ Cheers, -Aleksey From joe.darcy at oracle.com Thu Feb 4 18:32:31 2016 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 4 Feb 2016 10:32:31 -0800 Subject: RFR (XS) (urgent) 8149044: jdk/internal/misc/JavaLangAccess/FormatUnsigned.java fails all platforms In-Reply-To: <56B3987A.9090009@oracle.com> References: <56B3987A.9090009@oracle.com> Message-ID: <56B3993F.4030203@oracle.com> HI Aleksey, Looks fine. I assume any remaining JavaLangAccess have other tests :-) Thanks, -Joe On 2/4/2016 10:29 AM, Aleksey Shipilev wrote: > Hi, > > jdk/internal/misc/JavaLangAccess/FormatUnsigned.java started failing > after UUID.toString() optimizations purged the methods from JavaLangAccess: > https://bugs.openjdk.java.net/browse/JDK-8149044 > > My bad, I should have ran JavaLangAccess tests after the original > change. The fix is to remove the test, because all tested methods are > gone now: > http://cr.openjdk.java.net/~shade/8149044/webrev.01/ > > Cheers, > -Aleksey > From Roger.Riggs at Oracle.com Thu Feb 4 18:34:28 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 4 Feb 2016 13:34:28 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B2801A.7060508@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> Message-ID: <56B399B4.6020900@Oracle.com> Hi Stuart, On 2/3/2016 5:32 PM, Stuart Marks wrote: > > > On 2/2/16 7:05 PM, David Holmes wrote: >> On 3/02/2016 8:08 AM, Stuart Marks wrote: >>> It will be good to get this into the JDK. Lots of people have been >>> asking for this. >> >> I think this API is a big mistake. The primary usecase seems to be >> control-C >> interception for utilities like jshell. Adding a general purpose >> signal raising >> and handling mechanism to the JDK does not seem like a good solution >> to me. >> While you would need to use signal management under the covers I >> think it would >> be much cleaner to expose an API that actually captures what it is >> you want >> here: a mechanism to manage "interrupt" and "terminate" events at the >> VM level, >> in a clean cross-platform way. > > OK, I've looked some at the implementation, and there's more going on > here than meets the eye. > > I was under the impression (or at least I was hoping) that the API > would expose a carefully curated set of signals that are (a) known to > be useful to applications, and (b) are safe for the JVM to allow > applications to handle. Examples of this would include SIGWINCH and > SIGTSTP, which are common for Unix applications to want to handle, as > well as the Control-C (SIGINT) case that jshell among others want to > handle. The initial task was to provide a replacement for sun.misc.Signal. Sun.misc.Signal provides to Java the same set of signals the VM is able to handle and exposes via the JVM_findSignal, JVM_RegisterSignal, and JVM_RaiseSignal. The VM carefully handles the native state and delivers a Java safe notification that the signal occurred. > > But I tried out the patch and looked through the Hotspot signal > handling code, and the set of signals exposed is much broader than I > would have expected. On Mac OS X, the signals for which a handler can > be registered include the following: > > SIGABRT > SIGALRM > SIGBUS > SIGCHLD > SIGCONT > SIGEMT > SIGHUP > SIGINT > SIGIO > SIGPIPE > SIGPROF > SIGSYS > SIGTERM > SIGTRAP > SIGTSTP > SIGTTIN > SIGTTOU > SIGURG > SIGUSR1 > SIGUSR2 > SIGVTALRM > SIGWINCH > SIGXCPU > SIGXFSZ > > I'm quite surprised by this. It seems quite unwise to expose all of > these. Perhaps this is what David is concerned about. If so, I'm > starting to share his concern. Java is used as an application programming language and environment and as a system programming environment. Java applications are intended to be competitive with other programming environments, of which most support handling and raising of signals. The native command shells, python, etc all support signals to control and manage applications. Each supports use of signals within and between applications. We have encouraged the use of Java for similar programming, process and control functions. Signals are a well known and used mechanism to communicate between the OS and applications and between applications. Some signal use is restricted because the Java runtime uses them (SIGSEGV, SIGFPE, SIGILL, etc.) of them the application is restricted from using. The others are handled uniformly to notify the client and to be able to raise them. Signals are a valuable communication mechanism and I don't think the Java runtime should arbitrarily restrict signal use based on an unquantified risk. > > In addition, the signals for which a Signal instance can be gotten via > Signal.of(), but which cannot be handled (throws UOE), include: > > SIGFPE > SIGILL > SIGKILL > SIGQUIT > SIGSEGV > SIGSTOP The first 5, I'd understand can't be handled because the VM treats them specially. I'll need to dig into the VM differences to see why it is the way it is. > > It's very strange to expose Signal instances representing these > signals when they can't be handled. They can't be raised either (at > least in Roger's first patch) since raising a signal is prohibited if > there's no handler installed. Thanks, Roger > > I still think a signal-handling API, even a system-specific one, can > be useful for a well curated set of signals. But this implementation > seems to bring an internal interface directly out to the API. That > doesn't seem like the right approach. > > s'marks From aleksey.shipilev at oracle.com Thu Feb 4 18:46:00 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Thu, 4 Feb 2016 21:46:00 +0300 Subject: RFR (XS) (urgent) 8149044: jdk/internal/misc/JavaLangAccess/FormatUnsigned.java fails all platforms In-Reply-To: <56B3993F.4030203@oracle.com> References: <56B3987A.9090009@oracle.com> <56B3993F.4030203@oracle.com> Message-ID: <56B39C68.4030202@oracle.com> Thanks for a quick review, Joe! Pushed. -Aleksey On 02/04/2016 09:32 PM, joe darcy wrote: > HI Aleksey, > > Looks fine. I assume any remaining JavaLangAccess have other tests :-) > > Thanks, > > -Joe > > On 2/4/2016 10:29 AM, Aleksey Shipilev wrote: >> Hi, >> >> jdk/internal/misc/JavaLangAccess/FormatUnsigned.java started failing >> after UUID.toString() optimizations purged the methods from >> JavaLangAccess: >> https://bugs.openjdk.java.net/browse/JDK-8149044 >> >> My bad, I should have ran JavaLangAccess tests after the original >> change. The fix is to remove the test, because all tested methods are >> gone now: >> http://cr.openjdk.java.net/~shade/8149044/webrev.01/ >> >> Cheers, >> -Aleksey >> > From mandy.chung at oracle.com Thu Feb 4 20:38:37 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Feb 2016 12:38:37 -0800 Subject: RFR: 8072777: java/lang/ref/ReferenceEnqueuePending.java: some references aren't queued In-Reply-To: References: <9C4B1F92-29F6-4977-BB48-342ADF886B56@oracle.com> <56ADD196.9040809@gmail.com> Message-ID: > On Feb 1, 2016, at 4:05 PM, Kim Barrett wrote: > >> But this could be an opportunity to also clean-up the code a bit. The checkResult method has an 'obj' parameter whose purpose is not immediately obvious. [?] >> >> I don't know why such complications are necessary. Is the test supposed to also verify the requirement that a Reference whose referent is kept strongly reachable will not be enqueued? > > I was wondering about the odd stuff around obj as well. I don't think > it is an attempt to "also verify the requirement that a Reference > whose referent is kept strongly reachable will not be enqueued". > Rather, I think it is a kludgy way to avoid having the final weaky be > sometimes enqueued and sometimes not, depending on compiler > optimizations. > What you described matches what I understand. > Using Reference.reachabilityFence to keep obj and weaky live across > the checkResult as you suggested is one way to accomplish that, though > keeping only the final obj alive (as in the existing code) suffices to > keep the final weaky from being notified. I think reachabilityFence > and a better comment would be clearer though, so changing accordingly. > Glad to see the old way keeping an object being GC?ed or optimized out replaced. > New webrevs: > full: http://cr.openjdk.java.net/~kbarrett/8072777/webrev.01/ > incr: http://cr.openjdk.java.net/~kbarrett/8072777/webrev.01.inc/ Looks good. Mandy From hboehm at google.com Thu Feb 4 22:37:35 2016 From: hboehm at google.com (Hans Boehm) Date: Thu, 4 Feb 2016 14:37:35 -0800 Subject: JDK-8052260 (Reference.isEnqueued()) Discussion Message-ID: The discussion at https://bugs.openjdk.java.net/browse/JDK-8052260 correctly points out that the spec implies that this should be true if the reference has ever been enqueued, while the implementation returns false once it has been removed from the queue. The current proposal is to change the spec. I'd like to make a different proposal, based on the following observations: 1) I don't know of any way to write correct code that is oblivious to this difference. 2) AFAIK, this has been broken in this way for a long time, possibly from the beginning, i.e. for more than 15 years(?). The bug is dated 2014. Apparently nobody noticed for a long time. 3) I don't know of any useful correct code that uses the function as implemented. The implemented and proposed semantics seem to be inherently racey, at least if the associated queue is processed asynchronously. Martin Buchholz suggested debugging statistics as a possible use. That seems to be the best answer to date, but it still seems to me that a heap profile would generally serve better in such cases. 4) I've looked at as many clients as I could find. The number is in the single digits. I don't believe any of them are correct with the current implementation. The majority misuses isEnqueued() on References without an associated queue as a test to see if the reference has been cleared. In one case this introduced a potentially serious bug; in other cases it just slowed down the code. My conclusion is that this call currently only serves to inject bugs and slowdowns into code written by unwary users. I suggest deprecating it. From kim.barrett at oracle.com Thu Feb 4 22:58:02 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Thu, 4 Feb 2016 17:58:02 -0500 Subject: RFR: 8072777: java/lang/ref/ReferenceEnqueuePending.java: some references aren't queued In-Reply-To: References: <9C4B1F92-29F6-4977-BB48-342ADF886B56@oracle.com> <56ADD196.9040809@gmail.com> Message-ID: <447D3FCD-F726-47C2-BC28-548D6E7A91DB@oracle.com> > On Feb 4, 2016, at 3:38 PM, Mandy Chung wrote: > > >> On Feb 1, 2016, at 4:05 PM, Kim Barrett wrote: >> >>> But this could be an opportunity to also clean-up the code a bit. The checkResult method has an 'obj' parameter whose purpose is not immediately obvious. [?] >>> >>> I don't know why such complications are necessary. Is the test supposed to also verify the requirement that a Reference whose referent is kept strongly reachable will not be enqueued? >> >> I was wondering about the odd stuff around obj as well. I don't think >> it is an attempt to "also verify the requirement that a Reference >> whose referent is kept strongly reachable will not be enqueued". >> Rather, I think it is a kludgy way to avoid having the final weaky be >> sometimes enqueued and sometimes not, depending on compiler >> optimizations. >> > > What you described matches what I understand. > >> Using Reference.reachabilityFence to keep obj and weaky live across >> the checkResult as you suggested is one way to accomplish that, though >> keeping only the final obj alive (as in the existing code) suffices to >> keep the final weaky from being notified. I think reachabilityFence >> and a better comment would be clearer though, so changing accordingly. >> > > Glad to see the old way keeping an object being GC?ed or optimized out replaced. > >> New webrevs: >> full: http://cr.openjdk.java.net/~kbarrett/8072777/webrev.01/ >> incr: http://cr.openjdk.java.net/~kbarrett/8072777/webrev.01.inc/ > > > Looks good. > Mandy Thanks! From mikael.vidstedt at oracle.com Fri Feb 5 00:10:18 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Thu, 4 Feb 2016 16:10:18 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B3428A.3010704@redhat.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <56B3428A.3010704@redhat.com> Message-ID: <56B3E86A.40206@oracle.com> On 2016-02-04 04:22, Andrew Haley wrote: > On 02/02/2016 07:25 PM, Mikael Vidstedt wrote: >> Please review this change which introduces a Copy::conjoint_swap and an >> Unsafe.copySwapMemory method to call it from Java, along with the >> necessary changes to have java.nio.Bits call it instead of the Bits.c code. >> >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ > One other little thing: why are the byte-swapping methods in class nio.Bits > not called copySwapSomething? e.g.: That sure would be a better name, wouldn't it? I'm not going to be changing the Bits method names as part of this change, but it does seem like a very reasonable follow-up enhancement. Cheers, Mikael From mandy.chung at oracle.com Fri Feb 5 00:52:27 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Feb 2016 16:52:27 -0800 Subject: JDK-8052260 (Reference.isEnqueued()) Discussion In-Reply-To: References: Message-ID: <45540243-741D-4F5E-B818-A5B7E1D85195@oracle.com> > On Feb 4, 2016, at 2:37 PM, Hans Boehm wrote: > > The discussion at https://bugs.openjdk.java.net/browse/JDK-8052260 > correctly points out that the spec implies that this should be true if the > reference has ever been enqueued, while the implementation returns false > once it has been removed from the queue. The current proposal is to change > the spec. The spec needs clarification (not a spec change and no behavioral change). > I'd like to make a different proposal, based on the following > observations: > > 1) I don't know of any way to write correct code that is oblivious to this > difference. > 2) AFAIK, this has been broken in this way for a long time, possibly from > the beginning, i.e. for more than 15 years(?). The bug is dated 2014. > Apparently nobody noticed for a long time. AFAIK nothing is broken. > 3) I don't know of any useful correct code that uses the function as > implemented. The implemented and proposed semantics seem to be inherently > racey, at least if the associated queue is processed asynchronously. > Martin Buchholz suggested debugging statistics as a possible use. That > seems to be the best answer to date, but it still seems to me that a heap > profile would generally serve better in such cases. > 4) I've looked at as many clients as I could find. The number is in the > single digits. I don't believe any of them are correct with the current > implementation. The majority misuses isEnqueued() on References without an > associated queue as a test to see if the reference has been cleared. Ah, I see. Hmm.. should code review catch this? The spec is clear on this point though: If this reference object was not registered with a queue when it was created, then this method will always return false. > In one case this introduced a potentially serious bug; in other cases it just > slowed down the code. > > My conclusion is that this call currently only serves to inject bugs and > slowdowns into code written by unwary users. I suggest deprecating it. Fair point while deprecation may not be the right answer to address ?mis-reading? of the spec. On the other hand, this method does not seem to be useful. I have included your comment in the JBS issue to consider. Mandy From david.holmes at oracle.com Fri Feb 5 01:05:54 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 5 Feb 2016 11:05:54 +1000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B399B4.6020900@Oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> Message-ID: <56B3F572.3020502@oracle.com> On 5/02/2016 4:34 AM, Roger Riggs wrote: > Hi Stuart, > > On 2/3/2016 5:32 PM, Stuart Marks wrote: >> >> >> On 2/2/16 7:05 PM, David Holmes wrote: >>> On 3/02/2016 8:08 AM, Stuart Marks wrote: >>>> It will be good to get this into the JDK. Lots of people have been >>>> asking for this. >>> >>> I think this API is a big mistake. The primary usecase seems to be >>> control-C >>> interception for utilities like jshell. Adding a general purpose >>> signal raising >>> and handling mechanism to the JDK does not seem like a good solution >>> to me. >>> While you would need to use signal management under the covers I >>> think it would >>> be much cleaner to expose an API that actually captures what it is >>> you want >>> here: a mechanism to manage "interrupt" and "terminate" events at the >>> VM level, >>> in a clean cross-platform way. >> >> OK, I've looked some at the implementation, and there's more going on >> here than meets the eye. >> >> I was under the impression (or at least I was hoping) that the API >> would expose a carefully curated set of signals that are (a) known to >> be useful to applications, and (b) are safe for the JVM to allow >> applications to handle. Examples of this would include SIGWINCH and >> SIGTSTP, which are common for Unix applications to want to handle, as >> well as the Control-C (SIGINT) case that jshell among others want to >> handle. > The initial task was to provide a replacement for sun.misc.Signal. > Sun.misc.Signal provides to Java the same set of > signals the VM is able to handle and exposes via the JVM_findSignal, > JVM_RegisterSignal, and JVM_RaiseSignal. > The VM carefully handles the native state and delivers a Java safe > notification that the signal occurred. And sun.misc.Signal was a private internal API not intended for public use. Simply elevating that API to full fledged public , supported, core API is just wrong IMHO. >> >> But I tried out the patch and looked through the Hotspot signal >> handling code, and the set of signals exposed is much broader than I >> would have expected. On Mac OS X, the signals for which a handler can >> be registered include the following: >> >> SIGABRT >> SIGALRM >> SIGBUS >> SIGCHLD >> SIGCONT >> SIGEMT >> SIGHUP >> SIGINT >> SIGIO >> SIGPIPE >> SIGPROF >> SIGSYS >> SIGTERM >> SIGTRAP >> SIGTSTP >> SIGTTIN >> SIGTTOU >> SIGURG >> SIGUSR1 >> SIGUSR2 >> SIGVTALRM >> SIGWINCH >> SIGXCPU >> SIGXFSZ >> >> I'm quite surprised by this. It seems quite unwise to expose all of >> these. Perhaps this is what David is concerned about. If so, I'm >> starting to share his concern. It makes absolutely no sense to try and install handlers for the majority of those signals at the Java level. > Java is used as an application programming language and environment and > as a system programming environment. Please clarify that last statement. I would not, as I understand them, classify Java as a "systems programming" language. You need to drop into native code to do most "systems level" stuff in Java - and indeed you can do signal management there as well using the libjsig signal interposition library. Java applications are intended to be competitive with > other programming environments, of which > most support handling and raising of signals. The native command > shells, python, etc all support signals > to control and manage applications. Each supports use of signals within > and between applications. > We have encouraged the use of Java for similar programming, process and > control functions. > > Signals are a well known and used mechanism to communicate between the > OS and applications > and between applications. Some signal use is restricted because the > Java runtime > uses them (SIGSEGV, SIGFPE, SIGILL, etc.) > of them the application is restricted from using. The others are > handled uniformly to notify the client > and to be able to raise them. > > Signals are a valuable communication mechanism and I don't think the > Java runtime should > arbitrarily restrict signal use based on an unquantified risk. I think both parts of this are an overstatement. Java hasn't needed a public signal handling API for the last 20 years. It's only getting one now because modularity is forcing us to cut-off sun.misc.Signal from the few use-cases that do need it. And we now have a "shell" mechanism that also wants to do some process management. This is fundamentally about ctrl-C and ctrl-\ hooks because the native shells turn those into signals. >> >> In addition, the signals for which a Signal instance can be gotten via >> Signal.of(), but which cannot be handled (throws UOE), include: >> >> SIGFPE >> SIGILL >> SIGKILL >> SIGQUIT >> SIGSEGV >> SIGSTOP > The first 5, I'd understand can't be handled because the VM treats them > specially. > > I'll need to dig into the VM differences to see why it is the way it is. SIGSTOP can not be caught or ignored, hence no handler allowed. David ----- >> >> It's very strange to expose Signal instances representing these >> signals when they can't be handled. They can't be raised either (at >> least in Roger's first patch) since raising a signal is prohibited if >> there's no handler installed. > > Thanks, Roger > >> >> I still think a signal-handling API, even a system-specific one, can >> be useful for a well curated set of signals. But this implementation >> seems to bring an internal interface directly out to the API. That >> doesn't seem like the right approach. >> >> s'marks > From hboehm at google.com Fri Feb 5 01:17:52 2016 From: hboehm at google.com (Hans Boehm) Date: Thu, 4 Feb 2016 17:17:52 -0800 Subject: JDK-8052260 (Reference.isEnqueued()) Discussion In-Reply-To: <45540243-741D-4F5E-B818-A5B7E1D85195@oracle.com> References: <45540243-741D-4F5E-B818-A5B7E1D85195@oracle.com> Message-ID: Thanks! I thought the current spec fairly clearly states something other than what the implementation does: "Tells whether or not this reference object has been enqueued ..." I do agree that the name is inconsistent with that. If I didn't know better, I would still believe the sentence in the spec, since it gives you a stable "true" result, as opposed to an answer that could always be falsified by the time the method returns. I agree that code review SHOULD catch such bugs. It's possible that it caught most of them and 99.9% of code that used to use isEnqueued() no longer uses isEnqueued() as a result. But I'm not that much of an optimist. On Thu, Feb 4, 2016 at 4:52 PM, Mandy Chung wrote: > > > On Feb 4, 2016, at 2:37 PM, Hans Boehm wrote: > > > > The discussion at https://bugs.openjdk.java.net/browse/JDK-8052260 > > correctly points out that the spec implies that this should be true if > the > > reference has ever been enqueued, while the implementation returns false > > once it has been removed from the queue. The current proposal is to > change > > the spec. > > The spec needs clarification (not a spec change and no behavioral change). > > > I'd like to make a different proposal, based on the following > > observations: > > > > 1) I don't know of any way to write correct code that is oblivious to > this > > difference. > > 2) AFAIK, this has been broken in this way for a long time, possibly from > > the beginning, i.e. for more than 15 years(?). The bug is dated 2014. > > Apparently nobody noticed for a long time. > > AFAIK nothing is broken. > > > 3) I don't know of any useful correct code that uses the function as > > implemented. The implemented and proposed semantics seem to be > inherently > > racey, at least if the associated queue is processed asynchronously. > > Martin Buchholz suggested debugging statistics as a possible use. That > > seems to be the best answer to date, but it still seems to me that a heap > > profile would generally serve better in such cases. > > 4) I've looked at as many clients as I could find. The number is in the > > single digits. I don't believe any of them are correct with the current > > implementation. The majority misuses isEnqueued() on References without > an > > associated queue as a test to see if the reference has been cleared. > > Ah, I see. Hmm.. should code review catch this? > > The spec is clear on this point though: > > If this reference object was not registered with a queue when it was > created, then this method will always return false. > > > In one case this introduced a potentially serious bug; in other cases it > just > > slowed down the code. > > > > My conclusion is that this call currently only serves to inject bugs and > > slowdowns into code written by unwary users. I suggest deprecating it. > > Fair point while deprecation may not be the right answer to address > ?mis-reading? of the spec. On the other hand, this method does not seem > to be useful. I have included your comment in the JBS issue to consider. > > Mandy From stuart.marks at oracle.com Fri Feb 5 02:00:18 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Thu, 4 Feb 2016 18:00:18 -0800 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B3F572.3020502@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> Message-ID: <56B40232.20906@oracle.com> On 2/4/16 5:05 PM, David Holmes wrote: >> The initial task was to provide a replacement for sun.misc.Signal. >> Sun.misc.Signal provides to Java the same set of >> signals the VM is able to handle and exposes via the JVM_findSignal, >> JVM_RegisterSignal, and JVM_RaiseSignal. >> The VM carefully handles the native state and delivers a Java safe >> notification that the signal occurred. > > And sun.misc.Signal was a private internal API not intended for public use. > Simply elevating that API to full fledged public , supported, core API is just > wrong IMHO. I agree with David on this. Providing signals to Java bytecodes for JDK-internal use is one thing; exporting this as a public API for general use is altogether another. Just because Hotspot doesn't need a signal doesn't mean that the signal should be usable by any Java application. JDK internal users such as library code (whether native or Java) need to handle signals for internal implementation purposes. I see some evidence of this in Process, net, and nio. We need to make sure that an application's use of signals doesn't interfere with the library implementation's. s'marks From bitterfoxc at gmail.com Fri Feb 5 03:44:46 2016 From: bitterfoxc at gmail.com (ShinyaYoshida) Date: Fri, 5 Feb 2016 12:44:46 +0900 Subject: RFR:JDK-8146747:java.time.Duration.toNanos() and toMillis() exception on negative durations In-Reply-To: <56B2E143.7050308@oracle.com> References: <56AEF8B8.4070703@oracle.com> <56B1F008.5030800@oracle.com> <56B2E143.7050308@oracle.com> Message-ID: Hi, There is no problem. Thank you for the fix! shinyafox(Shinya Yoshida) 2016/02/04 14:28 "nadeesh tv" : > > Hi all, > > Please see the updated webrev > http://cr.openjdk.java.net/~ntv/8146747/webrev.01/ > Regards, > Nadeesh > On 2/3/2016 5:48 PM, nadeesh tv wrote: > >> Hi Shinya, >> Thnx. I will update it. >> Regards, >> Nadeesh >> On 2/3/2016 5:41 PM, ShinyaYoshida wrote: >> >>> Hi Nadeesh, >>> Almost LGTM!(But I'm not a reviewer;) ) >>> However I've noticed that you don't use NANOS_PER_SECOND at L1223 and >>> L1246. >>> Is there some reason not to use it? >>> >>> Regards, >>> shinyafox(Shinya Yoshida) >>> >>> 2016-02-01 15:18 GMT+09:00 nadeesh tv >> nadeesh.tv at oracle.com>>: >>> >>> Hi all, >>> >>> Please review following >>> >>> Bug Id : https://bugs.openjdk.java.net/browse/JDK-8146747 >>> >>> >>> Solution: Handled the negative duration separately >>> >>> webrev : http://cr.openjdk.java.net/~ntv/8146747/webrev.00/ >>> >>> >>> >>> -- Thanks and Regards, >>> Nadeesh TV >>> >>> >>> >> > -- > Thanks and Regards, > Nadeesh TV > > From mandy.chung at oracle.com Fri Feb 5 04:24:05 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 4 Feb 2016 20:24:05 -0800 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B3F572.3020502@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> Message-ID: > On Feb 4, 2016, at 5:05 PM, David Holmes wrote: > > Java hasn't needed a public signal handling API for the last 20 years. It's only getting one now because modularity is forcing us to cut-off sun.misc.Signal from the few use-cases that do need it. And we now have a "shell" mechanism that also wants to do some process management. This is fundamentally about ctrl-C and ctrl-\ hooks because the native shells turn those into signals. As a data point, the primary use case I gathered and reported by some customers using sun.misc.Signal is the ability to intercept control-C. SIGTERM, SIGKILL, SIGTRAP are also intercepted in addition to SIGINT [1] Mandy [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-June/004362.html From thomas.stuefe at gmail.com Fri Feb 5 09:08:47 2016 From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=) Date: Fri, 5 Feb 2016 10:08:47 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B40232.20906@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> <56B40232.20906@oracle.com> Message-ID: On Fri, Feb 5, 2016 at 3:00 AM, Stuart Marks wrote: > > > On 2/4/16 5:05 PM, David Holmes wrote: > >> The initial task was to provide a replacement for sun.misc.Signal. >>> Sun.misc.Signal provides to Java the same set of >>> signals the VM is able to handle and exposes via the JVM_findSignal, >>> JVM_RegisterSignal, and JVM_RaiseSignal. >>> The VM carefully handles the native state and delivers a Java safe >>> notification that the signal occurred. >>> >> >> And sun.misc.Signal was a private internal API not intended for public >> use. >> Simply elevating that API to full fledged public , supported, core API is >> just >> wrong IMHO. >> > > I agree with David on this. Providing signals to Java bytecodes for > JDK-internal use is one thing; exporting this as a public API for general > use is altogether another. > > Just because Hotspot doesn't need a signal doesn't mean that the signal > should be usable by any Java application. JDK internal users such as > library code (whether native or Java) need to handle signals for internal > implementation purposes. I see some evidence of this in Process, net, and > nio. We need to make sure that an application's use of signals doesn't > interfere with the library implementation's. > > s'marks > Would that not be a case for a new JEP? Introducing an official signal handling API feels to big to me for just a normal change. Having a JEP would give a good forum for discussions. That said, I can see both sides. I am not happy with exposing a generic Signal handling API, but I feel that I if this is really needed, we should very carefully select the signals we allow the user to manipulate, and clearly document this list and stick to it. Any signal in this list is a compatibility promise to the user and we cannot use this signal for ourselves in future releases. In my opinion we should then aim for compatibility among Posix platforms, e.g. a java program written on Linux x86 should work seamlessly on MacOs or on Linux with different architecture. So, we should leave out all signals which are either not supported universally or behave differently on different platforms. We also should exclude all signals out the VM uses for itself or may use for itself in the future (e.g. in the face of a new port to a new architecture). So, as examples, we should exclude from this list Posix realtime signals (platform dependend and do not exist on MacOS), SIGTRAP (used on PowerPC and Arch64 (?) by the VM), exotics like SIGDANGER or SIGCPUFAIL (aix only) etc. Also, of course, all error signals like SIGSEGV, SIGILL etc. Signals I think could be useful to expose would be anything needed to write UNIX terminal applications, like SIGTTIN/SIGTTOUT or SIGTSTP. Kind Regards, Thomas From paul.sandoz at oracle.com Fri Feb 5 13:00:42 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 5 Feb 2016 13:00:42 +0000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> Message-ID: <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> Hi, Nice use of C++ templates :-) Overall looks good. I too would prefer if we could move the argument checking out, perhaps even to the point of requiring callers do that rather than providing another method, for example for Buffer i think the arguments are known to be valid? I think in either case it is important to improve the documentation on the method stating the constraints on arguments, atomicity guarantees etc. I have a hunch that for the particular case of copying-with-swap for buffers i could get this to work work efficiently using Unsafe (three separate methods for each unit type of 2, 4 and 8 bytes), since IIUC the range is bounded to be less than Integer.MAX_VALUE so an int loop rather than a long loop can be used and therefore safe points checks will not be placed within the loop. However, i think what you have done is more generally applicable and could be made intrinsic. It would be a nice at some future point if it could be made a pure Java implementation and intrinsified where appropriate. ? John, regarding array mismatch there were issues with the efficiency of the unrolled loops with Unsafe access. (Since the loops were int bases there were no issues with safe point checks.) Roland recently fixed that so now code is generated that is competitive with direct array accesses. We drop into the stub intrinsic and leverage 128bits or 256bits where supported. Interestingly it seems the unrolled loop using Unsafe is now slightly faster than the stub using 128bit registers. I don?t know if that is due to unluckly alignment, and/or the stub needs to do some manual unrolling. In terms of code-cache efficiency the intrinsic is better. Paul. > On 4 Feb 2016, at 06:27, John Rose wrote: > > On Feb 2, 2016, at 11:25 AM, Mikael Vidstedt wrote: >> Please review this change which introduces a Copy::conjoint_swap and an Unsafe.copySwapMemory method to call it from Java, along with the necessary changes to have java.nio.Bits call it instead of the Bits.c code. >> >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ > > This is very good. > > I have some nit-picks: > > These days, when we introduce a new intrinsic (@HSIntrCand), > we write the argument checking code separately in a non-intrinsic > bytecode method. In this case, we don't (yet) have an intrinsic > binding for U.copy*, but we might in the future. (C intrinsifies > memcpy, which is a precedent.) In any case, I would prefer > if we could structure the argument checking code in a similar > way, with Unsafe.java containing both copySwapMemory > and a private copySwapMemory0. Then we can JIT-optimize > the safety checks. > > You might as well extend the same treatment to the pre-existing > copyMemory call. The most important check (and the only one > in U.copyMemory) is to ensure that the size_t operand has not > wrapped around from a Java negative value to a crazy-large > size_t value. That's the low-hanging fruit. Checking the pointers > (for null or oob) is more problematic, of course. Checking consistency > around elemSize is cheap and easy, so I agree that the U.copySM > should do that work also. Basically, Unsafe can do very basic > checks if there is a tricky user model to enforce, but it mustn't > "sign up" to guard the user against all errors. > > Rule of thumb: Unsafe calls don't throw NPEs, they just SEGV. > And the rare bit that *does* throw (IAE usually) should be placed > into Unsafe.java, not unsafe.cpp. (The best-practice rule for putting > argument checking code outside of the intrinsic is a newer one, > so Unsafe code might not always do this.) > > The comment "Generalizing it would be reasonable, but requires > card marking" is bogus, since we never byte-swap managed pointers. > > The test logic will flow a little smoother if your GenericPointer guy, > the onHeap version, stores the appropriate array base offset in his offset field. > You won't have to mention p.isOnHeap nearly so much, and the code will > set a slightly better example. > > The VM_ENTRY_BASE_FROM_LEAF macro is really cool. > > The C++ template code is cool also. It reminds me of the kind > of work Gosling's "Ace" processor could do, but now it's mainstreamed > for all to use in C++. We're going to get some of that goodness > in Project Valhalla with specialization logic. > > I find it amazing that the right way to code this in C is to > use memcpy for unaligned accesses and byte peek/poke > into registers for byte-swapping operators. I'm glad we > can write this code *once* for the JVM and JDK. > > Possible future work: If we can get a better handle on > writing vectorizable loops from Java, including Unsafe-based > ones, we can move some of the C code back up to Java. > Perhaps U.copy* calls for very short lengths deserved to > be broken out into small loops of U.get/put* (with alignment). > I think you experimented with this, and there were problems > with the JIT putting fail-safe memory barriers between > U.get/put* calls. Paul's work on Array.mismatch ran into > similar issues, with the right answer being to write manual > vector code in assembly. > > Anyway, you can count me as a reviewer. > > Thanks, > > ? John From paul.sandoz at oracle.com Fri Feb 5 13:18:43 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 5 Feb 2016 13:18:43 +0000 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <56AB5E6C.1010201@oracle.com> References: <56A21316.1070007@oracle.com> <56AB5E6C.1010201@oracle.com> Message-ID: <5349AA5B-0198-4B25-A42A-E63146CBA266@oracle.com> > On 29 Jan 2016, at 12:43, shilpi rastogi wrote: > > Hi All, > > Please review the updated patch- > > http://cr.openjdk.java.net/~srastogi/8138578/webrev.01/ > > I verified Lookup.unreflectSpecial() also throws java.lang.IllegalAccessException so updated the JavaDoc. > +1 Paul. From paul.sandoz at oracle.com Fri Feb 5 13:29:52 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 5 Feb 2016 13:29:52 +0000 Subject: RFR: 8071368 Use more concrete types for NamedFunction constants in the code In-Reply-To: <56B34F40.1010800@oracle.com> References: <56B34F40.1010800@oracle.com> Message-ID: <6804C2B3-DE2F-4DCF-B70D-6F3BC846B403@oracle.com> Hi Shilpi, In principle this change is ok, however these methods are kind of special because they are called by code generated from LambdaForms with erased types. IIUC the casts can serve as belts-and-braces checks just in case that generated code passes something that is not of the correct type. By removing the casts it?s possible to erroneously pass in something of another type that the runtime treats erroneously as if a MethodHandle or MethodType, and then strange things happen. The casts will ordinary be removed by the runtime compiler. By removing them from the Java code it might slightly improve C2 compilation time. Approval from Vladimir/John would be good. Paul. > On 4 Feb 2016, at 13:16, shilpi rastogi wrote: > > Hi All, > > Please review the following- > > https://bugs.openjdk.java.net/browse/JDK-8071368 > http://cr.openjdk.java.net/~srastogi/8071368/webrev.02/ > > Thanks, > Shilpi From sean.coffey at oracle.com Fri Feb 5 13:37:15 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 5 Feb 2016 13:37:15 +0000 Subject: RFR: JDK-8144144 - ORB destroy() leaks filedescriptors after unsuccessful connection In-Reply-To: <56AABC0C.6090306@oracle.com> References: <568FF6A5.4080700@oracle.com> <569FB2D3.6080607@oracle.com> <569FCF34.8020401@oracle.com> <56A11938.8040804@oracle.com> <56A20F09.2090904@oracle.com> <56AABC0C.6090306@oracle.com> Message-ID: <56B4A58B.8050500@oracle.com> Looks fine to me Mark. Regards, Sean. On 29/01/16 01:10, Mark Sheppard wrote: > Hi, > based on feedback from reviews the current version of this fix can > be found at > > http://cr.openjdk.java.net/~msheppar/8144144/webrev.05/ > > please oblige and review > > regards > Mark > > On 22/01/2016 11:14, Se?n Coffey wrote: >> >> On 21/01/16 17:45, Mark Sheppard wrote: >>> Thanks Sean >>> >>> i can revert the isClosed(), I put them in to make the access >>> consistent >> I'd question if the synchronized isClosed() and setClosed(boolean) >> helper methods are needed at all. The volatile closed boolean >> variable should be enough to manage this state flag. Either way, I >> don't think the synchronization around the closed variable is root >> cause of this issue and performance issues could be addressed in >> another change if necessary (playing safe) >> >> regards, >> Sean. >> >>> >>> regards >>> Mark >>> >>> On 20/01/2016 18:17, Se?n Coffey wrote: >>>> Hi Mark, >>>> >>>> SelectorImpl.java: >>>> >>>> line 125, could you use a 2 arg method call to dprint. It'll print >>>> the stacktrace instead : >>>> dprint(".close: selector.close: " + t); --> dprint(".close: >>>> selector.close", t); >>>> >>>> The "while (!isClosed()) " change introduces a new hot lock on >>>> closed variable. Hopefully, it won't impact performance too much. >>>> >>>> Looks good to me otherwise. >>>> >>>> Regards, >>>> Sean. >>>> >>>> On 20/01/16 16:16, Mark Sheppard wrote: >>>>> Hi, >>>>> an update has been made to the webrev >>>>> >>>>> http://cr.openjdk.java.net/~msheppar/8144144/webrev.03/ >>>>> >>>>> an anomaly was found in the select loop of the SelectorImpl.run() >>>>> method >>>>> some defensive programming, for selector null references, have >>>>> been added, also. >>>>> >>>>> regards >>>>> Mark >>>>> >>>>> On 08/01/2016 17:49, Mark Sheppard wrote: >>>>>> Hi >>>>>> please oblige and review the following changes >>>>>> http://cr.openjdk.java.net/~msheppar/8144144/webrev/ >>>>>> >>>>>> which addresses the issue >>>>>> https://bugs.openjdk.java.net/browse/JDK-8144144 >>>>>> >>>>>> the changes ensure that an Acceptor and its associated >>>>>> ServerSocket/ServerSocketChannel >>>>>> are closed. >>>>>> >>>>>> regards >>>>>> Mark >>>>> >>>> >>> >> > From vladimir.x.ivanov at oracle.com Fri Feb 5 15:48:55 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 5 Feb 2016 18:48:55 +0300 Subject: RFR: 8071368 Use more concrete types for NamedFunction constants in the code In-Reply-To: <6804C2B3-DE2F-4DCF-B70D-6F3BC846B403@oracle.com> References: <56B34F40.1010800@oracle.com> <6804C2B3-DE2F-4DCF-B70D-6F3BC846B403@oracle.com> Message-ID: <56B4C467.7040300@oracle.com> Proposed fix looks good. Quoting John: "The use of erased types (any ref => Object) in the MH runtime is an artifact of bootstrapping difficulties, early in the project. I hope it is not necessary any more." Best regards, Vladimir Ivanov On 2/5/16 4:29 PM, Paul Sandoz wrote: > Hi Shilpi, > > In principle this change is ok, however these methods are kind of special because they are called by code generated from LambdaForms with erased types. > > IIUC the casts can serve as belts-and-braces checks just in case that generated code passes something that is not of the correct type. By removing the casts it?s possible to erroneously pass in something of another type that the runtime treats erroneously as if a MethodHandle or MethodType, and then strange things happen. The casts will ordinary be removed by the runtime compiler. By removing them from the Java code it might slightly improve C2 compilation time. > > Approval from Vladimir/John would be good. > > Paul. > >> On 4 Feb 2016, at 13:16, shilpi rastogi wrote: >> >> Hi All, >> >> Please review the following- >> >> https://bugs.openjdk.java.net/browse/JDK-8071368 >> http://cr.openjdk.java.net/~srastogi/8071368/webrev.02/ >> >> Thanks, >> Shilpi > From vladimir.x.ivanov at oracle.com Fri Feb 5 16:29:52 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 5 Feb 2016 19:29:52 +0300 Subject: [9] RFR (S): 8148994: Replacing MH::invokeBasic with a direct call breaks LF customization In-Reply-To: <6CCE2F2C-1836-47CC-B13E-4371C2CE61FF@oracle.com> References: <56B37387.2040204@oracle.com> <6CCE2F2C-1836-47CC-B13E-4371C2CE61FF@oracle.com> Message-ID: <56B4CE00.3040809@oracle.com> Thanks, John. On 2/4/16 9:52 PM, John Rose wrote: > Looks good. Except I don't recall how is_inline interacts, if at all, with customization. It's not clear what is being gated, and which optimizations get let through. Inlining through MH::invokeBasic doesn't happen when there's @DontInline on the target. It is the case for GWT which has CountingWrapper on both paths. It starts with a reinvoker.dontInline LF and then switches to a reinvoker LF (after 30 invocations). If a compilation happens when reinvoker.dontInline is used, a direct call is generated and even if the branch becomes hot, all the calls goes through reinvoker.dontInline. Update to reinvoker LF isn't visible unless the dispatch goes through MH::invokeBasic linker. I had an impression that I introduced LF customization logic to the non-inlined case in GWT, but I can't find it in the code. So, it seems the regression is caused by repeated checks performed by reinvoker.dontInline LF and not by LF customization missed in generated code. Also, I stumbled upon a quirk in inlining policy: inlining through linkers can't be delayed. It happens when JIT hits some limit on IR size. So, right now inlining can stop in the middle of MH chain and a linker call is generated. But after JDK-8072008 there's no problem with delaying inlining. C2 can decide whether to keep the direct call or inline through it. So, I enabled late inlining for all linkers. (Surprisingly, no significant performance difference on nashorn.) So, I decided to keep invokeBasic linker call replacement for now, but piggypack LF customization on the stuck counting LF. Updated webrev: http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/hotspot http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/jdk * enabled late inlining for linkers; * add LF customization check in CountingWrapper::countDown, so when counting is stuck, the target will be customized. Testing: octane, JPRT. I'm working on JDK-8071793 [1] and will reconsider how LF customization on non-inlined branch in GWTs works given there is a more JVM-friendly mechanism to control inlining. With the proposed fix reinvoker.dontInline is never customized (only the target it points to) and serves as a trampoline between GWT & target LF. It would be better to jump right to the customized LF, if non-inlined branch becomes hot. Best regards, Vladimir Ivanov [1] https://bugs.openjdk.java.net/browse/JDK-8071793 "Prune rarely called invokedynamic call sites during inlining" > > Maybe a comment about that would be useful > Reviewed. > > ? John > >> On Feb 4, 2016, at 7:51 AM, Vladimir Ivanov wrote: >> >> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.00 >> https://bugs.openjdk.java.net/browse/JDK-8148994 >> >> JDK-8072008 [1] introduced an optimization to bypass linker calls in generated code when target method can't be inlined. >> >> Unfortunately, it doesn't work well with MH::invokeBasic(), because it clashes with LambdaForm customization: if a call is not inlined, it is important for performance to update (customize) corresponding LambdaForm on-the-fly. MH::invokeBasic() linker does MH -> LF -> Method* -> _from_compiled_entry traversal on every invocation, while direct call points right to the _from_compiled_entry. So, any LambdaForm updates aren't visible until recompilation. >> >> The fix is to keep MH::invokeBasic linker, so up-to-date LambdaForm instance is used on every invocation. >> >> Also, cleaned up relevant tests a bit. >> >> Testing: octane, JPRT. >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8072008 >> "Emit direct call instead of linkTo* for recursive indy/MH.invoke* calls" From ramanand.patil at oracle.com Fri Feb 5 17:15:56 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Fri, 5 Feb 2016 09:15:56 -0800 (PST) Subject: RFR: JDK-8135259: InetAddress.getAllByName only reports "unknown error" instead of actual cause Message-ID: <48fd26eb-350e-4166-8fe7-6883f8562cd5@default> Hi all Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8135259 Bug Description: Attempts to resolve a host unknown to the DNS server cause an UnknownHostException stating "unknown error" instead of "Name or service not known". Webrev: http://cr.openjdk.java.net/~rpatil/8135259/webrev.00/ Fix: Using a function call directly "gai_strerror(gai_error)" instead of using the function pointer which was declared but not initialized. Apart from this I have removed few other unused variables and function pointers. Brief description of how this bug was introduced is added to the bug comment. Regards, Ramanand. From xueming.shen at oracle.com Fri Feb 5 18:55:45 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 5 Feb 2016 10:55:45 -0800 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib Message-ID: <56B4F031.6040501@oracle.com> Hi Please help codereview the change to build the jdk9 runtime to use the system zlib on Solaris and Linux platforms by default. Issue: https://bugs.openjdk.java.net/browse/JDK-8031767 Webrev: http://cr.openjdk.java.net/~sherman/8031767/webrev/ Background info: Compression is heavily used in Java based big data/middle-ware applications. There are many products in market today that help compression performance either through software or hardware acceleration and most likely these products support the zlib interface as API, for example Intel's IPP library has a faster version of compression libraries. To configure the Java runtime to use the system zlib would make these acceleration capabilities available to java users through java.util.zip package directly. The jdk already has a build configuration option to build the jdk to use the system zlib via "--with-zlib=system" and the OSX is by default built to use the system zlib. This proposal is to propose to build the jdk to use the system zlib library (the zlib bundled by the underlying Solaris/ Linuxplatforms), instead of the binary built from source code jdk repository (current 1.2.8 from the open source zlib.org) Thanks, Sherman btw, attached is the similar change in the closed repo: autoconf/generated-configure.sh ------------------------------------------------------------- # Do not change or remove the following line, it is needed for consistency checks: -DATE_WHEN_GENERATED=1454436146 +DATE_WHEN_GENERATED=1454626552 ############################################################################### # # Initialization / Boot-strapping # ------------------------------------------------------------------------ @@ -58839,14 +58839,14 @@ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for which zlib to use" >&5 $as_echo_n "checking for which zlib to use... " >&6; } - DEFAULT_ZLIB=bundled - if test "x$OPENJDK_TARGET_OS" = xmacosx; then - # On macosx default is system...on others default is bundled DEFAULT_ZLIB=system + if test "x$OPENJDK_TARGET_OS" = xwindows; then + # On windows default is bundled...on others default is system + DEFAULT_ZLIB=bundled fi if test "x${ZLIB_FOUND}" != "xyes"; then # If we don't find any system...set default to bundled DEFAULT_ZLIB=bundled From Alan.Bateman at oracle.com Fri Feb 5 20:50:09 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 5 Feb 2016 20:50:09 +0000 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56B4F031.6040501@oracle.com> References: <56B4F031.6040501@oracle.com> Message-ID: <56B50B01.60003@oracle.com> On 05/02/2016 18:55, Xueming Shen wrote: > Hi > > Please help codereview the change to build the jdk9 runtime to use the > system zlib on > Solaris and Linux platforms by default. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8031767 > Webrev: http://cr.openjdk.java.net/~sherman/8031767/webrev/ I'm happy to see the default changed for Linux and Solaris builds. One thing is that the patch to lib-bundled.m4 switches the build on all non-Windows to use the system zlib. This may be something the SAP (AIX port maintainers) might want to comment on. -Alan. From mikael.vidstedt at oracle.com Fri Feb 5 22:21:47 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Fri, 5 Feb 2016 14:21:47 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> Message-ID: <56B5207B.8010107@oracle.com> I fully agree that moving the arguments checking up to Java makes more sense, and I've prepared new webrevs which do exactly that, including changes to address the other feedback from David, John and others: hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/hotspot/webrev/ jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/jdk/webrev/ Incremental webrevs for your convenience: hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/hotspot/webrev/ jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/jdk/webrev/ I have done some benchmarking of this code and for large copies (16MB+) this outperforms the old Bits.c implementation by *30-100%* depending on platform and exact element sizes! For smaller copies the additional checks which are now performed hurt performance on client VMs (80-90% of old impl), but with the server VMs I see performance on par with, or in most cases 5-10% better than the old implementation. There's a potentially statistically significant regression of ~3-4% for elemSize=2, but for now I'm going to declare success. There's certainly room for further improvements here, but this should at least do for addressing the original problem. I filed https://bugs.openjdk.java.net/browse/JDK-8149159 for moving the checks for Unsafe.copyMemory to Java, and will work on that next. I also filed https://bugs.openjdk.java.net/browse/JDK-8149162 to cover the potential renaming of the Bits methods to have more informative names. Finally, I filed https://bugs.openjdk.java.net/browse/JDK-8149163 to look at improving the behavior of Unsafe.addressSize(), after having spent too much time trying to understand why the performance of the new U.copySwapMemory Java checks wasn't quite living up to my expectations (spoiler alert: Unsafe.addressSize() is not intrinsified, so will always result in a call into the VM/unsafe.cpp). Finally, I - too - would like to see the copy-swap logic moved into Java, and as I mentioned I played around with that first before I decided to do the native implementation to address the immediate problem. Looking forward to what you find Paul! Cheers, Mikael On 2016-02-05 05:00, Paul Sandoz wrote: > Hi, > > Nice use of C++ templates :-) > > Overall looks good. > > I too would prefer if we could move the argument checking out, perhaps even to the point of requiring callers do that rather than providing another method, for example for Buffer i think the arguments are known to be valid? I think in either case it is important to improve the documentation on the method stating the constraints on arguments, atomicity guarantees etc. > > I have a hunch that for the particular case of copying-with-swap for buffers i could get this to work work efficiently using Unsafe (three separate methods for each unit type of 2, 4 and 8 bytes), since IIUC the range is bounded to be less than Integer.MAX_VALUE so an int loop rather than a long loop can be used and therefore safe points checks will not be placed within the loop. > > However, i think what you have done is more generally applicable and could be made intrinsic. It would be a nice at some future point if it could be made a pure Java implementation and intrinsified where appropriate. > > ? > > John, regarding array mismatch there were issues with the efficiency of the unrolled loops with Unsafe access. (Since the loops were int bases there were no issues with safe point checks.) Roland recently fixed that so now code is generated that is competitive with direct array accesses. We drop into the stub intrinsic and leverage 128bits or 256bits where supported. Interestingly it seems the unrolled loop using Unsafe is now slightly faster than the stub using 128bit registers. I don?t know if that is due to unluckly alignment, and/or the stub needs to do some manual unrolling. In terms of code-cache efficiency the intrinsic is better. > > Paul. > > > > > >> On 4 Feb 2016, at 06:27, John Rose wrote: >> >> On Feb 2, 2016, at 11:25 AM, Mikael Vidstedt wrote: >>> Please review this change which introduces a Copy::conjoint_swap and an Unsafe.copySwapMemory method to call it from Java, along with the necessary changes to have java.nio.Bits call it instead of the Bits.c code. >>> >>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ >>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ >> This is very good. >> >> I have some nit-picks: >> >> These days, when we introduce a new intrinsic (@HSIntrCand), >> we write the argument checking code separately in a non-intrinsic >> bytecode method. In this case, we don't (yet) have an intrinsic >> binding for U.copy*, but we might in the future. (C intrinsifies >> memcpy, which is a precedent.) In any case, I would prefer >> if we could structure the argument checking code in a similar >> way, with Unsafe.java containing both copySwapMemory >> and a private copySwapMemory0. Then we can JIT-optimize >> the safety checks. >> >> You might as well extend the same treatment to the pre-existing >> copyMemory call. The most important check (and the only one >> in U.copyMemory) is to ensure that the size_t operand has not >> wrapped around from a Java negative value to a crazy-large >> size_t value. That's the low-hanging fruit. Checking the pointers >> (for null or oob) is more problematic, of course. Checking consistency >> around elemSize is cheap and easy, so I agree that the U.copySM >> should do that work also. Basically, Unsafe can do very basic >> checks if there is a tricky user model to enforce, but it mustn't >> "sign up" to guard the user against all errors. >> >> Rule of thumb: Unsafe calls don't throw NPEs, they just SEGV. >> And the rare bit that *does* throw (IAE usually) should be placed >> into Unsafe.java, not unsafe.cpp. (The best-practice rule for putting >> argument checking code outside of the intrinsic is a newer one, >> so Unsafe code might not always do this.) >> >> The comment "Generalizing it would be reasonable, but requires >> card marking" is bogus, since we never byte-swap managed pointers. >> >> The test logic will flow a little smoother if your GenericPointer guy, >> the onHeap version, stores the appropriate array base offset in his offset field. >> You won't have to mention p.isOnHeap nearly so much, and the code will >> set a slightly better example. >> >> The VM_ENTRY_BASE_FROM_LEAF macro is really cool. >> >> The C++ template code is cool also. It reminds me of the kind >> of work Gosling's "Ace" processor could do, but now it's mainstreamed >> for all to use in C++. We're going to get some of that goodness >> in Project Valhalla with specialization logic. >> >> I find it amazing that the right way to code this in C is to >> use memcpy for unaligned accesses and byte peek/poke >> into registers for byte-swapping operators. I'm glad we >> can write this code *once* for the JVM and JDK. >> >> Possible future work: If we can get a better handle on >> writing vectorizable loops from Java, including Unsafe-based >> ones, we can move some of the C code back up to Java. >> Perhaps U.copy* calls for very short lengths deserved to >> be broken out into small loops of U.get/put* (with alignment). >> I think you experimented with this, and there were problems >> with the JIT putting fail-safe memory barriers between >> U.get/put* calls. Paul's work on Array.mismatch ran into >> similar issues, with the right answer being to write manual >> vector code in assembly. >> >> Anyway, you can count me as a reviewer. >> >> Thanks, >> >> ? John From david.holmes at oracle.com Sat Feb 6 00:54:47 2016 From: david.holmes at oracle.com (David Holmes) Date: Sat, 6 Feb 2016 10:54:47 +1000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> Message-ID: <56B54457.1000900@oracle.com> On 5/02/2016 2:24 PM, Mandy Chung wrote: > >> On Feb 4, 2016, at 5:05 PM, David Holmes wrote: >> >> Java hasn't needed a public signal handling API for the last 20 years. It's only getting one now because modularity is forcing us to cut-off sun.misc.Signal from the few use-cases that do need it. And we now have a "shell" mechanism that also wants to do some process management. This is fundamentally about ctrl-C and ctrl-\ hooks because the native shells turn those into signals. > > As a data point, the primary use case I gathered and reported by some customers using sun.misc.Signal is the ability to intercept control-C. SIGTERM, SIGKILL, SIGTRAP are also intercepted in addition to SIGINT [1] Note the signals SIGKILL and SIGSTOP cannot be caught or ignored. We are really talking about a very few specific, primarily process-management related, signals. Plus the general-purpose SIGUSR1, SIGUSR2. Regardless of whether I agree with this API or not, it does, as Stuart points out, require a JEP and to go through the normal rigorous process of determining whether an API is suitable for inclusion in the Java platform. Cheers, David ----- > Mandy > [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2015-June/004362.html > From peter.firmstone at zeus.net.au Sat Feb 6 11:39:36 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sat, 06 Feb 2016 21:39:36 +1000 Subject: ClassLoader deadlock Message-ID: <56B5DB78.1070502@zeus.net.au> Hmm, thought the new parallel lock stategy in ClassLoader wasn't deadlock prone? Guess I was wrong. Observation: On an unrelated occassion, I had a URLClassLoader synchronization hotspot (well not standard URLClassLoader, but a high performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI instead of URL DNS lookup), I solved that problem by thread confining class loading. I've never experienced deadlock using thread confinement, initially I was worried that there would be interdependencies, however I didn't experience a problem, as the single thread would always load the classes it needed. I'm wondering if multiple locks for ClassLoader's might be the wrong strategy, I found thread confinement was very performant. Regards, Peter. 2016-02-06 21:06:07 Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 runnable [0x00000000] java.lang.Thread.State: RUNNABLE "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 nid=0x111c waiting on condition [0x00000000] java.lang.Thread.State: RUNNABLE "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 waiting on condition [0x00000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 nid=0x10b8 runnable [0x00000000] java.lang.Thread.State: RUNNABLE "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for monitor entry [0x03ded000] java.lang.Thread.State: BLOCKED (on object monitor) at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) at org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) at org.apache.river.concurrent.StrongReference.(StrongReference.java:44) at org.apache.river.concurrent.StrongReference.(StrongReference.java:57) at org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) at org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) at org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) at java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) at java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) at java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) at java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) at java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) at java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) at java.lang.ClassLoader.loadClass(ClassLoader.java:404) at java.lang.ClassLoader.loadClass(ClassLoader.java:411) - locked <0x03f624b8> (a java.lang.Object) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) at org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) at org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) - locked <0x03ef8b30> (a org.cliffc.high_scale_lib.NonBlockingHashMap) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j ava:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 ) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in Object.wait() [0x03cff000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 nid=0x1398 in Object.wait() [0x144cf000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:502) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) - locked <0x03e05200> (a java.lang.ref.Reference$Lock) "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor entry [0x0185e000] java.lang.Thread.State: BLOCKED (on object monitor) at java.lang.ClassLoader.loadClass(ClassLoader.java:406) - waiting to lock <0x03f624b8> (a java.lang.Object) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) at org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) at org.apache.river.concurrent.StrongReference.(StrongReference.java:44) at org.apache.river.concurrent.StrongReference.(StrongReference.java:57) at org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) at org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) at org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) at java.lang.System.checkIO(System.java:253) at java.lang.System.setErr(System.java:199) at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 waiting on condition JNI global references: 19 Found one Java-level deadlock: ============================= "Thread-1": waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), which is held by "main" "main": waiting to lock monitor 0x14274a3c (object 0x03f624b8, a java.lang.Object), which is held by "Thread-1" Java stack information for the threads listed above: =================================================== "Thread-1": at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) at org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) at org.apache.river.concurrent.StrongReference.(StrongReference.java:44) at org.apache.river.concurrent.StrongReference.(StrongReference.java:57) at org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) at org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) at org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) at java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) at java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) at java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) at java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) at java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) at java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) at java.lang.ClassLoader.loadClass(ClassLoader.java:404) at java.lang.ClassLoader.loadClass(ClassLoader.java:411) - locked <0x03f624b8> (a java.lang.Object) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) at org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) at org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) - locked <0x03ef8b30> (a org.cliffc.high_scale_lib.NonBlockingHashMap) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j ava:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 ) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:744) "main": at java.lang.ClassLoader.loadClass(ClassLoader.java:406) - waiting to lock <0x03f624b8> (a java.lang.Object) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) at org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) at org.apache.river.concurrent.StrongReference.(StrongReference.java:44) at org.apache.river.concurrent.StrongReference.(StrongReference.java:57) at org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) at org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) at org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) at java.lang.System.checkIO(System.java:253) at java.lang.System.setErr(System.java:199) at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) Found 1 deadlock. From david.holmes at oracle.com Sat Feb 6 11:57:00 2016 From: david.holmes at oracle.com (David Holmes) Date: Sat, 6 Feb 2016 21:57:00 +1000 Subject: ClassLoader deadlock In-Reply-To: <56B5DB78.1070502@zeus.net.au> References: <56B5DB78.1070502@zeus.net.au> Message-ID: <56B5DF8C.5010201@oracle.com> On 6/02/2016 9:39 PM, Peter Firmstone wrote: > Hmm, thought the new parallel lock stategy in ClassLoader wasn't > deadlock prone? Guess I was wrong. The deadlock is introduced by a custom security manager. SM's play a critical role in many parts of the core libraries so if they utilize synchronization then they can easily introduce deadlock. > Observation: On an unrelated occassion, I had a URLClassLoader > synchronization hotspot (well not standard URLClassLoader, but a high > performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI > instead of URL DNS lookup), I solved that problem by thread confining > class loading. I've never experienced deadlock using thread > confinement, initially I was worried that there would be > interdependencies, however I didn't experience a problem, as the single > thread would always load the classes it needed. > > I'm wondering if multiple locks for ClassLoader's might be the wrong > strategy, I found thread confinement was very performant. Are you suggesting that a class-load request is actually transmitted to a single class-loading thread, while the initial thread blocks until the loading is complete? That would still seem prone to the same SM related deadlock scenario. And it would not generally be particularly performant. David > Regards, > > Peter. > > 2016-02-06 21:06:07 > Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): > > "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 > runnable [0x00000000] > java.lang.Thread.State: RUNNABLE > > "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 > nid=0x111c waiting on condition [0x00000000] > java.lang.Thread.State: RUNNABLE > > "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 > waiting on condition [0x00000000] > java.lang.Thread.State: RUNNABLE > > "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 nid=0x10b8 > runnable [0x00000000] > java.lang.Thread.State: RUNNABLE > > "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for > monitor entry [0x03ded000] > java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > > at > org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:44) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:57) > at > org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) > > at > org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) > at > org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > > at > java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) > at > java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) > > at > java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) > at > java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) > > at > java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) > > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) > at > java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) > > at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) > at java.lang.ClassLoader.loadClass(ClassLoader.java:404) > at java.lang.ClassLoader.loadClass(ClassLoader.java:411) > - locked <0x03f624b8> (a java.lang.Object) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) > > at > org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) > > at > org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) > > - locked <0x03ef8b30> (a > org.cliffc.high_scale_lib.NonBlockingHashMap) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j > > ava:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 > > ) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:744) > > "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in > Object.wait() [0x03cff000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) > - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) > at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) > > "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 > nid=0x1398 in Object.wait() [0x144cf000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) > at java.lang.Object.wait(Object.java:502) > at > java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) > - locked <0x03e05200> (a java.lang.ref.Reference$Lock) > > "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor > entry [0x0185e000] > java.lang.Thread.State: BLOCKED (on object monitor) > at java.lang.ClassLoader.loadClass(ClassLoader.java:406) > - waiting to lock <0x03f624b8> (a java.lang.Object) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > > at > org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:44) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:57) > at > org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) > > at > org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) > at > org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > > at java.lang.System.checkIO(System.java:253) > at java.lang.System.setErr(System.java:199) > at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) > > "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable > > "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 waiting on > condition > > JNI global references: 19 > > > Found one Java-level deadlock: > ============================= > "Thread-1": > waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), > which is held by "main" > "main": > waiting to lock monitor 0x14274a3c (object 0x03f624b8, a > java.lang.Object), > which is held by "Thread-1" > > Java stack information for the threads listed above: > =================================================== > "Thread-1": > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > > at > org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:44) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:57) > at > org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) > > at > org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) > at > org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > > at > java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) > at > java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) > > at > java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) > at > java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) > > at > java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) > > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) > at > java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) > > at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) > at java.lang.ClassLoader.loadClass(ClassLoader.java:404) > at java.lang.ClassLoader.loadClass(ClassLoader.java:411) > - locked <0x03f624b8> (a java.lang.Object) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) > > at > org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) > > at > org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) > > - locked <0x03ef8b30> (a > org.cliffc.high_scale_lib.NonBlockingHashMap) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) > at > java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j > > ava:180) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 > > ) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > > at java.lang.Thread.run(Thread.java:744) > "main": > at java.lang.ClassLoader.loadClass(ClassLoader.java:406) > - waiting to lock <0x03f624b8> (a java.lang.Object) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > > at > org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:44) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:57) > at > org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) > > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) > > at > org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) > at > org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > > at java.lang.System.checkIO(System.java:253) > at java.lang.System.setErr(System.java:199) > at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) > > Found 1 deadlock. From peter.firmstone at zeus.net.au Sat Feb 6 12:17:10 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sat, 06 Feb 2016 22:17:10 +1000 Subject: ClassLoader deadlock In-Reply-To: <56B5DF8C.5010201@oracle.com> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> Message-ID: <56B5E446.9020205@zeus.net.au> The security manager is non blocking, unfortunately java system classes aren't. The policy provider in use thread confines PermissionCollection instances, which never leave the cpu cache, they are discarded immediately after use. The problem here is that two different threads are attempting to load the same class at the same time. I'll have to make sure that all classes are loaded before the security manager becomes the system security manager, either that or eliminate the AccessControlContext check permission cache. While the standard java security manager and policy provider consume around 10% cpu, this consumes less than 1%. On 6/02/2016 9:57 PM, David Holmes wrote: > On 6/02/2016 9:39 PM, Peter Firmstone wrote: >> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >> deadlock prone? Guess I was wrong. > > The deadlock is introduced by a custom security manager. SM's play a > critical role in many parts of the core libraries so if they utilize > synchronization then they can easily introduce deadlock. > >> Observation: On an unrelated occassion, I had a URLClassLoader >> synchronization hotspot (well not standard URLClassLoader, but a high >> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >> instead of URL DNS lookup), I solved that problem by thread confining >> class loading. I've never experienced deadlock using thread >> confinement, initially I was worried that there would be >> interdependencies, however I didn't experience a problem, as the single >> thread would always load the classes it needed. >> >> I'm wondering if multiple locks for ClassLoader's might be the wrong >> strategy, I found thread confinement was very performant. > > Are you suggesting that a class-load request is actually transmitted > to a single class-loading thread, while the initial thread blocks > until the loading is complete? That would still seem prone to the same > SM related deadlock scenario. And it would not generally be > particularly performant. I thought that too, until I tried it. Would you like to see some performance figures? Cheers, Peter. > > David > >> Regards, >> >> Peter. >> >> 2016-02-06 21:06:07 >> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >> >> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >> runnable [0x00000000] >> java.lang.Thread.State: RUNNABLE >> >> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >> nid=0x111c waiting on condition [0x00000000] >> java.lang.Thread.State: RUNNABLE >> >> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 >> waiting on condition [0x00000000] >> java.lang.Thread.State: RUNNABLE >> >> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 nid=0x10b8 >> runnable [0x00000000] >> java.lang.Thread.State: RUNNABLE >> >> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >> monitor entry [0x03ded000] >> java.lang.Thread.State: BLOCKED (on object monitor) >> at >> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >> >> >> at >> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >> >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >> >> at >> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >> >> >> at >> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >> at >> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >> >> >> at >> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >> at >> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >> >> >> at >> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >> >> at >> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >> >> >> at >> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >> >> >> at >> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >> >> at >> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >> >> >> at >> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >> - locked <0x03f624b8> (a java.lang.Object) >> at >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at >> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >> >> >> at >> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >> >> >> - locked <0x03ef8b30> (a >> org.cliffc.high_scale_lib.NonBlockingHashMap) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at >> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >> >> >> ava:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >> >> >> ) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> >> >> at java.lang.Thread.run(Thread.java:744) >> >> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >> Object.wait() [0x03cff000] >> java.lang.Thread.State: WAITING (on object monitor) >> at java.lang.Object.wait(Native Method) >> - waiting on <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >> at >> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >> >> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >> nid=0x1398 in Object.wait() [0x144cf000] >> java.lang.Thread.State: WAITING (on object monitor) >> at java.lang.Object.wait(Native Method) >> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >> at java.lang.Object.wait(Object.java:502) >> at >> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >> >> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor >> entry [0x0185e000] >> java.lang.Thread.State: BLOCKED (on object monitor) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >> - waiting to lock <0x03f624b8> (a java.lang.Object) >> at >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at >> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >> >> >> at >> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >> >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >> >> at >> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >> >> >> at >> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >> at >> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >> >> >> at java.lang.System.checkIO(System.java:253) >> at java.lang.System.setErr(System.java:199) >> at >> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >> >> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >> >> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 waiting on >> condition >> >> JNI global references: 19 >> >> >> Found one Java-level deadlock: >> ============================= >> "Thread-1": >> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >> which is held by "main" >> "main": >> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >> java.lang.Object), >> which is held by "Thread-1" >> >> Java stack information for the threads listed above: >> =================================================== >> "Thread-1": >> at >> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >> >> >> at >> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >> >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >> >> at >> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >> >> >> at >> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >> at >> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >> >> >> at >> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >> at >> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >> >> >> at >> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >> >> at >> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >> >> >> at >> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >> >> >> at >> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >> >> at >> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >> >> >> at >> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >> - locked <0x03f624b8> (a java.lang.Object) >> at >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at >> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >> >> >> at >> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >> >> >> - locked <0x03ef8b30> (a >> org.cliffc.high_scale_lib.NonBlockingHashMap) >> at >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >> at >> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >> >> >> ava:180) >> at >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >> >> >> ) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> >> >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> >> >> at java.lang.Thread.run(Thread.java:744) >> "main": >> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >> - waiting to lock <0x03f624b8> (a java.lang.Object) >> at >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at >> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >> >> >> at >> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >> >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >> >> at >> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >> >> >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >> >> >> at >> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >> at >> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >> >> >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >> >> >> at java.lang.System.checkIO(System.java:253) >> at java.lang.System.setErr(System.java:199) >> at >> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >> >> Found 1 deadlock. From amaembo at gmail.com Sat Feb 6 13:29:45 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sat, 6 Feb 2016 19:29:45 +0600 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> Message-ID: <1489632171.20160206192945@gmail.com> Hello! PS> I still disagree and pushing back on the support for splitting PS> after partial traversal. It?s not a pattern i think we should PS> encourage and propagate so such behaviour can be generally relied PS> upon, and predominantly for edge cases. That?s where the PS> complexity string is being pulled on. While your fix in isolation PS> is not terribly complex, it is more complex than the alternative PS> (which was actually the intent of the current impl, we just forget to include the check). I still don't like doing this, but as Brian agreed with you [1], seems I have no other choice. Here's updated webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r3/ With best regards, Tagir Valeev. [1] https://twitter.com/BrianGoetz/status/694985109628387328 PS> Paul. >> On 3 Feb 2016, at 12:19, Tagir F. Valeev wrote: >> >> Hello! >> >> Here's updated webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r2/ >> >> PS> My inclination is to keep this simple and not support splitting after partial traversal. >> >> PS> Sometimes splitting after partial traversal might be more complex >> PS> not support (e.g. ArrayList). In other cases it?s more complex to >> PS> support and in such cases i would argue it is not worth it since >> PS> this kind of functionality is an edge case. >> >> I would still like that this problem is fixed (i.e. support splitting >> after advance, not just return null). This is probably an edge case >> for JDK, but might be crucial for library writers. As a library writer >> I actually had bad times working around this issue. While returning >> null instead of incorrect splitting would indeed be helpful, it will >> unnecessarily remove parallelization capabilities in some cases. For >> example, sometimes it's desired to extract the first element from the >> stream and create the stream from the tail. Returning null here would >> mean that the resulting stream will never split after that. >> >> To my opinion this fix is not very complex. It just adds several lines >> into single method (trySplit()). I added a comment which clarifies the >> intention. Other changes like extraction of initPusher() do not add >> complexity. Actually they reduce the complexity as four identical >> lambdas are merged into one. This patch actually reduces the size of >> created class files. With javac 9-ea+99 StreamSpliterators.java is >> compiled into 79004 bytes after my patch and 79472 bytes before my >> patch. Also this patch does not require primitive specializations and >> adds no overhead for common case when traversal is not started. >> >> PS> Testing wise you are right to be concerned about the increase in >> PS> test execution time. The lack of flatMap is definitely an >> PS> omission. In this case I think it is ok to replace map with >> PS> flatMap, as both result in stuff going into the holding buffer, >> PS> and we can use the smaller data sets, e.g. >> PS> "StreamTestData.small?, that were recently added. >> >> I replaced all the datasets with .small alternatives (I think it's ok >> here as WrappingSpliterator behavior does not differ much for various >> sources) and removed all .map tests. Now the test works faster >> (like 25 sec -> 17 sec on by box). Please check. >> >> With best regards, >> Tagir Valeev. >> >> PS. >> >> PS> Paul. >> >>>> On 2 Feb 2016, at 09:24, Tagir F. Valeev wrote: >>>> >>>> Please review and sponsor this fix: >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8148838 >>>> http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r1/ >>>> >>>> When buffer traversal is already started, and split is requested, then >>>> the existing buffer should be carefully transferred to the prefix >>>> part. >>>> >>>> The only problem I see here is the testing time. Due to >>>> permuteFunctions() call it increases significantly what I added the >>>> flatMap() test. If this becomes an issue, I can write new simple test >>>> which tests flatMap() only (without permutations with other >>>> intermediate operations). >>>> >>>> With best regards, >>>> Tagir Valeev. >>>> >> From peter.levart at gmail.com Sat Feb 6 18:27:29 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sat, 6 Feb 2016 19:27:29 +0100 Subject: ClassLoader deadlock In-Reply-To: <56B5E446.9020205@zeus.net.au> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> <56B5E446.9020205@zeus.net.au> Message-ID: <56B63B11.20903@gmail.com> On 02/06/2016 01:17 PM, Peter Firmstone wrote: > The security manager is non blocking, unfortunately java system > classes aren't. It doesn't seem so. At least, I can't find where main thread locks the 0x040ebee8 monitor: "Thread-1": waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), which is held by "main" "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor entry [0x0185e000] java.lang.Thread.State: BLOCKED (on object monitor) at java.lang.ClassLoader.loadClass(ClassLoader.java:406) - waiting to lock <0x03f624b8> (a java.lang.Object) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) at org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) at org.apache.river.concurrent.StrongReference.(StrongReference.java:44) at org.apache.river.concurrent.StrongReference.(StrongReference.java:57) at org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) at org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) at org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) at java.lang.System.checkIO(System.java:253) at java.lang.System.setErr(System.java:199) at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) ...no mention of 0x040ebee8 in main thread...?! Peter > > The policy provider in use thread confines PermissionCollection > instances, which never leave the cpu cache, they are discarded > immediately after use. > > The problem here is that two different threads are attempting to load > the same class at the same time. > > I'll have to make sure that all classes are loaded before the security > manager becomes the system security manager, either that or eliminate > the AccessControlContext check permission cache. > > While the standard java security manager and policy provider consume > around 10% cpu, this consumes less than 1%. > > On 6/02/2016 9:57 PM, David Holmes wrote: >> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>> deadlock prone? Guess I was wrong. >> >> The deadlock is introduced by a custom security manager. SM's play a >> critical role in many parts of the core libraries so if they utilize >> synchronization then they can easily introduce deadlock. >> >>> Observation: On an unrelated occassion, I had a URLClassLoader >>> synchronization hotspot (well not standard URLClassLoader, but a high >>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>> instead of URL DNS lookup), I solved that problem by thread confining >>> class loading. I've never experienced deadlock using thread >>> confinement, initially I was worried that there would be >>> interdependencies, however I didn't experience a problem, as the single >>> thread would always load the classes it needed. >>> >>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>> strategy, I found thread confinement was very performant. >> >> Are you suggesting that a class-load request is actually transmitted >> to a single class-loading thread, while the initial thread blocks >> until the loading is complete? That would still seem prone to the >> same SM related deadlock scenario. And it would not generally be >> particularly performant. > > I thought that too, until I tried it. > > Would you like to see some performance figures? > > Cheers, > > Peter. > >> >> David >> >>> Regards, >>> >>> Peter. >>> >>> 2016-02-06 21:06:07 >>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>> >>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>> runnable [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>> nid=0x111c waiting on condition [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 >>> waiting on condition [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 >>> nid=0x10b8 >>> runnable [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>> monitor entry [0x03ded000] >>> java.lang.Thread.State: BLOCKED (on object monitor) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at >>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>> at >>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>> >>> >>> at >>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>> >>> >>> at >>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>> - locked <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>> >>> >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>> >>> >>> - locked <0x03ef8b30> (a >>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>> >>> >>> ava:180) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>> >>> >>> ) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> >>> >>> at java.lang.Thread.run(Thread.java:744) >>> >>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>> Object.wait() [0x03cff000] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> - waiting on <0x03e056d8> (a >>> java.lang.ref.ReferenceQueue$Lock) >>> at >>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>> at >>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>> at >>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>> >>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>> nid=0x1398 in Object.wait() [0x144cf000] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >>> at java.lang.Object.wait(Object.java:502) >>> at >>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >>> >>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>> monitor >>> entry [0x0185e000] >>> java.lang.Thread.State: BLOCKED (on object monitor) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at java.lang.System.checkIO(System.java:253) >>> at java.lang.System.setErr(System.java:199) >>> at >>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>> >>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>> >>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 >>> waiting on >>> condition >>> >>> JNI global references: 19 >>> >>> >>> Found one Java-level deadlock: >>> ============================= >>> "Thread-1": >>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>> which is held by "main" >>> "main": >>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>> java.lang.Object), >>> which is held by "Thread-1" >>> >>> Java stack information for the threads listed above: >>> =================================================== >>> "Thread-1": >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at >>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>> at >>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>> >>> >>> at >>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>> >>> >>> at >>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>> - locked <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>> >>> >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>> >>> >>> - locked <0x03ef8b30> (a >>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>> >>> >>> ava:180) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>> >>> >>> ) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> >>> >>> at java.lang.Thread.run(Thread.java:744) >>> "main": >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at java.lang.System.checkIO(System.java:253) >>> at java.lang.System.setErr(System.java:199) >>> at >>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>> >>> Found 1 deadlock. > From david.holmes at oracle.com Sat Feb 6 21:28:15 2016 From: david.holmes at oracle.com (David Holmes) Date: Sun, 7 Feb 2016 07:28:15 +1000 Subject: ClassLoader deadlock In-Reply-To: <56B5E446.9020205@zeus.net.au> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> <56B5E446.9020205@zeus.net.au> Message-ID: <56B6656F.70002@oracle.com> On 6/02/2016 10:17 PM, Peter Firmstone wrote: > The security manager is non blocking, unfortunately java system classes > aren't. The security manager has a lot of dependencies on other code/classes and so requires lots of additional classloading to execute as desired. When the SM is itself used by the classloading mechanism then you can indeed get into problems due to "recursive" use of the SM and/or nested classloading. These risks/constraints are probably not spelt out at all in the javadoc for SecurityManager. > The policy provider in use thread confines PermissionCollection > instances, which never leave the cpu cache, they are discarded > immediately after use. > > The problem here is that two different threads are attempting to load > the same class at the same time. > > I'll have to make sure that all classes are loaded before the security > manager becomes the system security manager, either that or eliminate > the AccessControlContext check permission cache. Yes aggressive classloading may be able to avoid this problem. > While the standard java security manager and policy provider consume > around 10% cpu, this consumes less than 1%. > > On 6/02/2016 9:57 PM, David Holmes wrote: >> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>> deadlock prone? Guess I was wrong. >> >> The deadlock is introduced by a custom security manager. SM's play a >> critical role in many parts of the core libraries so if they utilize >> synchronization then they can easily introduce deadlock. >> >>> Observation: On an unrelated occassion, I had a URLClassLoader >>> synchronization hotspot (well not standard URLClassLoader, but a high >>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>> instead of URL DNS lookup), I solved that problem by thread confining >>> class loading. I've never experienced deadlock using thread >>> confinement, initially I was worried that there would be >>> interdependencies, however I didn't experience a problem, as the single >>> thread would always load the classes it needed. >>> >>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>> strategy, I found thread confinement was very performant. >> >> Are you suggesting that a class-load request is actually transmitted >> to a single class-loading thread, while the initial thread blocks >> until the loading is complete? That would still seem prone to the same >> SM related deadlock scenario. And it would not generally be >> particularly performant. > > I thought that too, until I tried it. > > Would you like to see some performance figures? I can quite believe that there are circumstances where this can be performant. But in the general case the thread communication overhead etc would seem likely to make this less performant. Cheers, David > Cheers, > > Peter. > >> >> David >> >>> Regards, >>> >>> Peter. >>> >>> 2016-02-06 21:06:07 >>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>> >>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>> runnable [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>> nid=0x111c waiting on condition [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 >>> waiting on condition [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 nid=0x10b8 >>> runnable [0x00000000] >>> java.lang.Thread.State: RUNNABLE >>> >>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>> monitor entry [0x03ded000] >>> java.lang.Thread.State: BLOCKED (on object monitor) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at >>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>> at >>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>> >>> >>> at >>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>> >>> >>> at >>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>> - locked <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>> >>> >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>> >>> >>> - locked <0x03ef8b30> (a >>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>> >>> >>> ava:180) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>> >>> >>> ) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> >>> >>> at java.lang.Thread.run(Thread.java:744) >>> >>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>> Object.wait() [0x03cff000] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> - waiting on <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>> at >>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>> >>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>> nid=0x1398 in Object.wait() [0x144cf000] >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >>> at java.lang.Object.wait(Object.java:502) >>> at >>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >>> >>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor >>> entry [0x0185e000] >>> java.lang.Thread.State: BLOCKED (on object monitor) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at java.lang.System.checkIO(System.java:253) >>> at java.lang.System.setErr(System.java:199) >>> at >>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>> >>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>> >>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 waiting on >>> condition >>> >>> JNI global references: 19 >>> >>> >>> Found one Java-level deadlock: >>> ============================= >>> "Thread-1": >>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>> which is held by "main" >>> "main": >>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>> java.lang.Object), >>> which is held by "Thread-1" >>> >>> Java stack information for the threads listed above: >>> =================================================== >>> "Thread-1": >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at >>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>> at >>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>> >>> >>> at >>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>> >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>> >>> at >>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>> >>> >>> at >>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>> - locked <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>> >>> >>> at >>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>> >>> >>> - locked <0x03ef8b30> (a >>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>> at >>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>> at >>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>> >>> >>> ava:180) >>> at >>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>> >>> >>> ) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>> >>> >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>> >>> >>> at java.lang.Thread.run(Thread.java:744) >>> "main": >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> >>> >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> >>> >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> >>> >>> at java.lang.System.checkIO(System.java:253) >>> at java.lang.System.setErr(System.java:199) >>> at >>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>> >>> Found 1 deadlock. > From david.holmes at oracle.com Sat Feb 6 21:31:22 2016 From: david.holmes at oracle.com (David Holmes) Date: Sun, 7 Feb 2016 07:31:22 +1000 Subject: ClassLoader deadlock In-Reply-To: <56B63B11.20903@gmail.com> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> <56B5E446.9020205@zeus.net.au> <56B63B11.20903@gmail.com> Message-ID: <56B6662A.4020507@oracle.com> On 7/02/2016 4:27 AM, Peter Levart wrote: > > > On 02/06/2016 01:17 PM, Peter Firmstone wrote: >> The security manager is non blocking, unfortunately java system >> classes aren't. > > It doesn't seem so. At least, I can't find where main thread locks the > 0x040ebee8 monitor: > > "Thread-1": > waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), > which is held by "main" I think this is an internal monitor acquisition by the VM in the class initialization code. It won't show up on the thread dump - at least in the version of the VM being run here. Latest JDK 9 may behave differently. Cheers, David > > "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor > entry [0x0185e000] > java.lang.Thread.State: BLOCKED (on object monitor) > at java.lang.ClassLoader.loadClass(ClassLoader.java:406) > - waiting to lock <0x03f624b8> (a java.lang.Object) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > at > org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:44) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:57) > at > org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) > at > org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) > at > org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > at java.lang.System.checkIO(System.java:253) > at java.lang.System.setErr(System.java:199) > at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) > > > ...no mention of 0x040ebee8 in main thread...?! > > Peter > >> >> The policy provider in use thread confines PermissionCollection >> instances, which never leave the cpu cache, they are discarded >> immediately after use. >> >> The problem here is that two different threads are attempting to load >> the same class at the same time. >> >> I'll have to make sure that all classes are loaded before the security >> manager becomes the system security manager, either that or eliminate >> the AccessControlContext check permission cache. >> >> While the standard java security manager and policy provider consume >> around 10% cpu, this consumes less than 1%. >> >> On 6/02/2016 9:57 PM, David Holmes wrote: >>> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>>> deadlock prone? Guess I was wrong. >>> >>> The deadlock is introduced by a custom security manager. SM's play a >>> critical role in many parts of the core libraries so if they utilize >>> synchronization then they can easily introduce deadlock. >>> >>>> Observation: On an unrelated occassion, I had a URLClassLoader >>>> synchronization hotspot (well not standard URLClassLoader, but a high >>>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>>> instead of URL DNS lookup), I solved that problem by thread confining >>>> class loading. I've never experienced deadlock using thread >>>> confinement, initially I was worried that there would be >>>> interdependencies, however I didn't experience a problem, as the single >>>> thread would always load the classes it needed. >>>> >>>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>>> strategy, I found thread confinement was very performant. >>> >>> Are you suggesting that a class-load request is actually transmitted >>> to a single class-loading thread, while the initial thread blocks >>> until the loading is complete? That would still seem prone to the >>> same SM related deadlock scenario. And it would not generally be >>> particularly performant. >> >> I thought that too, until I tried it. >> >> Would you like to see some performance figures? >> >> Cheers, >> >> Peter. >> >>> >>> David >>> >>>> Regards, >>>> >>>> Peter. >>>> >>>> 2016-02-06 21:06:07 >>>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>>> >>>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>>> runnable [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>>> nid=0x111c waiting on condition [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 >>>> waiting on condition [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 >>>> nid=0x10b8 >>>> runnable [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>>> monitor entry [0x03ded000] >>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at >>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>> at >>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>> >>>> >>>> at >>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>> - locked <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>> >>>> >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>> >>>> >>>> - locked <0x03ef8b30> (a >>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> at >>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>> >>>> >>>> ava:180) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>> >>>> >>>> ) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> >>>> >>>> at java.lang.Thread.run(Thread.java:744) >>>> >>>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>>> Object.wait() [0x03cff000] >>>> java.lang.Thread.State: WAITING (on object monitor) >>>> at java.lang.Object.wait(Native Method) >>>> - waiting on <0x03e056d8> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> at >>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>>> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>>> at >>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>>> at >>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>>> >>>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>>> nid=0x1398 in Object.wait() [0x144cf000] >>>> java.lang.Thread.State: WAITING (on object monitor) >>>> at java.lang.Object.wait(Native Method) >>>> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >>>> at java.lang.Object.wait(Object.java:502) >>>> at >>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>>> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >>>> >>>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>>> monitor >>>> entry [0x0185e000] >>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at java.lang.System.checkIO(System.java:253) >>>> at java.lang.System.setErr(System.java:199) >>>> at >>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>> >>>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>>> >>>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 >>>> waiting on >>>> condition >>>> >>>> JNI global references: 19 >>>> >>>> >>>> Found one Java-level deadlock: >>>> ============================= >>>> "Thread-1": >>>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>>> which is held by "main" >>>> "main": >>>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>>> java.lang.Object), >>>> which is held by "Thread-1" >>>> >>>> Java stack information for the threads listed above: >>>> =================================================== >>>> "Thread-1": >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at >>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>> at >>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>> >>>> >>>> at >>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>> - locked <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>> >>>> >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>> >>>> >>>> - locked <0x03ef8b30> (a >>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> at >>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>> >>>> >>>> ava:180) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>> >>>> >>>> ) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> >>>> >>>> at java.lang.Thread.run(Thread.java:744) >>>> "main": >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at java.lang.System.checkIO(System.java:253) >>>> at java.lang.System.setErr(System.java:199) >>>> at >>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>> >>>> Found 1 deadlock. >> > From peter.levart at gmail.com Sun Feb 7 08:37:16 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 7 Feb 2016 09:37:16 +0100 Subject: [concurrency-interest] ClassLoader deadlock In-Reply-To: <56B6668A.3030106@zeus.net.au> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> <56B5E446.9020205@zeus.net.au> <56B63B11.20903@gmail.com> <56B6668A.3030106@zeus.net.au> Message-ID: <56B7023C.5030708@gmail.com> On 02/06/2016 10:32 PM, Peter wrote: > The 0x040ebee8 monitor is most likely being held by native code. > > Regards, > > Peter Firmstone. Ok, but where? The deadlock report says it is held by main thread. Somewhere between the following two java frames? "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor entry [0x0185e000] ... ... at java.lang.ClassLoader.loadClass(ClassLoader.java:357) --here-- at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) As I understand this is where JVM resolves some class that for the 1st time while ReferenceCollection.iterator is being executed. So before calling-back into ClassLoader.loadClass, it acquires a monitor lock on some int[] object? Anyway. Looking at the following part of Thread-1 stacktrace: at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) at java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) at java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) at java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) at java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) at java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) at java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) ...I can see you are using some early release of JDK 8 where ThreadLocalRandom initialization involved obtaining a hardware address of some interface. This in turn checks for security manager. Since JDK 8u45 this code has been removed and only current time is used to derive initial seed, so you might not see this deadlock any more with JDK 8u45 and later. Regards, Peter > > > On 7/02/2016 4:27 AM, Peter Levart wrote: >> >> >> On 02/06/2016 01:17 PM, Peter Firmstone wrote: >>> The security manager is non blocking, unfortunately java system >>> classes aren't. >> >> It doesn't seem so. At least, I can't find where main thread locks >> the 0x040ebee8 monitor: >> >> "Thread-1": >> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >> which is held by "main" >> >> >> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >> monitor entry [0x0185e000] >> java.lang.Thread.State: BLOCKED (on object monitor) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >> - waiting to lock <0x03f624b8> (a java.lang.Object) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at >> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >> at >> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >> at >> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >> at >> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >> at >> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >> at >> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >> at >> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >> at >> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >> at java.lang.System.checkIO(System.java:253) >> at java.lang.System.setErr(System.java:199) >> at >> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >> >> >> ...no mention of 0x040ebee8 in main thread...?! >> >> Peter >> >>> >>> The policy provider in use thread confines PermissionCollection >>> instances, which never leave the cpu cache, they are discarded >>> immediately after use. >>> >>> The problem here is that two different threads are attempting to >>> load the same class at the same time. >>> >>> I'll have to make sure that all classes are loaded before the >>> security manager becomes the system security manager, either that or >>> eliminate the AccessControlContext check permission cache. >>> >>> While the standard java security manager and policy provider consume >>> around 10% cpu, this consumes less than 1%. >>> >>> On 6/02/2016 9:57 PM, David Holmes wrote: >>>> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>>>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>>>> deadlock prone? Guess I was wrong. >>>> >>>> The deadlock is introduced by a custom security manager. SM's play >>>> a critical role in many parts of the core libraries so if they >>>> utilize synchronization then they can easily introduce deadlock. >>>> >>>>> Observation: On an unrelated occassion, I had a URLClassLoader >>>>> synchronization hotspot (well not standard URLClassLoader, but a high >>>>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>>>> instead of URL DNS lookup), I solved that problem by thread confining >>>>> class loading. I've never experienced deadlock using thread >>>>> confinement, initially I was worried that there would be >>>>> interdependencies, however I didn't experience a problem, as the >>>>> single >>>>> thread would always load the classes it needed. >>>>> >>>>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>>>> strategy, I found thread confinement was very performant. >>>> >>>> Are you suggesting that a class-load request is actually >>>> transmitted to a single class-loading thread, while the initial >>>> thread blocks until the loading is complete? That would still seem >>>> prone to the same SM related deadlock scenario. And it would not >>>> generally be particularly performant. >>> >>> I thought that too, until I tried it. >>> >>> Would you like to see some performance figures? >>> >>> Cheers, >>> >>> Peter. >>> >>>> >>>> David >>>> >>>>> Regards, >>>>> >>>>> Peter. >>>>> >>>>> 2016-02-06 21:06:07 >>>>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>>>> >>>>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>>>> runnable [0x00000000] >>>>> java.lang.Thread.State: RUNNABLE >>>>> >>>>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>>>> nid=0x111c waiting on condition [0x00000000] >>>>> java.lang.Thread.State: RUNNABLE >>>>> >>>>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 >>>>> nid=0x13d8 >>>>> waiting on condition [0x00000000] >>>>> java.lang.Thread.State: RUNNABLE >>>>> >>>>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 >>>>> nid=0x10b8 >>>>> runnable [0x00000000] >>>>> java.lang.Thread.State: RUNNABLE >>>>> >>>>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>>>> monitor entry [0x03ded000] >>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>> at >>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>> >>>>> >>>>> at >>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>> >>>>> at >>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>> >>>>> >>>>> at >>>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>> - locked <0x03f624b8> (a java.lang.Object) >>>>> at >>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> at >>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>> >>>>> >>>>> at >>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>> >>>>> >>>>> - locked <0x03ef8b30> (a >>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>> at >>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>> >>>>> at >>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>> at >>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>> >>>>> >>>>> ava:180) >>>>> at >>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>> >>>>> >>>>> ) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>> >>>>> >>>>> at java.lang.Thread.run(Thread.java:744) >>>>> >>>>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>>>> Object.wait() [0x03cff000] >>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>> at java.lang.Object.wait(Native Method) >>>>> - waiting on <0x03e056d8> (a >>>>> java.lang.ref.ReferenceQueue$Lock) >>>>> at >>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>>>> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>>>> at >>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>>>> at >>>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>>>> >>>>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>>>> nid=0x1398 in Object.wait() [0x144cf000] >>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>> at java.lang.Object.wait(Native Method) >>>>> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >>>>> at java.lang.Object.wait(Object.java:502) >>>>> at >>>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>>>> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >>>>> >>>>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>>>> monitor >>>>> entry [0x0185e000] >>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>>> at >>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> at >>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>> >>>>> >>>>> at java.lang.System.checkIO(System.java:253) >>>>> at java.lang.System.setErr(System.java:199) >>>>> at >>>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>> >>>>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>>>> >>>>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 >>>>> waiting on >>>>> condition >>>>> >>>>> JNI global references: 19 >>>>> >>>>> >>>>> Found one Java-level deadlock: >>>>> ============================= >>>>> "Thread-1": >>>>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>>>> which is held by "main" >>>>> "main": >>>>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>>>> java.lang.Object), >>>>> which is held by "Thread-1" >>>>> >>>>> Java stack information for the threads listed above: >>>>> =================================================== >>>>> "Thread-1": >>>>> at >>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>> >>>>> >>>>> at >>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>> >>>>> at >>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>> >>>>> at >>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>> >>>>> >>>>> at >>>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>> - locked <0x03f624b8> (a java.lang.Object) >>>>> at >>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> at >>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>> >>>>> >>>>> at >>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>> >>>>> >>>>> - locked <0x03ef8b30> (a >>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>> at >>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>> >>>>> at >>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>> at >>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>> >>>>> >>>>> ava:180) >>>>> at >>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>> >>>>> >>>>> ) >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>> >>>>> >>>>> at >>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>> >>>>> >>>>> at java.lang.Thread.run(Thread.java:744) >>>>> "main": >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>>> at >>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>> at >>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>> >>>>> at >>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>> >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>> >>>>> at >>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>> >>>>> >>>>> at >>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>> >>>>> >>>>> at java.lang.System.checkIO(System.java:253) >>>>> at java.lang.System.setErr(System.java:199) >>>>> at >>>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>> >>>>> Found 1 deadlock. >>> >> >> >> _______________________________________________ >> Concurrency-interest mailing list >> Concurrency-interest at cs.oswego.edu >> http://cs.oswego.edu/mailman/listinfo/concurrency-interest > From amaembo at gmail.com Sun Feb 7 08:45:35 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sun, 7 Feb 2016 14:45:35 +0600 Subject: Using BaseStream.isParallel() after calling terminal operation Message-ID: <1295943164.20160207144535@gmail.com> Hello! Currently the isParallel() spec says [1]: > Calling this method after invoking an terminal stream operation > method may yield unpredictable results. The spliterator() spec says [2]: > This is a terminal operation. As a consequence, we cannot legally call isParallel() after calling spliterator(). However it's actually called in Stream.concat() implementation [3] as well as in Stream.takeWhile()/dropWhile() default implementations [4], [5]. The same is for primitive streams. In all of these cases (especially in concat case) it's possible that third-party Stream interface implementation is passed, which, according to spec, can legally return garbage or even throw IllegalStateException upon calling isParallel(). So to my understanding, either spec or implementations should be corrected to match each other. My suggestion is to make exceptions for calling isParallel() after spliterator() or iterator() terminal operation: isParallel() should still return correct value. Though probably it should be additionally specified what is considered as correct value in this case. What do you think? Should we bother about such small thing? With best regards, Tagir Valeev [1] http://download.java.net/jdk9/docs/api/java/util/stream/BaseStream.html#isParallel-- [2] http://download.java.net/jdk9/docs/api/java/util/stream/BaseStream.html#spliterator-- [3] http://hg.openjdk.java.net/jdk9/dev/jdk/file/fddcdea594f5/src/java.base/share/classes/java/util/stream/Stream.java#l1248 [4] http://hg.openjdk.java.net/jdk9/dev/jdk/file/fddcdea594f5/src/java.base/share/classes/java/util/stream/Stream.java#l542 [5] http://hg.openjdk.java.net/jdk9/dev/jdk/file/fddcdea594f5/src/java.base/share/classes/java/util/stream/Stream.java#l608 From peter.firmstone at zeus.net.au Sun Feb 7 10:04:15 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sun, 07 Feb 2016 20:04:15 +1000 Subject: Fwd: Re: [concurrency-interest] ClassLoader deadlock Message-ID: <56B7169F.700@zeus.net.au> Thanks Peter & David, That's good to know. I've altered the SecurityManager to perform a permission check prior to becoming the security manager, this ensures the cache has been created, to avoid any recursive deadlock prone calls. The class that both threads are attempting to load and init is called org.apache.river.concurrent.ReferenceIterator. The code can be found here: https://github.com/pfirmstone/river-internet/tree/Input-validation-for-Serialization Cheers, Peter. On 7/02/2016 6:37 PM, Peter Levart wrote: > > > On 02/06/2016 10:32 PM, Peter wrote: >> The 0x040ebee8 monitor is most likely being held by native code. >> >> Regards, >> >> Peter Firmstone. > > Ok, but where? The deadlock report says it is held by main thread. > Somewhere between the following two java frames? > > "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for > monitor entry [0x0185e000] > ... > ... > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > --here-- > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > > > As I understand this is where JVM resolves some class that for the 1st > time while ReferenceCollection.iterator is being executed. So before > calling-back into ClassLoader.loadClass, it acquires a monitor lock on > some int[] object? > > > Anyway. Looking at the following part of Thread-1 stacktrace: > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > at > java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) > at > java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) > at > java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) > at > java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) > at > java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) > at > java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) > > ...I can see you are using some early release of JDK 8 where > ThreadLocalRandom initialization involved obtaining a hardware address > of some interface. This in turn checks for security manager. Since JDK > 8u45 this code has been removed and only current time is used to > derive initial seed, so you might not see this deadlock any more with > JDK 8u45 and later. > > Regards, Peter > >> >> >> On 7/02/2016 4:27 AM, Peter Levart wrote: >>> >>> >>> On 02/06/2016 01:17 PM, Peter Firmstone wrote: >>>> The security manager is non blocking, unfortunately java system >>>> classes aren't. >>> >>> It doesn't seem so. At least, I can't find where main thread locks >>> the 0x040ebee8 monitor: >>> >>> "Thread-1": >>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>> which is held by "main" >>> >>> >>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>> monitor entry [0x0185e000] >>> java.lang.Thread.State: BLOCKED (on object monitor) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>> - waiting to lock<0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> at java.lang.System.checkIO(System.java:253) >>> at java.lang.System.setErr(System.java:199) >>> at >>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>> >>> >>> ...no mention of 0x040ebee8 in main thread...?! >>> >>> Peter >>> >>>> >>>> The policy provider in use thread confines PermissionCollection >>>> instances, which never leave the cpu cache, they are discarded >>>> immediately after use. >>>> >>>> The problem here is that two different threads are attempting to >>>> load the same class at the same time. >>>> >>>> I'll have to make sure that all classes are loaded before the >>>> security manager becomes the system security manager, either that >>>> or eliminate the AccessControlContext check permission cache. >>>> >>>> While the standard java security manager and policy provider >>>> consume around 10% cpu, this consumes less than 1%. >>>> >>>> On 6/02/2016 9:57 PM, David Holmes wrote: >>>>> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>>>>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>>>>> deadlock prone? Guess I was wrong. >>>>> >>>>> The deadlock is introduced by a custom security manager. SM's play >>>>> a critical role in many parts of the core libraries so if they >>>>> utilize synchronization then they can easily introduce deadlock. >>>>> >>>>>> Observation: On an unrelated occassion, I had a URLClassLoader >>>>>> synchronization hotspot (well not standard URLClassLoader, but a >>>>>> high >>>>>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>>>>> instead of URL DNS lookup), I solved that problem by thread >>>>>> confining >>>>>> class loading. I've never experienced deadlock using thread >>>>>> confinement, initially I was worried that there would be >>>>>> interdependencies, however I didn't experience a problem, as the >>>>>> single >>>>>> thread would always load the classes it needed. >>>>>> >>>>>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>>>>> strategy, I found thread confinement was very performant. >>>>> >>>>> Are you suggesting that a class-load request is actually >>>>> transmitted to a single class-loading thread, while the initial >>>>> thread blocks until the loading is complete? That would still seem >>>>> prone to the same SM related deadlock scenario. And it would not >>>>> generally be particularly performant. >>>> >>>> I thought that too, until I tried it. >>>> >>>> Would you like to see some performance figures? >>>> >>>> Cheers, >>>> >>>> Peter. >>>> >>>>> >>>>> David >>>>> >>>>>> Regards, >>>>>> >>>>>> Peter. >>>>>> >>>>>> 2016-02-06 21:06:07 >>>>>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>>>>> >>>>>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>>>>> runnable [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>>>>> nid=0x111c waiting on condition [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 >>>>>> nid=0x13d8 >>>>>> waiting on condition [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 >>>>>> nid=0x10b8 >>>>>> runnable [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>>>>> monitor entry [0x03ded000] >>>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at >>>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>>> >>>>>> >>>>>> at >>>>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>>> - locked<0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>>> >>>>>> >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>>> >>>>>> >>>>>> - locked<0x03ef8b30> (a >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>>> at >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>>> >>>>>> >>>>>> ava:180) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>>> >>>>>> >>>>>> ) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>>> >>>>>> >>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>> >>>>>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>>>>> Object.wait() [0x03cff000] >>>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>>> at java.lang.Object.wait(Native Method) >>>>>> - waiting on<0x03e056d8> (a >>>>>> java.lang.ref.ReferenceQueue$Lock) >>>>>> at >>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>>>>> - locked<0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>>>>> at >>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>>>>> at >>>>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>>>>> >>>>>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>>>>> nid=0x1398 in Object.wait() [0x144cf000] >>>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>>> at java.lang.Object.wait(Native Method) >>>>>> - waiting on<0x03e05200> (a java.lang.ref.Reference$Lock) >>>>>> at java.lang.Object.wait(Object.java:502) >>>>>> at >>>>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>>>>> - locked<0x03e05200> (a java.lang.ref.Reference$Lock) >>>>>> >>>>>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>>>>> monitor >>>>>> entry [0x0185e000] >>>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>>> - waiting to lock<0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at java.lang.System.checkIO(System.java:253) >>>>>> at java.lang.System.setErr(System.java:199) >>>>>> at >>>>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>>> >>>>>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>>>>> >>>>>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 >>>>>> waiting on >>>>>> condition >>>>>> >>>>>> JNI global references: 19 >>>>>> >>>>>> >>>>>> Found one Java-level deadlock: >>>>>> ============================= >>>>>> "Thread-1": >>>>>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>>>>> which is held by "main" >>>>>> "main": >>>>>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>>>>> java.lang.Object), >>>>>> which is held by "Thread-1" >>>>>> >>>>>> Java stack information for the threads listed above: >>>>>> =================================================== >>>>>> "Thread-1": >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at >>>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>>> >>>>>> >>>>>> at >>>>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>>> - locked<0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>>> >>>>>> >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>>> >>>>>> >>>>>> - locked<0x03ef8b30> (a >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>>> at >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>>> >>>>>> >>>>>> ava:180) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>>> >>>>>> >>>>>> ) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>>> >>>>>> >>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>> "main": >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>>> - waiting to lock<0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at java.lang.System.checkIO(System.java:253) >>>>>> at java.lang.System.setErr(System.java:199) >>>>>> at >>>>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>>> >>>>>> Found 1 deadlock. >>>> >>> >>> >>> _______________________________________________ >>> Concurrency-interest mailing list >>> Concurrency-interest at cs.oswego.edu >>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest >> > From peter.levart at gmail.com Sun Feb 7 10:53:54 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 7 Feb 2016 11:53:54 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more Message-ID: <56B72242.7050102@gmail.com> Hi, sun.misc.Cleaner has been moved to internal package jdk.internal.ref recently [1] to clean-up sun.misc namespace. But now that: - we have comparable public API (java.lang.ref.Cleaner & Cleanable) [2] - we have an internal shared java.lang.ref.Cleaner instance (jdk.internal.ref.CleanerFactory.cleaner()) - sun.misc.Cleaner is not a special kind of Reference any more in the JVM [3] ...I think there's no reason to keep this special internal API any more. It can be replaced with public API. I propose to remove jdk.internal.ref.Cleaner class and replace its usages with java.lang.ref.Cleaner and friends [4]. What do you say? Regards, Peter [1] https://bugs.openjdk.java.net/browse/JDK-8148117 [2] https://bugs.openjdk.java.net/browse/JDK-8138696 [3] https://bugs.openjdk.java.net/browse/JDK-8143847 [4] http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.01/ From jeremymanson at google.com Sun Feb 7 19:01:18 2016 From: jeremymanson at google.com (Jeremy Manson) Date: Sun, 7 Feb 2016 11:01:18 -0800 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B72242.7050102@gmail.com> References: <56B72242.7050102@gmail.com> Message-ID: Hadoop seems to use sun.misc.Cleaner: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-common/2.7.1/org/apache/hadoop/crypto/CryptoStreamUtils.java So you may want to keep it around transitionally (a la Unsafe). Jeremy On Sun, Feb 7, 2016 at 2:53 AM, Peter Levart wrote: > Hi, > > sun.misc.Cleaner has been moved to internal package jdk.internal.ref > recently [1] to clean-up sun.misc namespace. But now that: > > - we have comparable public API (java.lang.ref.Cleaner & Cleanable) [2] > - we have an internal shared java.lang.ref.Cleaner instance > (jdk.internal.ref.CleanerFactory.cleaner()) > - sun.misc.Cleaner is not a special kind of Reference any more in the JVM > [3] > > ...I think there's no reason to keep this special internal API any more. > It can be replaced with public API. > > I propose to remove jdk.internal.ref.Cleaner class and replace its usages > with java.lang.ref.Cleaner and friends [4]. > > What do you say? > > Regards, Peter > > > [1] https://bugs.openjdk.java.net/browse/JDK-8148117 > [2] https://bugs.openjdk.java.net/browse/JDK-8138696 > [3] https://bugs.openjdk.java.net/browse/JDK-8143847 > [4] > http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.01/ > > > > > > > From peter.levart at gmail.com Sun Feb 7 22:20:24 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 7 Feb 2016 23:20:24 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> Message-ID: <56B7C328.3060800@gmail.com> On 02/07/2016 08:01 PM, Jeremy Manson wrote: > Hadoop seems to use sun.misc.Cleaner: > > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-common/2.7.1/org/apache/hadoop/crypto/CryptoStreamUtils.java > > So you may want to keep it around transitionally (a la Unsafe). JEP 260 [1] was listing sun.misc.Cleaner as a critical API, but recently, it was decided to "hide" it away into jdk.internal.ref package which will not be exported [2]. The reasoning was this: "sun.misc.Cleaner ( was previously listed as a critical internal API, but on further investigation has been moved to an open issue, for the following reasons: 1) its primary use in the JDK is within NIO direct buffers to release native memory. The base module cannot have a dependency on jdk.unsupported so will need to be updated to use an alternative cleaner, 2) the usage of Cleaner outside the JDK, as determined by corpus analysis, has largely been observed to hack into private fields of the internal NIO direct buffer classes to explicitly release native memory. As stated in 1), the type of the cleaner used by NIO direct buffers will have to change. Given this, and the fact that JDK 9 has a new general purposed cleaner API, java.lang.ref.Cleaner, the value of keep sun.misc.Cleaner is questionable." If the decision to remove sun.misc.Cleaner was partly influenced by the desire to maintain just 2 instead of 3 Cleaner(s), then my proposal to migrate JDK code to the public API might enable Oracle to reconsider keeping sun.misc.Cleaner. [1] https://bugs.openjdk.java.net/browse/JDK-8132928 [2] https://bugs.openjdk.java.net/browse/JDK-8148117 Regards, Peter > > Jeremy > > On Sun, Feb 7, 2016 at 2:53 AM, Peter Levart > wrote: > > Hi, > > sun.misc.Cleaner has been moved to internal package > jdk.internal.ref recently [1] to clean-up sun.misc namespace. But > now that: > > - we have comparable public API (java.lang.ref.Cleaner & > Cleanable) [2] > - we have an internal shared java.lang.ref.Cleaner instance > (jdk.internal.ref.CleanerFactory.cleaner()) > - sun.misc.Cleaner is not a special kind of Reference any more in > the JVM [3] > > ...I think there's no reason to keep this special internal API any > more. It can be replaced with public API. > > I propose to remove jdk.internal.ref.Cleaner class and replace its > usages with java.lang.ref.Cleaner and friends [4]. > > What do you say? > > Regards, Peter > > > [1] https://bugs.openjdk.java.net/browse/JDK-8148117 > [2] https://bugs.openjdk.java.net/browse/JDK-8138696 > [3] https://bugs.openjdk.java.net/browse/JDK-8143847 > [4] > http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.01/ > > > > > > > > From peter.levart at gmail.com Sun Feb 7 22:32:34 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 7 Feb 2016 23:32:34 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B72242.7050102@gmail.com> References: <56B72242.7050102@gmail.com> Message-ID: <56B7C602.5020705@gmail.com> Hi, I 1st thought that my version of jtreg did not want to parse the new JDK 9 version strings as it kept saying: Error: cannot determine version for JDK: build/linux-x86_64-normal-server-release/images/jdk ...but the fact was that the patched JDK did not even want to boot-up. So in order to make this actually work, I had to make some adjustments: - lambdas and method references in the java.lang.ref.Cleaner implementation had to be replaced with alternatives as the Cleaner is needed early in the startup sequence. - [sun.misc|jdk.internal.ref].Cleaner.create(referent, action) allowed null action and just returned null in that case while java.lang.ref.Cleaner.register(referent, action) throws NPE. There was only one such usage where null action could get passed-in - when a zero-sized MappedByteBuffer was constructed. Here's the version which passes java/lang/ref and java/nio tests: http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/ Regards, Peter On 02/07/2016 11:53 AM, Peter Levart wrote: > Hi, > > sun.misc.Cleaner has been moved to internal package jdk.internal.ref > recently [1] to clean-up sun.misc namespace. But now that: > > - we have comparable public API (java.lang.ref.Cleaner & Cleanable) [2] > - we have an internal shared java.lang.ref.Cleaner instance > (jdk.internal.ref.CleanerFactory.cleaner()) > - sun.misc.Cleaner is not a special kind of Reference any more in the > JVM [3] > > ...I think there's no reason to keep this special internal API any > more. It can be replaced with public API. > > I propose to remove jdk.internal.ref.Cleaner class and replace its > usages with java.lang.ref.Cleaner and friends [4]. > > What do you say? > > Regards, Peter > > > [1] https://bugs.openjdk.java.net/browse/JDK-8148117 > [2] https://bugs.openjdk.java.net/browse/JDK-8138696 > [3] https://bugs.openjdk.java.net/browse/JDK-8143847 > [4] > http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.01/ > > > > > > From peter.levart at gmail.com Sun Feb 7 22:57:02 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 7 Feb 2016 23:57:02 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B7C328.3060800@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> Message-ID: <56B7CBBE.3070007@gmail.com> On 02/07/2016 11:20 PM, Peter Levart wrote: > > > On 02/07/2016 08:01 PM, Jeremy Manson wrote: >> Hadoop seems to use sun.misc.Cleaner: >> >> http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-common/2.7.1/org/apache/hadoop/crypto/CryptoStreamUtils.java >> >> So you may want to keep it around transitionally (a la Unsafe). > > JEP 260 [1] was listing sun.misc.Cleaner as a critical API, but > recently, it was decided to "hide" it away into jdk.internal.ref > package which will not be exported [2]. The reasoning was this: > > "sun.misc.Cleaner ( was previously listed as a critical internal API, > but on further investigation has been moved to an open issue, for the > following reasons: 1) its primary use in the JDK is within NIO direct > buffers to release native memory. The base module cannot have a > dependency on jdk.unsupported so will need to be updated to use an > alternative cleaner, 2) the usage of Cleaner outside the JDK, as > determined by corpus analysis, has largely been observed to hack into > private fields of the internal NIO direct buffer classes to explicitly > release native memory. As stated in 1), the type of the cleaner used > by NIO direct buffers will have to change. Given this, and the fact > that JDK 9 has a new general purposed cleaner API, > java.lang.ref.Cleaner, the value of keep sun.misc.Cleaner is > questionable." > > If the decision to remove sun.misc.Cleaner was partly influenced by > the desire to maintain just 2 instead of 3 Cleaner(s), then my > proposal to migrate JDK code to the public API might enable Oracle to > reconsider keeping sun.misc.Cleaner. OTOH, what hadoop is doing is exactly the usage described in the above reasoning for removing sun.misc.Cleaner. Hadoop use case: public static void freeDB(ByteBuffer buffer) { if (buffer instanceof sun.nio.ch.DirectBuffer) { final sun.misc.Cleaner bufferCleaner = ((sun.nio.ch.DirectBuffer) buffer).cleaner(); bufferCleaner.clean(); } } can be rewritten using reflection to be compatible with either sun.misc.Cleaner (on JDK 8 or less) or java.lang.ref.Cleaner$Cleanable (on JDK 9 or more): static final Method cleanMethod; static { try { Class cleanerOrCleanable; try { cleanerOrCleanable = Class.forName("sun.misc.Cleaner"); } catch (ClassNotFoundException e1) { try { cleanerOrCleanable = Class.forName("java.lang.ref.Cleaner$Cleanable"); } catch (ClassNotFoundException e2) { e2.addSuppressed(e1); throw e2; } } cleanMethod = cleanerOrCleanable.getDeclaredMethod("clean"); } catch (Exception e) { throw new Error(e); } } public static void freeDB_JDK8or9(ByteBuffer buffer) { if (buffer instanceof sun.nio.ch.DirectBuffer) { final Object bufferCleaner = ((sun.nio.ch.DirectBuffer) buffer).cleaner(); try { if (bufferCleaner != null) { cleanMethod.invoke(bufferCleaner); } } catch (InvocationTargetException | IllegalAccessException e) { throw new Error(e); } } } I thought about this use case and even kept the name of the method on sun.nio.ch.DirectBuffer::cleaner to make things easier although the return type in my patched DirectBufer is called java.lang.ref.Cleaner$Cleanable. Regards, Peter > > [1] https://bugs.openjdk.java.net/browse/JDK-8132928 > [2] https://bugs.openjdk.java.net/browse/JDK-8148117 > > Regards, Peter > >> >> Jeremy >> >> On Sun, Feb 7, 2016 at 2:53 AM, Peter Levart > > wrote: >> >> Hi, >> >> sun.misc.Cleaner has been moved to internal package >> jdk.internal.ref recently [1] to clean-up sun.misc namespace. But >> now that: >> >> - we have comparable public API (java.lang.ref.Cleaner & >> Cleanable) [2] >> - we have an internal shared java.lang.ref.Cleaner instance >> (jdk.internal.ref.CleanerFactory.cleaner()) >> - sun.misc.Cleaner is not a special kind of Reference any more in >> the JVM [3] >> >> ...I think there's no reason to keep this special internal API >> any more. It can be replaced with public API. >> >> I propose to remove jdk.internal.ref.Cleaner class and replace >> its usages with java.lang.ref.Cleaner and friends [4]. >> >> What do you say? >> >> Regards, Peter >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8148117 >> [2] https://bugs.openjdk.java.net/browse/JDK-8138696 >> [3] https://bugs.openjdk.java.net/browse/JDK-8143847 >> [4] >> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.01/ >> >> >> >> >> >> >> >> > From masayoshi.okutsu at oracle.com Sun Feb 7 22:59:55 2016 From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu) Date: Mon, 8 Feb 2016 07:59:55 +0900 Subject: RFR: 8148446: (tz) Support tzdata2016a In-Reply-To: References: <3683f81e-a349-4dbe-8bee-780a0e454fd2@default> <7050b3f7-d566-4097-bdcc-9cbf76c272e4@default> <56B2F3E5.1070703@oracle.com> Message-ID: <56B7CC6B.6030800@oracle.com> Looks good to me. Masayoshi On 2/4/2016 5:55 PM, Ramanand Patil wrote: > Hi Masayoshi/all, > Please review the updated Webrev at: http://cr.openjdk.java.net/~rpatil/8148446/webrev.01/ > > Regards, > Ramanand. > > -----Original Message----- > From: Masayoshi Okutsu > Sent: Thursday, February 04, 2016 12:17 PM > To: Ramanand Patil; i18n-dev at openjdk.java.net > Cc: core-libs-dev at openjdk.java.net > Subject: Re: RFR: 8148446: (tz) Support tzdata2016a > > Hi Ramanand, > > I noticed that the zones in Yakutsk Time [1] seem to have their own names, such as "Khandyga Time" for Asia/Khandyga, and you seem to follow that convention for Asia/Chita. That causes some mismatch between the long names and abbreviations. > > I dag out some past tzdata fixes to see how that happened. What I found out is that the 2013b fix [2] used "Yakutsk Time", but that the 2013c [3] fix used "Khandyga Time". 2013b went to JDK 7 updates and earlier ones and 2013c went to 8. JDK 9 inherits the 8 fix. > > I prefer to restore the 2013b convention for Asia/Yakutsk, Asia/Chita, and Asia/Khandyga to have: > > {"Yakutsk Time", "YAKT", > "Yakutsk Summer Time", "YAKST", > "Yakutsk Time", "YAKT"} > > Thanks, > Masayoshi > > [1] https://en.wikipedia.org/wiki/Yakutsk_Time > [2] http://hg.openjdk.java.net/jdk7u/jdk7u-dev/jdk/rev/b8009df64dc8 > [3] http://hg.openjdk.java.net/jdk8/jdk8/jdk/rev/ae35fdbab949 > > On 2/2/2016 8:00 PM, Ramanand Patil wrote: >> HI all, >> >> Please review the latest TZDATA integration (tzdata2016a) to JDK9. >> >> Bug - https://bugs.openjdk.java.net/browse/JDK-8148446 >> >> Webrev - http://cr.openjdk.java.net/~rpatil/8148446/webrev.00/ >> >> >> >> All the TimeZone related tests are passed after integration. >> >> >> >> Regards, >> >> Ramanand. From uschindler at apache.org Sun Feb 7 23:25:21 2016 From: uschindler at apache.org (Uwe Schindler) Date: Mon, 8 Feb 2016 00:25:21 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B7CBBE.3070007@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B7CBBE.3070007@gmail.com> Message-ID: <002001d161fe$d2753860$775fa920$@apache.org> Hi Peter, as discussed before in the other thread about move to jdk.internal: this looks fine to Apache Lucene. We have a separate issue to fix this for Java 9: https://issues.apache.org/jira/browse/LUCENE-6989 Currently the patch on the Lucene issue tries to cast the jdk.internal.Cleaner to Runnable; but with your patch, one would just need to cast to java.lang.ref.Cleaner$Cleanable (which is public anyways) and call clean(). The Runnable workaround was done for Lucene, Hadoop, and Netty, but with your current patch this extra hack will be obsolete: http://cr.openjdk.java.net/~chegar/8148117/src/java.base/share/classes/jdk/internal/ref/Cleaner.java.udiff.html So, I am fine with both solutions as workaround, if we can enforce unmapping - until an official and "safe" solution is found. I was discussing with Andrew Haley and Mark Reinhold on FOSDEM about an "official way" to unmap ByteBuffers and there seems to be a really cool idea to make MappedByteBuffer/DirectByteBuffer implement Closeable, so unmapping may work without the horrible performance degradion on every access by using some extra safepoint and changed volatile semantics using a annotation in Hotspot. Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Peter Levart > Sent: Sunday, February 07, 2016 11:57 PM > To: Jeremy Manson > Cc: Core-Libs-Dev > Subject: Re: We don't need jdk.internal.ref.Cleaner any more > > > > On 02/07/2016 11:20 PM, Peter Levart wrote: > > > > > > On 02/07/2016 08:01 PM, Jeremy Manson wrote: > >> Hadoop seems to use sun.misc.Cleaner: > >> > >> > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/ha > doop-common/2.7.1/org/apache/hadoop/crypto/CryptoStreamUtils.java > >> > >> So you may want to keep it around transitionally (a la Unsafe). > > > > JEP 260 [1] was listing sun.misc.Cleaner as a critical API, but > > recently, it was decided to "hide" it away into jdk.internal.ref > > package which will not be exported [2]. The reasoning was this: > > > > "sun.misc.Cleaner ( was previously listed as a critical internal API, > > but on further investigation has been moved to an open issue, for the > > following reasons: 1) its primary use in the JDK is within NIO direct > > buffers to release native memory. The base module cannot have a > > dependency on jdk.unsupported so will need to be updated to use an > > alternative cleaner, 2) the usage of Cleaner outside the JDK, as > > determined by corpus analysis, has largely been observed to hack into > > private fields of the internal NIO direct buffer classes to explicitly > > release native memory. As stated in 1), the type of the cleaner used > > by NIO direct buffers will have to change. Given this, and the fact > > that JDK 9 has a new general purposed cleaner API, > > java.lang.ref.Cleaner, the value of keep sun.misc.Cleaner is > > questionable." > > > > If the decision to remove sun.misc.Cleaner was partly influenced by > > the desire to maintain just 2 instead of 3 Cleaner(s), then my > > proposal to migrate JDK code to the public API might enable Oracle to > > reconsider keeping sun.misc.Cleaner. > > OTOH, what hadoop is doing is exactly the usage described in the above > reasoning for removing sun.misc.Cleaner. > > Hadoop use case: > > public static void freeDB(ByteBuffer buffer) { > if (buffer instanceof sun.nio.ch.DirectBuffer) { > final sun.misc.Cleaner bufferCleaner = > ((sun.nio.ch.DirectBuffer) buffer).cleaner(); > bufferCleaner.clean(); > } > } > > can be rewritten using reflection to be compatible with either > sun.misc.Cleaner (on JDK 8 or less) or java.lang.ref.Cleaner$Cleanable > (on JDK 9 or more): > > static final Method cleanMethod; > > static { > try { > Class cleanerOrCleanable; > try { > cleanerOrCleanable = Class.forName("sun.misc.Cleaner"); > } catch (ClassNotFoundException e1) { > try { > cleanerOrCleanable = > Class.forName("java.lang.ref.Cleaner$Cleanable"); > } catch (ClassNotFoundException e2) { > e2.addSuppressed(e1); > throw e2; > } > } > cleanMethod = cleanerOrCleanable.getDeclaredMethod("clean"); > } catch (Exception e) { > throw new Error(e); > } > } > > public static void freeDB_JDK8or9(ByteBuffer buffer) { > if (buffer instanceof sun.nio.ch.DirectBuffer) { > final Object bufferCleaner = > ((sun.nio.ch.DirectBuffer) buffer).cleaner(); > > try { > if (bufferCleaner != null) { > cleanMethod.invoke(bufferCleaner); > } > } catch (InvocationTargetException | IllegalAccessException > e) { > throw new Error(e); > } > } > } > > > I thought about this use case and even kept the name of the method on > sun.nio.ch.DirectBuffer::cleaner to make things easier although the > return type in my patched DirectBufer is called > java.lang.ref.Cleaner$Cleanable. > > > Regards, Peter > > > > > [1] https://bugs.openjdk.java.net/browse/JDK-8132928 > > [2] https://bugs.openjdk.java.net/browse/JDK-8148117 > > > > Regards, Peter > > > >> > >> Jeremy > >> > >> On Sun, Feb 7, 2016 at 2:53 AM, Peter Levart >> > wrote: > >> > >> Hi, > >> > >> sun.misc.Cleaner has been moved to internal package > >> jdk.internal.ref recently [1] to clean-up sun.misc namespace. But > >> now that: > >> > >> - we have comparable public API (java.lang.ref.Cleaner & > >> Cleanable) [2] > >> - we have an internal shared java.lang.ref.Cleaner instance > >> (jdk.internal.ref.CleanerFactory.cleaner()) > >> - sun.misc.Cleaner is not a special kind of Reference any more in > >> the JVM [3] > >> > >> ...I think there's no reason to keep this special internal API > >> any more. It can be replaced with public API. > >> > >> I propose to remove jdk.internal.ref.Cleaner class and replace > >> its usages with java.lang.ref.Cleaner and friends [4]. > >> > >> What do you say? > >> > >> Regards, Peter > >> > >> > >> [1] https://bugs.openjdk.java.net/browse/JDK-8148117 > >> [2] https://bugs.openjdk.java.net/browse/JDK-8138696 > >> [3] https://bugs.openjdk.java.net/browse/JDK-8143847 > >> [4] > >> http://cr.openjdk.java.net/~plevart/jdk9- > dev/removeInternalCleaner/webrev.01/ > >> dev/removeInternalCleaner/webrev.01/> > >> > >> > >> > >> > >> > >> > >> > > From david.holmes at oracle.com Mon Feb 8 00:14:16 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 8 Feb 2016 10:14:16 +1000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B5207B.8010107@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> <56B5207B.8010107@oracle.com> Message-ID: <56B7DDD8.2050701@oracle.com> On 6/02/2016 8:21 AM, Mikael Vidstedt wrote: > > I fully agree that moving the arguments checking up to Java makes more > sense, and I've prepared new webrevs which do exactly that, including > changes to address the other feedback from David, John and others: Shouldn't the lowest-level do_conjoint_swap routines at least check preconditions with asserts to catch the cases where the calling Java code has failed to do the right thing? The other Copy methods seem to do this. David ----- > hotspot: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/hotspot/webrev/ > > jdk: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/jdk/webrev/ > > Incremental webrevs for your convenience: > > hotspot: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/hotspot/webrev/ > > jdk: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/jdk/webrev/ > > > > I have done some benchmarking of this code and for large copies (16MB+) > this outperforms the old Bits.c implementation by *30-100%* depending on > platform and exact element sizes! For smaller copies the additional > checks which are now performed hurt performance on client VMs (80-90% of > old impl), but with the server VMs I see performance on par with, or in > most cases 5-10% better than the old implementation. There's a > potentially statistically significant regression of ~3-4% for > elemSize=2, but for now I'm going to declare success. There's certainly > room for further improvements here, but this should at least do for > addressing the original problem. > > > I filed https://bugs.openjdk.java.net/browse/JDK-8149159 for moving the > checks for Unsafe.copyMemory to Java, and will work on that next. I also > filed https://bugs.openjdk.java.net/browse/JDK-8149162 to cover the > potential renaming of the Bits methods to have more informative names. > Finally, I filed https://bugs.openjdk.java.net/browse/JDK-8149163 to > look at improving the behavior of Unsafe.addressSize(), after having > spent too much time trying to understand why the performance of the new > U.copySwapMemory Java checks wasn't quite living up to my expectations > (spoiler alert: Unsafe.addressSize() is not intrinsified, so will always > result in a call into the VM/unsafe.cpp). > > > Finally, I - too - would like to see the copy-swap logic moved into > Java, and as I mentioned I played around with that first before I > decided to do the native implementation to address the immediate > problem. Looking forward to what you find Paul! > > Cheers, > Mikael > > On 2016-02-05 05:00, Paul Sandoz wrote: >> Hi, >> >> Nice use of C++ templates :-) >> >> Overall looks good. >> >> I too would prefer if we could move the argument checking out, perhaps >> even to the point of requiring callers do that rather than providing >> another method, for example for Buffer i think the arguments are known >> to be valid? I think in either case it is important to improve the >> documentation on the method stating the constraints on arguments, >> atomicity guarantees etc. >> >> I have a hunch that for the particular case of copying-with-swap for >> buffers i could get this to work work efficiently using Unsafe (three >> separate methods for each unit type of 2, 4 and 8 bytes), since IIUC >> the range is bounded to be less than Integer.MAX_VALUE so an int loop >> rather than a long loop can be used and therefore safe points checks >> will not be placed within the loop. >> >> However, i think what you have done is more generally applicable and >> could be made intrinsic. It would be a nice at some future point if it >> could be made a pure Java implementation and intrinsified where >> appropriate. >> >> ? >> >> John, regarding array mismatch there were issues with the efficiency >> of the unrolled loops with Unsafe access. (Since the loops were int >> bases there were no issues with safe point checks.) Roland recently >> fixed that so now code is generated that is competitive with direct >> array accesses. We drop into the stub intrinsic and leverage 128bits >> or 256bits where supported. Interestingly it seems the unrolled loop >> using Unsafe is now slightly faster than the stub using 128bit >> registers. I don?t know if that is due to unluckly alignment, and/or >> the stub needs to do some manual unrolling. In terms of code-cache >> efficiency the intrinsic is better. >> >> Paul. >> >> >> >> >> >>> On 4 Feb 2016, at 06:27, John Rose wrote: >>> >>> On Feb 2, 2016, at 11:25 AM, Mikael Vidstedt >>> wrote: >>>> Please review this change which introduces a Copy::conjoint_swap and >>>> an Unsafe.copySwapMemory method to call it from Java, along with the >>>> necessary changes to have java.nio.Bits call it instead of the >>>> Bits.c code. >>>> >>>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ >>>> >>>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ >>>> >>> This is very good. >>> >>> I have some nit-picks: >>> >>> These days, when we introduce a new intrinsic (@HSIntrCand), >>> we write the argument checking code separately in a non-intrinsic >>> bytecode method. In this case, we don't (yet) have an intrinsic >>> binding for U.copy*, but we might in the future. (C intrinsifies >>> memcpy, which is a precedent.) In any case, I would prefer >>> if we could structure the argument checking code in a similar >>> way, with Unsafe.java containing both copySwapMemory >>> and a private copySwapMemory0. Then we can JIT-optimize >>> the safety checks. >>> >>> You might as well extend the same treatment to the pre-existing >>> copyMemory call. The most important check (and the only one >>> in U.copyMemory) is to ensure that the size_t operand has not >>> wrapped around from a Java negative value to a crazy-large >>> size_t value. That's the low-hanging fruit. Checking the pointers >>> (for null or oob) is more problematic, of course. Checking consistency >>> around elemSize is cheap and easy, so I agree that the U.copySM >>> should do that work also. Basically, Unsafe can do very basic >>> checks if there is a tricky user model to enforce, but it mustn't >>> "sign up" to guard the user against all errors. >>> >>> Rule of thumb: Unsafe calls don't throw NPEs, they just SEGV. >>> And the rare bit that *does* throw (IAE usually) should be placed >>> into Unsafe.java, not unsafe.cpp. (The best-practice rule for putting >>> argument checking code outside of the intrinsic is a newer one, >>> so Unsafe code might not always do this.) >>> >>> The comment "Generalizing it would be reasonable, but requires >>> card marking" is bogus, since we never byte-swap managed pointers. >>> >>> The test logic will flow a little smoother if your GenericPointer guy, >>> the onHeap version, stores the appropriate array base offset in his >>> offset field. >>> You won't have to mention p.isOnHeap nearly so much, and the code will >>> set a slightly better example. >>> >>> The VM_ENTRY_BASE_FROM_LEAF macro is really cool. >>> >>> The C++ template code is cool also. It reminds me of the kind >>> of work Gosling's "Ace" processor could do, but now it's mainstreamed >>> for all to use in C++. We're going to get some of that goodness >>> in Project Valhalla with specialization logic. >>> >>> I find it amazing that the right way to code this in C is to >>> use memcpy for unaligned accesses and byte peek/poke >>> into registers for byte-swapping operators. I'm glad we >>> can write this code *once* for the JVM and JDK. >>> >>> Possible future work: If we can get a better handle on >>> writing vectorizable loops from Java, including Unsafe-based >>> ones, we can move some of the C code back up to Java. >>> Perhaps U.copy* calls for very short lengths deserved to >>> be broken out into small loops of U.get/put* (with alignment). >>> I think you experimented with this, and there were problems >>> with the JIT putting fail-safe memory barriers between >>> U.get/put* calls. Paul's work on Array.mismatch ran into >>> similar issues, with the right answer being to write manual >>> vector code in assembly. >>> >>> Anyway, you can count me as a reviewer. >>> >>> Thanks, >>> >>> ? John > From Alan.Bateman at oracle.com Mon Feb 8 06:27:31 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 8 Feb 2016 06:27:31 +0000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B7C328.3060800@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> Message-ID: <56B83553.3020202@oracle.com> On 07/02/2016 22:20, Peter Levart wrote: > : > > If the decision to remove sun.misc.Cleaner was partly influenced by > the desire to maintain just 2 instead of 3 Cleaner(s), then my > proposal to migrate JDK code to the public API might enable Oracle to > reconsider keeping sun.misc.Cleaner. The main issue driving this is the design principles that we have listed in JEP 200. We don't want a standard module (java.base in this case) exporting a non-standard API. This means surgery to jettison the sun.misc package from java.base and move it to its own module (jdk.unsupported is the proposal in JEP 260). This is painful of course but we are at least in good shape for the most critical internal API, sun.misc.Unsafe. For sun.misc.Cleaner then the original proposal was for it to be a critical internal API too but it become clear that changing the type of internal/private fields in the NIO buffer and channel classes would break libraries that have been hacking into those fields. If they are changing away then there seems little motive to keep sun.misc.Cleaner so Chris moved into into jdk.internal.ref for now. As to whether we even need to keep jdk.internal.ref.Cleaner then I think the only remaining question was whether the special handling of Cleaner in the Reference implementation was worth it or not. It may not be, in which case your current proposal to just remove seems right. -Alan. From stuart.marks at oracle.com Mon Feb 8 07:02:31 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Sun, 7 Feb 2016 23:02:31 -0800 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B54457.1000900@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> <56B54457.1000900@oracle.com> Message-ID: <56B83D87.7030908@oracle.com> On 2/5/16 4:54 PM, David Holmes wrote: > Regardless of whether I agree with this API or not, it does, as Stuart points > out, require a JEP and to go through the normal rigorous process of determining > whether an API is suitable for inclusion in the Java platform. Note, it was Thomas St?fe who suggested a JEP for this. s'marks From chris.hegarty at oracle.com Mon Feb 8 09:16:54 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 09:16:54 +0000 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B83D87.7030908@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> <56B54457.1000900@oracle.com> <56B83D87.7030908@oracle.com> Message-ID: On 8 Feb 2016, at 07:02, Stuart Marks wrote: > On 2/5/16 4:54 PM, David Holmes wrote: >> Regardless of whether I agree with this API or not, it does, as Stuart points >> out, require a JEP and to go through the normal rigorous process of determining >> whether an API is suitable for inclusion in the Java platform. > > Note, it was Thomas St?fe who suggested a JEP for this. Here is my take on the situation: - The proposal appears to be a relatively small API, and, in my opinion, does not meet any of the criteria to be considered under the JEP process. - The proposal is for a low-level API. It is not expected to be widely used. Maybe a less prominent position in the package hierarchy would be less offensive? - This is not strictly JEP 260 related, but the proposal has been, somewhat, motivated by the fact that 260 will have to make changes in this area. ( Roger has already proven in his webrevs that sun.misc Signal can be implemented by a purely private signal implementation ) - I would ague that this proposed API is in the spirit of Java ( Java-like ), since a Java process may be able send a signal, through the process API [1] , and then handle that signal through this API. It is safe ( does NOT crash) , implementable ( best-effort on different platforms ). Accepting the platform limitations, it is still possible that one Java process can send a signal to another Java process, on the same system. This seems useful. -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-4914493 From paul.sandoz at oracle.com Mon Feb 8 09:26:37 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 8 Feb 2016 10:26:37 +0100 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <1489632171.20160206192945@gmail.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> <1489632171.20160206192945@gmail.com> Message-ID: <6A23376A-20E8-4717-B979-26D698E8DB29@oracle.com> > On 6 Feb 2016, at 14:29, Tagir F. Valeev wrote: > > Hello! > > PS> I still disagree and pushing back on the support for splitting > PS> after partial traversal. It?s not a pattern i think we should > PS> encourage and propagate so such behaviour can be generally relied > PS> upon, and predominantly for edge cases. That?s where the > PS> complexity string is being pulled on. While your fix in isolation > PS> is not terribly complex, it is more complex than the alternative > PS> (which was actually the intent of the current impl, we just forget to include the check). > > I still don't like doing this, but as Brian agreed with you [1], seems > I have no other choice. Thanks for accommodating. > Here's updated webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r3/ > 188 public Spliterator trySplit() { 189 if (isParallel && !finished) { 190 init(); 191 192 if (buffer != null && buffer.count() > 0) // partial traversal started 193 return null; Why don?t you check if "buffer == null? at #189? i.e. similar to forEachRemaining: @Override public void forEachRemaining(Consumer consumer) { if (buffer == null && !finished) { For clarity and consistency we should key off the "buffer == null? partial traversal guard. The state of "buffer != null" and ?buffer.count() == 0? will be when traversal has completed i.e. "finished == true" (see fillBuffer). Paul. From chris.hegarty at oracle.com Mon Feb 8 09:33:35 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 09:33:35 +0000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B83553.3020202@oracle.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> Message-ID: On 8 Feb 2016, at 06:27, Alan Bateman wrote: > > On 07/02/2016 22:20, Peter Levart wrote: >> : >> >> If the decision to remove sun.misc.Cleaner was partly influenced by the desire to maintain just 2 instead of 3 Cleaner(s), then my proposal to migrate JDK code to the public API might enable Oracle to reconsider keeping sun.misc.Cleaner. > > The main issue driving this is the design principles that we have listed in JEP 200. We don't want a standard module (java.base in this case) exporting a non-standard API. This means surgery to jettison the sun.misc package from java.base and move it to its own module (jdk.unsupported is the proposal in JEP 260). This is painful of course but we are at least in good shape for the most critical internal API, sun.misc.Unsafe. > > For sun.misc.Cleaner then the original proposal was for it to be a critical internal API too but it become clear that changing the type of internal/private fields in the NIO buffer and channel classes would break libraries that have been hacking into those fields. If they are changing away then there seems little motive to keep sun.misc.Cleaner so Chris moved into into jdk.internal.ref for now. As to whether we even need to keep jdk.internal.ref.Cleaner then I think the only remaining question was whether the special handling of Cleaner in the Reference implementation was worth it or not. It may not be, in which case your current proposal to just remove seems right. Alan?s summary of the situation is spot on. Before moving sun.misc.Cleaner to jdk.internal.ref, I did enquire if it would be possible to just remove it and use the new public Cleaner, but I got feedback that there were some issues with failing NIO tests ( it appeared that Cleaners were not running as quickly as possible ). I didn?t look further into that at the time, since the VM still had special knowledge of these cleaners, but as you say this is now removed. It is probably a good time to revisit this. -Chris. From chris.hegarty at oracle.com Mon Feb 8 09:35:20 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 09:35:20 +0000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> Message-ID: <3B811DCE-8A04-4EB0-9DA0-8477B83D9D8F@oracle.com> On 7 Feb 2016, at 19:01, Jeremy Manson wrote: > Hadoop seems to use sun.misc.Cleaner: > > http://grepcode.com/file/repo1.maven.org/maven2/org.apache.hadoop/hadoop-common/2.7.1/org/apache/hadoop/crypto/CryptoStreamUtils.java There is an issue in the Hadoop JIRA tracking this: https://issues.apache.org/jira/browse/HADOOP-12760 -Chris. > So you may want to keep it around transitionally (a la Unsafe). > > Jeremy > > On Sun, Feb 7, 2016 at 2:53 AM, Peter Levart wrote: > >> Hi, >> >> sun.misc.Cleaner has been moved to internal package jdk.internal.ref >> recently [1] to clean-up sun.misc namespace. But now that: >> >> - we have comparable public API (java.lang.ref.Cleaner & Cleanable) [2] >> - we have an internal shared java.lang.ref.Cleaner instance >> (jdk.internal.ref.CleanerFactory.cleaner()) >> - sun.misc.Cleaner is not a special kind of Reference any more in the JVM >> [3] >> >> ...I think there's no reason to keep this special internal API any more. >> It can be replaced with public API. >> >> I propose to remove jdk.internal.ref.Cleaner class and replace its usages >> with java.lang.ref.Cleaner and friends [4]. >> >> What do you say? >> >> Regards, Peter >> >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8148117 >> [2] https://bugs.openjdk.java.net/browse/JDK-8138696 >> [3] https://bugs.openjdk.java.net/browse/JDK-8143847 >> [4] >> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.01/ >> >> >> >> >> >> >> From sean.coffey at oracle.com Mon Feb 8 09:42:22 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 8 Feb 2016 09:42:22 +0000 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56B4F031.6040501@oracle.com> References: <56B4F031.6040501@oracle.com> Message-ID: <56B862FE.5000000@oracle.com> Is there an option to fall back to the older v.1.2.8 implementation if necessary ? It would certainly alleviate issues for any applications that might run into issues with the latest and greatest zlib libraries. JDK-8133206 would be one such example. Regards, Sean. On 05/02/2016 18:55, Xueming Shen wrote: > Hi > > Please help codereview the change to build the jdk9 runtime to use the > system zlib on > Solaris and Linux platforms by default. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8031767 > Webrev: http://cr.openjdk.java.net/~sherman/8031767/webrev/ > > Background info: > > Compression is heavily used in Java based big data/middle-ware > applications. > There are many products in market today that help compression performance > either through software or hardware acceleration and most likely these > products > support the zlib interface as API, for example Intel's IPP library has > a faster > version of compression libraries. To configure the Java runtime to use > the system > zlib would make these acceleration capabilities available to java > users through > java.util.zip package directly. The jdk already has a build > configuration option > to build the jdk to use the system zlib via "--with-zlib=system" and > the OSX is > by default built to use the system zlib. This proposal is to propose > to build > the jdk to use the system zlib library (the zlib bundled by the > underlying Solaris/ > Linuxplatforms), instead of the binary built from source code jdk > repository > (current 1.2.8 from the open source zlib.org) > > Thanks, > Sherman > > > btw, attached is the similar change in the closed repo: > autoconf/generated-configure.sh > ------------------------------------------------------------- > > # Do not change or remove the following line, it is needed for > consistency checks: > -DATE_WHEN_GENERATED=1454436146 > +DATE_WHEN_GENERATED=1454626552 > > ############################################################################### > > # > # Initialization / Boot-strapping > # > > ------------------------------------------------------------------------ > > @@ -58839,14 +58839,14 @@ > > > { $as_echo "$as_me:${as_lineno-$LINENO}: checking for which zlib to > use" >&5 > $as_echo_n "checking for which zlib to use... " >&6; } > > - DEFAULT_ZLIB=bundled > - if test "x$OPENJDK_TARGET_OS" = xmacosx; then > - # On macosx default is system...on others default is bundled > DEFAULT_ZLIB=system > + if test "x$OPENJDK_TARGET_OS" = xwindows; then > + # On windows default is bundled...on others default is system > + DEFAULT_ZLIB=bundled > fi > > if test "x${ZLIB_FOUND}" != "xyes"; then > # If we don't find any system...set default to bundled > DEFAULT_ZLIB=bundled > > From Alan.Bateman at oracle.com Mon Feb 8 09:55:47 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 8 Feb 2016 10:55:47 +0100 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56B862FE.5000000@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> Message-ID: <56B86623.4090602@oracle.com> On 08/02/2016 10:42, Se?n Coffey wrote: > Is there an option to fall back to the older v.1.2.8 implementation if > necessary ? It would certainly alleviate issues for any applications > that might run into issues with the latest and greatest zlib > libraries. JDK-8133206 would be one such example. Just at build time but if the zlib on the platform is broken then it impacts tools and probably lots of things. I would assume the OS would patch such issues quickly. In the case of JDK-8133206 then was the issue addressed in the libzip wrapper or in the zlib code? I thought it was the former. On a fallback or some way to configure at launch time then Sandhya Viswanathan (Intel) has a proposal here last year. I think we mostly agreed on that thread that switching the build to use the system zlib by default would be better. -Alan From Alan.Bateman at oracle.com Mon Feb 8 10:06:50 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 8 Feb 2016 11:06:50 +0100 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B83D87.7030908@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> <56B54457.1000900@oracle.com> <56B83D87.7030908@oracle.com> Message-ID: <56B868BA.1000707@oracle.com> On 08/02/2016 07:02, Stuart Marks wrote: > > > On 2/5/16 4:54 PM, David Holmes wrote: >> Regardless of whether I agree with this API or not, it does, as >> Stuart points >> out, require a JEP and to go through the normal rigorous process of >> determining >> whether an API is suitable for inclusion in the Java platform. > > Note, it was Thomas St?fe who suggested a JEP for this. It's a good question. It is an explicit non-goal of JEP 260 to propose replacements of internal APIs. Providing a standard API for handling control-C or OS signals is of course strongly connected with the efforts in JEP 260 but isn't a goal. The JEP process provides guidelines as to when a JEP should be drafted. In this case it seems like the API is significant and that a JEP would make be very useful. If nothing else, the JEP could document alternatives considered, like for example a specific API for handling control-C/VM termination and other specific use-cases as opposed to exposing OS signals to applications. -Alan From paul.sandoz at oracle.com Mon Feb 8 10:09:46 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 8 Feb 2016 11:09:46 +0100 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: Message-ID: > On 1 Feb 2016, at 19:45, Martin Buchholz wrote: > > After much debate on what to do when CompleteableFuture.whenComplete > encounters an exception in both the source and the action, we chose > what was acceptable to the most people - add the action's exception to > the source exception as a suppressed exception. And added usage > guidelines. And gave handle "top billing" over whenComplete. > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ > > and the usual miscellaneous improvements. +1 Looks like the diffs got swapped for EmptyNavigableSet.java in the misc patch: http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/patches/jsr166-jdk9-integration-miscellaneous diff --git a/test/java/util/Collections/EmptyNavigableSet.java b/test/java/util/Collections/EmptyNavigableSet.java --- a/test/java/util/Collections/EmptyNavigableSet.java +++ b/test/java/util/Collections/EmptyNavigableSet.java @@ -284,7 +284,7 @@ navigableSet.subSet(last, true, first, false); }, IllegalArgumentException.class, description - + ": Must throw IllegalArgumentException when fromElement is not less than toElement."); + + ": Must throw IllegalArgumentException when fromElement is not less then then toElement.?); Paul. From peter.levart at gmail.com Mon Feb 8 10:46:11 2016 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 8 Feb 2016 11:46:11 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B83553.3020202@oracle.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> Message-ID: <56B871F3.8070504@gmail.com> On 02/08/2016 07:27 AM, Alan Bateman wrote: > > On 07/02/2016 22:20, Peter Levart wrote: >> : >> >> If the decision to remove sun.misc.Cleaner was partly influenced by >> the desire to maintain just 2 instead of 3 Cleaner(s), then my >> proposal to migrate JDK code to the public API might enable Oracle to >> reconsider keeping sun.misc.Cleaner. > > The main issue driving this is the design principles that we have > listed in JEP 200. We don't want a standard module (java.base in this > case) exporting a non-standard API. This means surgery to jettison the > sun.misc package from java.base and move it to its own module > (jdk.unsupported is the proposal in JEP 260). This is painful of > course but we are at least in good shape for the most critical > internal API, sun.misc.Unsafe. > > For sun.misc.Cleaner then the original proposal was for it to be a > critical internal API too but it become clear that changing the type > of internal/private fields in the NIO buffer and channel classes would > break libraries that have been hacking into those fields. If they are > changing away then there seems little motive to keep sun.misc.Cleaner > so Chris moved into into jdk.internal.ref for now. As to whether we > even need to keep jdk.internal.ref.Cleaner then I think the only > remaining question was whether the special handling of Cleaner in the > Reference implementation was worth it or not. It may not be, in which > case your current proposal to just remove seems right. > > -Alan. The special-casing of sun.misc.Cleaner in ReferenceHandler thread is there to avoid bringing another thread into the processing pipeline, right? Presumably to optimize the latency of clean-up operations. But it seems that introducing another thread to the pipeline is not the source of problematic latency when cleaning-up direct/mapped byte buffers. The source of latency is the strategy of generational GC which promptly discovers and enqueues young References with young referents, but is not so prompt with old References having old referents. If special-casing in ReferenceHandler is removed then an opportunity opens to get rid of ReferenceHandler thread altogether. VM/GC could enqueue Reference(s) to their respective ReferenceQueue(s) directly, so everyone could enjoy the benefits sun.misc.Cleaner has... Regards, Peter From chris.hegarty at oracle.com Mon Feb 8 10:52:39 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 10:52:39 +0000 Subject: RFR [9] 8148861: Remove sun.misc.ManagedLocalsThread from jaxws Message-ID: Miran, JDK-8056152 added a new constructor to java.lang.Thread for constructing Threads that do not inherit inheritable thread-local initial values. All usages of sun.misc.ManagedLocalsThread in the base module were also replaced with this new constructor. Given there is now a supported API for creating such threads, other areas of the JDK should be updated to use it, where applicable. If all usages are replaced, then sun.misc.ManagedLocalsThread can be removed. The following patch, along with 'hg rm ThreadHelper.java', should be sufficient. It will remove the dependency on an internal JDK API, but adds a new dependency on Java SE 9. Is this OK, or does this code need to access the new Java SE 9 constructor using refection? diff --git a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java --- a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java +++ b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java @@ -108,7 +108,8 @@ } public Thread newThread(Runnable r) { - Thread t = ThreadHelper.createNewThread(r); - t.setName(namePrefix + threadNumber.getAndIncrement()); + Thread t = new Thread(null, r, namePrefix + threadNumber.getAndIncrement(), 0, false); t.setName(namePrefix + threadNumber.getAndIncrement()); if (!t.isDaemon()) { t.setDaemon(true); Complete webrev: http://cr.openjdk.java.net/~chegar/8148861/ Is is possible, when agreed, to get this small change pushed upstream ? -Chris. From peter.levart at gmail.com Mon Feb 8 10:58:34 2016 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 8 Feb 2016 11:58:34 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> Message-ID: <56B874DA.80001@gmail.com> On 02/08/2016 10:33 AM, Chris Hegarty wrote: > On 8 Feb 2016, at 06:27, Alan Bateman wrote: > >> On 07/02/2016 22:20, Peter Levart wrote: >>> : >>> >>> If the decision to remove sun.misc.Cleaner was partly influenced by the desire to maintain just 2 instead of 3 Cleaner(s), then my proposal to migrate JDK code to the public API might enable Oracle to reconsider keeping sun.misc.Cleaner. >> The main issue driving this is the design principles that we have listed in JEP 200. We don't want a standard module (java.base in this case) exporting a non-standard API. This means surgery to jettison the sun.misc package from java.base and move it to its own module (jdk.unsupported is the proposal in JEP 260). This is painful of course but we are at least in good shape for the most critical internal API, sun.misc.Unsafe. >> >> For sun.misc.Cleaner then the original proposal was for it to be a critical internal API too but it become clear that changing the type of internal/private fields in the NIO buffer and channel classes would break libraries that have been hacking into those fields. If they are changing away then there seems little motive to keep sun.misc.Cleaner so Chris moved into into jdk.internal.ref for now. As to whether we even need to keep jdk.internal.ref.Cleaner then I think the only remaining question was whether the special handling of Cleaner in the Reference implementation was worth it or not. It may not be, in which case your current proposal to just remove seems right. > Alan?s summary of the situation is spot on. > > Before moving sun.misc.Cleaner to jdk.internal.ref, I did enquire if it would be > possible to just remove it and use the new public Cleaner, but I got feedback > that there were some issues with failing NIO tests ( it appeared that Cleaners > were not running as quickly as possible ). I didn?t look further into that at the > time, since the VM still had special knowledge of these cleaners, but as you > say this is now removed. It is probably a good time to revisit this. > > -Chris. Hi Chris, Are you referring to the following test: test/java/nio/Buffer/DirectBufferAllocTest.java ? This test was written specifically for the following issue: https://bugs.openjdk.java.net/browse/JDK-6857566 ...which exercises multi-threaded allocation of direct buffers. Even sun.misc.Cleaner was not quick enough to promptly deallocate them, so allocation path was patched to help ReferenceHandler thread while re-attempting to allocate new direct buffers. Transitioning to java.lang.ref.Cleaner which uses additional thread to process Cleanable(s), direct-buffer allocation path must help the Cleaner thread too. See: http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/src/java.base/share/classes/java/nio/Bits.java.sdiff.html ...for the place where this is performed... Regards, Peter From chris.hegarty at oracle.com Mon Feb 8 11:15:49 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 11:15:49 +0000 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length Message-ID: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> Low hanging fruit to avoid unnecessary allocations when deserializing. readUTF knows the string size ahead of the read and can avoid expandCapacity() by constructing the StringBuilder with the expected size. It is an implementation detail, but if the size is larger than Integer.MAX_VALUE, then OOM can be thrown, as is the case in the implementation today. diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java @@ -3144,7 +3144,9 @@ * utflen bytes. */ private String readUTFBody(long utflen) throws IOException { - StringBuilder sbuf = new StringBuilder(); + if (utflen > Integer.MAX_VALUE) + throw new OutOfMemoryError("UTF length, " + utflen + ", too big."); + StringBuilder sbuf = new StringBuilder((int)utflen); if (!blkmode) { end = pos = 0; } -Chris. From shilpi.rastogi at oracle.com Mon Feb 8 13:15:48 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Mon, 08 Feb 2016 18:45:48 +0530 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <5349AA5B-0198-4B25-A42A-E63146CBA266@oracle.com> References: <56A21316.1070007@oracle.com> <56AB5E6C.1010201@oracle.com> <5349AA5B-0198-4B25-A42A-E63146CBA266@oracle.com> Message-ID: <56B89504.6010902@oracle.com> Gentle Reminder! On 2/5/2016 6:48 PM, Paul Sandoz wrote: >> On 29 Jan 2016, at 12:43, shilpi rastogi wrote: >> >> Hi All, >> >> Please review the updated patch- >> >> http://cr.openjdk.java.net/~srastogi/8138578/webrev.01/ >> >> I verified Lookup.unreflectSpecial() also throws java.lang.IllegalAccessException so updated the JavaDoc. >> > +1 > > Paul. From shilpi.rastogi at oracle.com Mon Feb 8 13:18:49 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Mon, 08 Feb 2016 18:48:49 +0530 Subject: RFR: 8071368 Use more concrete types for NamedFunction constants in the code In-Reply-To: <56B4C467.7040300@oracle.com> References: <56B34F40.1010800@oracle.com> <6804C2B3-DE2F-4DCF-B70D-6F3BC846B403@oracle.com> <56B4C467.7040300@oracle.com> Message-ID: <56B895B9.40401@oracle.com> Gentle Reminder! On 2/5/2016 9:18 PM, Vladimir Ivanov wrote: > Proposed fix looks good. > > Quoting John: "The use of erased types (any ref => Object) in the MH > runtime is an artifact of bootstrapping difficulties, early in the > project. I hope it is not necessary any more." > > Best regards, > Vladimir Ivanov > > On 2/5/16 4:29 PM, Paul Sandoz wrote: >> Hi Shilpi, >> >> In principle this change is ok, however these methods are kind of >> special because they are called by code generated from LambdaForms >> with erased types. >> >> IIUC the casts can serve as belts-and-braces checks just in case that >> generated code passes something that is not of the correct type. By >> removing the casts it?s possible to erroneously pass in something of >> another type that the runtime treats erroneously as if a MethodHandle >> or MethodType, and then strange things happen. The casts will >> ordinary be removed by the runtime compiler. By removing them from >> the Java code it might slightly improve C2 compilation time. >> >> Approval from Vladimir/John would be good. >> >> Paul. >> >>> On 4 Feb 2016, at 13:16, shilpi rastogi >>> wrote: >>> >>> Hi All, >>> >>> Please review the following- >>> >>> https://bugs.openjdk.java.net/browse/JDK-8071368 >>> http://cr.openjdk.java.net/~srastogi/8071368/webrev.02/ >>> >>> Thanks, >>> Shilpi >> From vladimir.x.ivanov at oracle.com Mon Feb 8 13:26:04 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 8 Feb 2016 16:26:04 +0300 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <56AB5E6C.1010201@oracle.com> References: <56A21316.1070007@oracle.com> <56AB5E6C.1010201@oracle.com> Message-ID: <56B8976C.2010808@oracle.com> Looks good. Best regards, Vladimir Ivanov On 1/29/16 3:43 PM, shilpi rastogi wrote: > Hi All, > > Please review the updated patch- > > http://cr.openjdk.java.net/~srastogi/8138578/webrev.01/ > > I verified Lookup.unreflectSpecial() also throws > java.lang.IllegalAccessException so updated the JavaDoc. > > Thanks, > Shilpi > >> >> -------- Original Message -------- >> Subject: Re: RFR 8138578:MethodHandles.Lookup.findSpecial() >> Javadoc fails to consider static methods >> Date: Fri, 22 Jan 2016 10:30:29 +0100 >> From: Paul Sandoz >> CC: core-libs-dev at openjdk.java.net >> >> >> >> Hi Shilpi, >> >> Can you also double check Lookup.unreflectSpecial, it?s documentation >> might require updating too. >> >> Separating the ?or? statements with a comma would help readability in >> the JavaDoc (same applies to the the patch for findVirtual). >> >> Paul. >> >> > On 22 Jan 2016, at 10:22, shilpi rastogi >> wrote: >> > >> > >> > Gentle Reminder! >> > >> > -------- Original Message -------- >> > Subject: [9u-dev] 8138578:MethodHandles.Lookup.findSpecial() >> Javadoc fails to consider static methods >> > Date: Wed, 20 Jan 2016 18:12:30 +0530 >> > From: shilpi rastogi >> > Organization: Oracle Corporation >> > To: core-libs-dev at openjdk.java.net >> > >> > >> > >> > Hi All, >> > >> > Please review my fix related to java doc. >> > >> > MethodHandles.Lookup.findSpecial() Javadoc fails to consider static >> methods- >> > >> > MethodHandles.Lookup.findSpecial() throws IllegalAccessException if >> the target method is static. >> > >> > Bug link-https://bugs.openjdk.java.net/browse/JDK-8138578 >> >> > Webrev link-http://cr.openjdk.java.net/~srastogi/8138578/webrev.00/ >> >> > >> > Thnaks, >> > Shilpi >> > >> > >> >> >> > From magnus.ihse.bursie at oracle.com Mon Feb 8 13:42:18 2016 From: magnus.ihse.bursie at oracle.com (Magnus Ihse Bursie) Date: Mon, 8 Feb 2016 14:42:18 +0100 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56B50B01.60003@oracle.com> References: <56B4F031.6040501@oracle.com> <56B50B01.60003@oracle.com> Message-ID: <56B89B3A.6070002@oracle.com> On 2016-02-05 21:50, Alan Bateman wrote: > On 05/02/2016 18:55, Xueming Shen wrote: >> Hi >> >> Please help codereview the change to build the jdk9 runtime to use >> the system zlib on >> Solaris and Linux platforms by default. >> >> Issue: https://bugs.openjdk.java.net/browse/JDK-8031767 >> Webrev: http://cr.openjdk.java.net/~sherman/8031767/webrev/ Looks good to me. > I'm happy to see the default changed for Linux and Solaris builds. One > thing is that the patch to lib-bundled.m4 switches the build on all > non-Windows to use the system zlib. This may be something the SAP (AIX > port maintainers) might want to comment on. As a rule of thumb, almost everyone else than Oracle uses system libraries as far as possible. :) Also, changing the default is seldom a problem for anyone, as long as the old behavior can be achieved by a properly crafted configure command line. /Magnus > > -Alan. From amaembo at gmail.com Mon Feb 8 13:53:23 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Mon, 8 Feb 2016 19:53:23 +0600 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <6A23376A-20E8-4717-B979-26D698E8DB29@oracle.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> <1489632171.20160206192945@gmail.com> <6A23376A-20E8-4717-B979-26D698E8DB29@oracle.com> Message-ID: <701936850.20160208195323@gmail.com> Hello! PS> 188 public Spliterator trySplit() { PS> 189 if (isParallel && !finished) { PS> 190 init(); PS> 191 PS> 192 if (buffer != null && buffer.count() > 0) // partial traversal started PS> 193 return null; PS> Why don?t you check if "buffer == null? at #189? i.e. similar to forEachRemaining: That would make minimal behavioral change to fix this issue (fix flatMap keys only, but not affect other intermediate ops which were working correctly). Well, if buffer == null check is enough, here's update: http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r4/ (test unchanged) With best regards, Tagir Valeev. From paul.sandoz at oracle.com Mon Feb 8 13:55:24 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 8 Feb 2016 14:55:24 +0100 Subject: RFR-8148748: ArrayList.subList().spliterator() is not late-binding In-Reply-To: <534936148.20160204215527@gmail.com> References: <27476765.20160129103224@gmail.com> <564410418.20160202102813@gmail.com> <1084445298.20160203182024@gmail.com> <534936148.20160204215527@gmail.com> Message-ID: Hi Tagir, It?s certainly easier to review if one avoids the temptation to change other things at the same time :-) There is a possible simplified approach to sharing one Spliterator impl between ArrayList and ArrayList.SubList. The initial (origin, fence) for ArrayList can be (0, -1) and ArrayList.SubList it can be (offset, -1), as before, then one can make it static passing in an instance of AbstractList: if ((hi = fence) < 0) { expectedModCount = lst.modCount; // shared field on AbstractList hi = fence = index + lst.size(); // virtual call :-( } Unfortunately that requires a virtual call to lst.size(). I don?t know if we could improve matters using a private interface contract to initialise the state e.g.: if ((hi = fence) < 0) { hi = initializer.init(this); // only two possible impls of initializer } However, since ArrayList is quite a fundamental class my inclination is to provide a separate implementation, as SubList already does for ListIterator. Then we don?t perturb the optimal case, extra fields with branching or otherwise. Paul. > On 4 Feb 2016, at 16:55, Tagir F. Valeev wrote: > > Hello! > > Thank you for clarification. As far as I know currently JIT-compiled > null checks are almost always implicit (via trapped page-fault). Well, > ok, I'm not a Hotspot expert (probably for C1 this still matters). I > will keep this code as is. > > Webrev was not changed: > http://cr.openjdk.java.net/~tvaleev/webrev/8148748/r1/ > Any more reviews, please? > > With best regards, > Tagir Valeev. > > MB> On Wed, Feb 3, 2016 at 4:20 AM, Tagir F. Valeev wrote: > >>> Some more thoughts about forEachRemaining: >>> >>> To me it seems that >>> if ((a = lst.elementData) != null) >>> is a redundant check as well as in current ArrayList implementation >>> elementData is never null. So it can be replaced with simple >>> assignment. > > MB> The null check for something that is provably non-null is a sign of > MB> the hand of Doug Lea. > MB> I'm often tempted to try to remove them from j.u.concurrent as well, > MB> but I resist. > MB> They may help hotspot generate better code. > From paul.sandoz at oracle.com Mon Feb 8 14:05:57 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 8 Feb 2016 15:05:57 +0100 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <701936850.20160208195323@gmail.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> <1489632171.20160206192945@gmail.com> <6A23376A-20E8-4717-B979-26D698E8DB29@oracle.com> <701936850.20160208195323@gmail.com> Message-ID: <7C1931B7-8641-4DC3-82F9-F9F36287F339@oracle.com> > On 8 Feb 2016, at 14:53, Tagir F. Valeev wrote: > > Hello! > > PS> 188 public Spliterator trySplit() { > PS> 189 if (isParallel && !finished) { > PS> 190 init(); > PS> 191 > PS> 192 if (buffer != null && buffer.count() > 0) // partial traversal started > PS> 193 return null; > > PS> Why don?t you check if "buffer == null? at #189? i.e. similar to forEachRemaining: > > That would make minimal behavioral change to fix this issue (fix > flatMap keys only, but not affect other intermediate ops which were > working correctly). Well, if buffer == null check is enough, here's > update: > Many thanks, yes it?s sufficient (finished == true when buffer != null && buffer.count() == 0). Would you mind updating the date in the license headers and the test with the bug id. Then i will push. I forgot to tell you about those little things, see the following for one of your fixes i pushed with such updates (to avoid any email/review latency): http://hg.openjdk.java.net/jdk9/dev/jdk/rev/4a497e746019 Paul. > http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r4/ > (test unchanged) > > With best regards, > Tagir Valeev. > From kubota.yuji at gmail.com Mon Feb 8 14:10:34 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Mon, 8 Feb 2016 23:10:34 +0900 Subject: [DONG] Re: [DING] Re: [PING] Potential infinite waiting at JMXConnection#createConnection Message-ID: Hi all, Could someone review this fix? Thanks, Yuji 2016-02-04 2:27 GMT+09:00 KUBOTA Yuji : > Hi all, > > Could someone please review and sponsor this fix ? > I write the details of this issue again. Please review it. > > =Problem= > Potential infinite waiting at TCPChannel#createConnection. > > This method flushes the DataOutputStream without the socket > timeout settings when choose stream protocol [1]. If connection lost > or the destination server do not return response during the flush, > this method wait forever because the timeout settings is set the > default value of SO_TIMEOUT, i.e., infinite. > > [1]: http://hg.openjdk.java.net/jdk9/dev/jdk/file/7adef1c3afd5/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java#l227 > > I think this issue is rarely, however serious. > > =Reproduce= > I write a test program to reproduce. You can reproduce by the below. > > * hg clone http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/ > * cd fixLoopAtJMXConnectorFactory; mvn package > * setting "stop_time" at debugcontrol.properties if you need. > * java -cp .:target/debugcontrol-1.0-SNAPSHOT.jar debugcontrol.DebugController > > This program keep to wait at TCPChannel#createConnection due to > this issue. After "debugcontroltest.stop_time" ms, this program release > the waiting by sending quit to jdb which is stopping the destination > server. Finally, return 2. > > =Solution= > Set timeout by using property-configured value: > sun.rmi.transport.tcp.responseTimeout. > > My patch is below. > http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch > > If you run the test program with modified JDK9 by my patch, the test > program will get java.net.SocketTimeoutException after the connection > timeout happen, then return 0. > > Thanks, > Yuji. > > > 2016-01-13 23:31 GMT+09:00 KUBOTA Yuji : >> Hi all, >> >> Can somebody please review and sponsor this fix ? >> >> Thanks, >> Yuji >> >> 2016-01-05 17:56 GMT+09:00 KUBOTA Yuji : >>> Hi Jaroslav and core-libs-dev, >>> >>> Thank Jaroslav for your kindness! >>> >>> For core-libs-dev members, links the information about this issue. >>> >>> * details of problem >>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002152.html >>> >>> * patch >>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch >>> >>> * testcase for reproduce >>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/testProgram >>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-December/018415.html >>> >>> Could you please review these reports? >>> Hope this patch helps to community. >>> >>> Thanks, >>> Yuji >>> >>> 2016-01-04 23:51 GMT+09:00 Jaroslav Bachorik : >>>> Hi Yuji, >>>> >>>> On 4.1.2016 15:14, KUBOTA Yuji wrote: >>>>> >>>>> Hi all, >>>>> >>>>> Could you please review this patch? >>>> >>>> >>>> Sorry for the long delay. Shanliang has not been present for some time and >>>> probably this slipped the attention of the others. >>>> >>>> However, core-libs mailing list might be more appropriate place to review >>>> this change since you are dealing with s.r.t.t.TCPChannel >>>> (http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch) >>>> >>>> Regards, >>>> >>>> -JB- From amaembo at gmail.com Mon Feb 8 14:23:37 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Mon, 8 Feb 2016 20:23:37 +0600 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <7C1931B7-8641-4DC3-82F9-F9F36287F339@oracle.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> <1489632171.20160206192945@gmail.com> <6A23376A-20E8-4717-B979-26D698E8DB29@oracle.com> <701936850.20160208195323@gmail.com> <7C1931B7-8641-4DC3-82F9-F9F36287F339@oracle.com> Message-ID: <910515365.20160208202337@gmail.com> Hello! PS> Would you mind updating the date in the license headers and the PS> test with the bug id. Then i will push. I forgot to tell you about PS> those little things, see the following for one of your fixes i PS> pushed with such updates (to avoid any email/review latency): Ah, sorry. I saw this, but always forget to add to my patch. Will take care about these things in future. Updated: http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r5/ Also updated 8147505 webrev correspondingly (seems it's also not pushed yet): http://cr.openjdk.java.net/~tvaleev/webrev/8147505/r3/ With best regards, Tagir Valeev. From amaembo at gmail.com Mon Feb 8 14:53:52 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Mon, 8 Feb 2016 20:53:52 +0600 Subject: RFR-8148748: ArrayList.subList().spliterator() is not late-binding In-Reply-To: References: <27476765.20160129103224@gmail.com> <564410418.20160202102813@gmail.com> <1084445298.20160203182024@gmail.com> <534936148.20160204215527@gmail.com> Message-ID: <1877923692.20160208205352@gmail.com> Hello! Sorry, I don't understand how passing AbstractList would help. We still need direct access to the elementData array of the original ArrayList for efficiency, and this field is also late-binding, so we need to have a reference to the ArrayList as well even in SubList case. SubList reference is also necessary, because we need to get size and modCount from it. So we have need both ArrayList and SubList and cannot replace them with single AbstractList. Note that my current implementation does not introduce new virtual calls at all. Surely it's possible to duplicate the Spliterator implementation. I have some doubts that it's reasonable, because this would increase the maintenance cost. Current Spliterator does the job pretty well for both SubList and original ArrayList. Nevertheless, if you think it's better I can create separate class (ArrayListSubListSpliterator? ArrayListSubSpliterator? SubListSpliterator?) leaving the original ArrayListSpliterator as is. I'm just worrying a little that my changes might conflict with Ivan Gerasimov's pending 8079136 issue, so probably it would be better to wait till it's reviewed and pushed... With best regards, Tagir Valeev. PS> Hi Tagir, PS> It?s certainly easier to review if one avoids the temptation to PS> change other things at the same time :-) PS> There is a possible simplified approach to sharing one PS> Spliterator impl between ArrayList and ArrayList.SubList. The PS> initial (origin, fence) for ArrayList can be (0, -1) and PS> ArrayList.SubList it can be (offset, -1), as before, then one can PS> make it static passing in an instance of AbstractList: PS> if ((hi = fence) < 0) { PS> expectedModCount = lst.modCount; // shared field on AbstractList PS> hi = fence = index + lst.size(); // virtual call :-( PS> } PS> Unfortunately that requires a virtual call to lst.size(). I don?t PS> know if we could improve matters using a private interface PS> contract to initialise the state e.g.: PS> if ((hi = fence) < 0) { PS> hi = initializer.init(this); // only two possible impls of initializer PS> } PS> However, since ArrayList is quite a fundamental class my PS> inclination is to provide a separate implementation, as SubList PS> already does for ListIterator. Then we don?t perturb the optimal PS> case, extra fields with branching or otherwise. PS> Paul. >> On 4 Feb 2016, at 16:55, Tagir F. Valeev wrote: >> >> Hello! >> >> Thank you for clarification. As far as I know currently JIT-compiled >> null checks are almost always implicit (via trapped page-fault). Well, >> ok, I'm not a Hotspot expert (probably for C1 this still matters). I >> will keep this code as is. >> >> Webrev was not changed: >> http://cr.openjdk.java.net/~tvaleev/webrev/8148748/r1/ >> Any more reviews, please? >> >> With best regards, >> Tagir Valeev. >> >> MB> On Wed, Feb 3, 2016 at 4:20 AM, Tagir F. Valeev wrote: >> >>>> Some more thoughts about forEachRemaining: >>>> >>>> To me it seems that >>>> if ((a = lst.elementData) != null) >>>> is a redundant check as well as in current ArrayList implementation >>>> elementData is never null. So it can be replaced with simple >>>> assignment. >> >> MB> The null check for something that is provably non-null is a sign of >> MB> the hand of Doug Lea. >> MB> I'm often tempted to try to remove them from j.u.concurrent as well, >> MB> but I resist. >> MB> They may help hotspot generate better code. >> From paul.sandoz at oracle.com Mon Feb 8 14:54:01 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 8 Feb 2016 15:54:01 +0100 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <910515365.20160208202337@gmail.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> <1489632171.20160206192945@gmail.com> <6A23376A-20E8-4717-B979-26D698E8DB29@oracle.com> <701936850.20160208195323@gmail.com> <7C1931B7-8641-4DC3-82F9-F9F36287F339@oracle.com> <910515365.20160208202337@gmail.com> Message-ID: <79DEB8D9-CC6B-482F-BE0F-2A6A70C19EF4@oracle.com> > On 8 Feb 2016, at 15:23, Tagir F. Valeev wrote: > > Hello! > > PS> Would you mind updating the date in the license headers and the > PS> test with the bug id. Then i will push. I forgot to tell you about > PS> those little things, see the following for one of your fixes i > PS> pushed with such updates (to avoid any email/review latency): > > Ah, sorry. I saw this, but always forget to add to my patch. Will take > care about these things in future. Updated: > > http://cr.openjdk.java.net/~tvaleev/webrev/8148838/r5/ > Thanks. Pushed. > Also updated 8147505 webrev correspondingly (seems it's also not > pushed yet): > http://cr.openjdk.java.net/~tvaleev/webrev/8147505/r3/ > Right not pushed yet, i was pondering if i should submit an internal CCC as it is arguably a minor behavioural change. I'll sit on it for another day. BTW we usually place multiple bugs ids on the same line using under one @bug annotation. Paul. From paul.sandoz at oracle.com Mon Feb 8 15:10:28 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 8 Feb 2016 16:10:28 +0100 Subject: RFR-8148748: ArrayList.subList().spliterator() is not late-binding In-Reply-To: <1877923692.20160208205352@gmail.com> References: <27476765.20160129103224@gmail.com> <564410418.20160202102813@gmail.com> <1084445298.20160203182024@gmail.com> <534936148.20160204215527@gmail.com> <1877923692.20160208205352@gmail.com> Message-ID: <3C9BC5F4-EEE1-4F79-943D-37822E7E5512@oracle.com> > On 8 Feb 2016, at 15:53, Tagir F. Valeev wrote: > > Hello! > > Sorry, I don't understand how passing AbstractList would help. Yeah, sorry ignore what i sad about AbstractList, was not thinking properly in my attempt to unify. > We > still need direct access to the elementData array of the original > ArrayList for efficiency, and this field is also late-binding, so we > need to have a reference to the ArrayList as well even in SubList > case. SubList reference is also necessary, because we need to get size > and modCount from it. So we have need both ArrayList and SubList and > cannot replace them with single AbstractList. Note that my current > implementation does not introduce new virtual calls at all. > > Surely it's possible to duplicate the Spliterator implementation. I > have some doubts that it's reasonable, because this would increase the > maintenance cost. Current Spliterator does the job pretty well for > both SubList and original ArrayList. Nevertheless, if you think it's > better I can create separate class (ArrayListSubListSpliterator? > ArrayListSubSpliterator? SubListSpliterator?) leaving the original > ArrayListSpliterator as is. > Re: maintenance, ordinarily i would agree with you, but ArrayList is kind of special being probably the most used collection class. Using an anon-impl for SubList.spliterator seem ok in that respect. > I'm just worrying a little that my changes might conflict with Ivan > Gerasimov's pending 8079136 issue, so probably it would be better to > wait till it's reviewed and pushed? > Yes, i was wondering about that too. Paul. From chris.hegarty at oracle.com Mon Feb 8 15:34:34 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 15:34:34 +0000 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> Message-ID: <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> It was suggested to me off-list that the implementation should choose a reasonable initial capacity value ,to size the StringBuilder, rather than the value read from the stream ( in case of bad or corrupt data ). So the proposed changes are: diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java @@ -3144,7 +3144,9 @@ * utflen bytes. */ private String readUTFBody(long utflen) throws IOException { - StringBuilder sbuf = new StringBuilder(); + // a reasonably initial capacity based on the UTF length + int initialCapacity = Math.max((int)utflen, 10_000); + StringBuilder sbuf = new StringBuilder(initialCapacity); if (!blkmode) { end = pos = 0; } -Chris. On 8 Feb 2016, at 11:15, Chris Hegarty wrote: > Low hanging fruit to avoid unnecessary allocations when deserializing. > readUTF knows the string size ahead of the read and can avoid > expandCapacity() by constructing the StringBuilder with the expected size. > > It is an implementation detail, but if the size is larger than Integer.MAX_VALUE, > then OOM can be thrown, as is the case in the implementation today. > > diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java > --- a/src/java.base/share/classes/java/io/ObjectInputStream.java > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java > @@ -3144,7 +3144,9 @@ > * utflen bytes. > */ > private String readUTFBody(long utflen) throws IOException { > - StringBuilder sbuf = new StringBuilder(); > + if (utflen > Integer.MAX_VALUE) > + throw new OutOfMemoryError("UTF length, " + utflen + ", too big."); > + StringBuilder sbuf = new StringBuilder((int)utflen); > if (!blkmode) { > end = pos = 0; > } > > -Chris. From chris.hegarty at oracle.com Mon Feb 8 16:20:28 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 16:20:28 +0000 Subject: RFR [9] 8148861: Remove sun.misc.ManagedLocalsThread from jaxws In-Reply-To: References: Message-ID: <31B1F016-7C24-4C1E-AA0F-D06DFF95C160@oracle.com> Receiving some feedback off list, this code needs to be able to run on older releases, so the new Thread constructor, added in JDK 9, should be accessed reflectively. Webrev updated in-place: http://cr.openjdk.java.net/~chegar/8148861/ -Chris. On 8 Feb 2016, at 10:52, Chris Hegarty wrote: > Miran, > > JDK-8056152 added a new constructor to java.lang.Thread for constructing > Threads that do not inherit inheritable thread-local initial values. All usages > of sun.misc.ManagedLocalsThread in the base module were also replaced > with this new constructor. Given there is now a supported API for creating such > threads, other areas of the JDK should be updated to use it, where applicable. > If all usages are replaced, then sun.misc.ManagedLocalsThread can be removed. > > > The following patch, along with 'hg rm ThreadHelper.java', should be sufficient. > It will remove the dependency on an internal JDK API, but adds a new dependency > on Java SE 9. Is this OK, or does this code need to access the new Java SE 9 > constructor using refection? > > diff --git a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java > --- a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java > +++ b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java > @@ -108,7 +108,8 @@ > } > > public Thread newThread(Runnable r) { > - Thread t = ThreadHelper.createNewThread(r); > - t.setName(namePrefix + threadNumber.getAndIncrement()); > + Thread t = new Thread(null, r, namePrefix + threadNumber.getAndIncrement(), 0, false); > t.setName(namePrefix + threadNumber.getAndIncrement()); > if (!t.isDaemon()) { > t.setDaemon(true); > > Complete webrev: > http://cr.openjdk.java.net/~chegar/8148861/ > > Is is possible, when agreed, to get this small change pushed upstream ? > > -Chris. From michael.haupt at oracle.com Mon Feb 8 16:39:13 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Mon, 8 Feb 2016 17:39:13 +0100 Subject: RFR 8138578:MethodHandles.Lookup.findSpecial() Javadoc fails to consider static methods In-Reply-To: <56B8976C.2010808@oracle.com> References: <56A21316.1070007@oracle.com> <56AB5E6C.1010201@oracle.com> <56B8976C.2010808@oracle.com> Message-ID: <1F8FF00D-CEC6-4D67-898F-898B4A7B3A32@oracle.com> Hi all, lower-case thumbs up, and I'll be happy to sponsor this push. Best, Michael > Am 08.02.2016 um 14:26 schrieb Vladimir Ivanov : > > Looks good. > > Best regards, > Vladimir Ivanov > > On 1/29/16 3:43 PM, shilpi rastogi wrote: >> Hi All, >> >> Please review the updated patch- >> >> http://cr.openjdk.java.net/~srastogi/8138578/webrev.01/ >> >> I verified Lookup.unreflectSpecial() also throws >> java.lang.IllegalAccessException so updated the JavaDoc. >> >> Thanks, >> Shilpi -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From mandy.chung at oracle.com Mon Feb 8 17:10:54 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 8 Feb 2016 09:10:54 -0800 Subject: RFR [9] 8148861: Remove sun.misc.ManagedLocalsThread from jaxws In-Reply-To: <31B1F016-7C24-4C1E-AA0F-D06DFF95C160@oracle.com> References: <31B1F016-7C24-4C1E-AA0F-D06DFF95C160@oracle.com> Message-ID: > On Feb 8, 2016, at 8:20 AM, Chris Hegarty wrote: > > Receiving some feedback off list, this code needs to be able to run on > older releases, so the new Thread constructor, added in JDK 9, should > be accessed reflectively. Webrev updated in-place: > > http://cr.openjdk.java.net/~chegar/8148861/ > The original version only uses sun.misc.ManagedLocalsThread when running the JDK?s JAXWS version; but for standalone, it will create a new Thread. This difference is only significant when this change is port to upstream that standalone JAXWS is running on JDK 8 and older release. Mandy > -Chris. > > On 8 Feb 2016, at 10:52, Chris Hegarty wrote: > >> Miran, >> >> JDK-8056152 added a new constructor to java.lang.Thread for constructing >> Threads that do not inherit inheritable thread-local initial values. All usages >> of sun.misc.ManagedLocalsThread in the base module were also replaced >> with this new constructor. Given there is now a supported API for creating such >> threads, other areas of the JDK should be updated to use it, where applicable. >> If all usages are replaced, then sun.misc.ManagedLocalsThread can be removed. >> >> >> The following patch, along with 'hg rm ThreadHelper.java', should be sufficient. >> It will remove the dependency on an internal JDK API, but adds a new dependency >> on Java SE 9. Is this OK, or does this code need to access the new Java SE 9 >> constructor using refection? >> >> diff --git a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java >> --- a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java >> +++ b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java >> @@ -108,7 +108,8 @@ >> } >> >> public Thread newThread(Runnable r) { >> - Thread t = ThreadHelper.createNewThread(r); >> - t.setName(namePrefix + threadNumber.getAndIncrement()); >> + Thread t = new Thread(null, r, namePrefix + threadNumber.getAndIncrement(), 0, false); >> t.setName(namePrefix + threadNumber.getAndIncrement()); >> if (!t.isDaemon()) { >> t.setDaemon(true); >> >> Complete webrev: >> http://cr.openjdk.java.net/~chegar/8148861/ >> >> Is is possible, when agreed, to get this small change pushed upstream ? >> >> -Chris. > From roger.riggs at oracle.com Mon Feb 8 18:48:38 2016 From: roger.riggs at oracle.com (Roger Riggs) Date: Mon, 8 Feb 2016 13:48:38 -0500 Subject: RFR 9: 8087286 Need a way to handle control-C and possibly some other signals In-Reply-To: <56B868BA.1000707@oracle.com> References: <56AF819C.5020905@Oracle.com> <56B128D2.2050301@oracle.com> <56B16E8D.6060208@oracle.com> <56B2801A.7060508@oracle.com> <56B399B4.6020900@Oracle.com> <56B3F572.3020502@oracle.com> <56B54457.1000900@oracle.com> <56B83D87.7030908@oracle.com> <56B868BA.1000707@oracle.com> Message-ID: <56B8E306.80706@oracle.com> Hi, I would not (did not) see this as significant but given the discussion I'll break it down into two independent updates: 1) for JEP 260, clone the s.m.Signal Api into a JDK internal package for use with java.lang.Terminator and jshell with s.m.Signal layered on top that will end up in the jdk.unsupported package 2) Work on more restrictive API later Roger On 2/8/16 5:06 AM, Alan Bateman wrote: > > On 08/02/2016 07:02, Stuart Marks wrote: >> >> >> On 2/5/16 4:54 PM, David Holmes wrote: >>> Regardless of whether I agree with this API or not, it does, as >>> Stuart points >>> out, require a JEP and to go through the normal rigorous process of >>> determining >>> whether an API is suitable for inclusion in the Java platform. >> >> Note, it was Thomas St?fe who suggested a JEP for this. > > It's a good question. > > It is an explicit non-goal of JEP 260 to propose replacements of > internal APIs. Providing a standard API for handling control-C or OS > signals is of course strongly connected with the efforts in JEP 260 > but isn't a goal. > > The JEP process provides guidelines as to when a JEP should be > drafted. In this case it seems like the API is significant and that a > JEP would make be very useful. If nothing else, the JEP could document > alternatives considered, like for example a specific API for handling > control-C/VM termination and other specific use-cases as opposed to > exposing OS signals to applications. > > -Alan From chris.hegarty at oracle.com Mon Feb 8 21:40:54 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 8 Feb 2016 21:40:54 +0000 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> Message-ID: <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> And of course, this should read... > On 8 Feb 2016, at 15:34, Chris Hegarty wrote: > > It was suggested to me off-list that the implementation should choose a > reasonable initial capacity value ,to size the StringBuilder, rather than > the value read from the stream ( in case of bad or corrupt data ). So the > proposed changes are: > > diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java > --- a/src/java.base/share/classes/java/io/ObjectInputStream.java > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java > @@ -3144,7 +3144,9 @@ > * utflen bytes. > */ > private String readUTFBody(long utflen) throws IOException { > - StringBuilder sbuf = new StringBuilder(); > + // a reasonably initial capacity based on the UTF length > + int initialCapacity = Math.min((int)utflen, 16384); > + StringBuilder sbuf = new StringBuilder(initialCapacity); > if (!blkmode) { > end = pos = 0; > } > > > > -Chris. > > On 8 Feb 2016, at 11:15, Chris Hegarty wrote: > >> Low hanging fruit to avoid unnecessary allocations when deserializing. >> readUTF knows the string size ahead of the read and can avoid >> expandCapacity() by constructing the StringBuilder with the expected size. >> >> It is an implementation detail, but if the size is larger than Integer.MAX_VALUE, >> then OOM can be thrown, as is the case in the implementation today. From aleksey.shipilev at oracle.com Mon Feb 8 21:54:46 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 9 Feb 2016 00:54:46 +0300 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> Message-ID: <56B90EA6.7020406@oracle.com> On 09.02.2016 00:40, Chris Hegarty wrote: > And of course, this should read... > > >> On 8 Feb 2016, at 15:34, Chris Hegarty wrote: >> >> It was suggested to me off-list that the implementation should choose a >> reasonable initial capacity value ,to size the StringBuilder, rather than >> the value read from the stream ( in case of bad or corrupt data ). So the >> proposed changes are: >> >> diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java >> --- a/src/java.base/share/classes/java/io/ObjectInputStream.java >> +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java >> @@ -3144,7 +3144,9 @@ >> * utflen bytes. >> */ >> private String readUTFBody(long utflen) throws IOException { >> - StringBuilder sbuf = new StringBuilder(); >> + // a reasonably initial capacity based on the UTF length >> + int initialCapacity = Math.min((int)utflen, 16384); >> + StringBuilder sbuf = new StringBuilder(initialCapacity); >> if (!blkmode) { >> end = pos = 0; >> } Still, the corrupted stream may call readUTFBody(1L + Integer.MAX_VALUE), that yields initial capacity of -2147483648, which in turn fails StringBuilder(...) with NegativeArraySizeException. But I wonder if that is actually a valid UTF8 input, because you cannot have that large of a String... -Aleksey From kim.barrett at oracle.com Mon Feb 8 21:58:20 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Mon, 8 Feb 2016 16:58:20 -0500 Subject: JDK-8052260 (Reference.isEnqueued()) Discussion In-Reply-To: <45540243-741D-4F5E-B818-A5B7E1D85195@oracle.com> References: <45540243-741D-4F5E-B818-A5B7E1D85195@oracle.com> Message-ID: > On Feb 4, 2016, at 7:52 PM, Mandy Chung wrote: > > >> On Feb 4, 2016, at 2:37 PM, Hans Boehm wrote: >> >> The discussion at https://bugs.openjdk.java.net/browse/JDK-8052260 >> correctly points out that the spec implies that this should be true if the >> reference has ever been enqueued, while the implementation returns false >> once it has been removed from the queue. The current proposal is to change >> the spec. > > The spec needs clarification (not a spec change and no behavioral change). > >> I'd like to make a different proposal, based on the following >> observations: >> >> 1) I don't know of any way to write correct code that is oblivious to this >> difference. >> 2) AFAIK, this has been broken in this way for a long time, possibly from >> the beginning, i.e. for more than 15 years(?). The bug is dated 2014. >> Apparently nobody noticed for a long time. > > AFAIK nothing is broken. What's broken is that the implementation for isEnqueued does not match its specification, as discussed in the original report for JDK-8052260. That is, I think the present wording is pretty clear, and is not consistent with the present implementation. So I disagree with the classification of the proposed wording change as a "clarification". Rather, I would describe it as a *change* to the specification to match the implementation. Alternatively, one could address the problem by fixing the implementation to match the specification; the bug report suggests a possible method for doing so. I think the implemented behavior is pretty nearly useless, and also confusing, which agrees with Hans Boehm's finding that none of the uses he examined were correct under that behavior. I think the specified behavior is marginally more useful than the implemented behavior. So I would prefer fixing the implementation rather than changing the specification. OTOH, I don't think this function has very much real utility, and don't have any objection to seeing it removed (after any required deprecation period). There may be value to having a new predicate function in this area, to test whether a reference has been cleared or has a null referent or some such. Now that we're clearing phantom references too (JDK-8071507), such an operation would be useful for them too, and would remove any specialness for phantom references with null queues. But that can be the subject of a separate RFE. (Note that there may be some trickiness to the implementation of such an operation, due to some GCs needing read barriers for referent accesses. For example, when using G1, the obvious implementation of comparing the referent field's value to null could unnecessarily extend the lifetime of the referent.) From mandy.chung at oracle.com Mon Feb 8 22:14:39 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 8 Feb 2016 14:14:39 -0800 Subject: [concurrency-interest] ClassLoader deadlock In-Reply-To: <56B71440.5000108@zeus.net.au> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> <56B5E446.9020205@zeus.net.au> <56B63B11.20903@gmail.com> <56B6668A.3030106@zeus.net.au> <56B7023C.5030708@gmail.com> <56B71440.5000108@zeus.net.au> Message-ID: Out of curiousity, what class loader is using to define org.apache.river.api.security.CombinerSecurityManager? Mandy > On Feb 7, 2016, at 1:54 AM, Peter wrote: > > Thanks Peter & David, > > That's good to know. > > I've altered the SecurityManager to perform a permission check prior to becoming the security manager, this ensures the cache has been created, to avoid any recursive deadlock prone calls. > > The class that both threads are attempting to load and init is called org.apache.river.concurrent.ReferenceIterator. > > The code can be found here: > > https://github.com/pfirmstone/river-internet/tree/Input-validation-for-Serialization > > Cheers, > > Peter. > > > On 7/02/2016 6:37 PM, Peter Levart wrote: >> >> >> On 02/06/2016 10:32 PM, Peter wrote: >>> The 0x040ebee8 monitor is most likely being held by native code. >>> >>> Regards, >>> >>> Peter Firmstone. >> >> Ok, but where? The deadlock report says it is held by main thread. Somewhere between the following two java frames? >> >> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor entry [0x0185e000] >> ... >> ... >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> --here-- >> at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >> >> >> As I understand this is where JVM resolves some class that for the 1st time while ReferenceCollection.iterator is being executed. So before calling-back into ClassLoader.loadClass, it acquires a monitor lock on some int[] object? >> >> >> Anyway. Looking at the following part of Thread-1 stacktrace: >> >> at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >> at java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >> at java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >> at java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >> at java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >> at java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >> at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >> at java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >> >> ...I can see you are using some early release of JDK 8 where ThreadLocalRandom initialization involved obtaining a hardware address of some interface. This in turn checks for security manager. Since JDK 8u45 this code has been removed and only current time is used to derive initial seed, so you might not see this deadlock any more with JDK 8u45 and later. >> >> Regards, Peter >> >>> >>> >>> On 7/02/2016 4:27 AM, Peter Levart wrote: >>>> >>>> >>>> On 02/06/2016 01:17 PM, Peter Firmstone wrote: >>>>> The security manager is non blocking, unfortunately java system classes aren't. >>>> >>>> It doesn't seem so. At least, I can't find where main thread locks the 0x040ebee8 monitor: >>>> >>>> "Thread-1": >>>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>>> which is held by "main" >>>> >>>> >>>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor entry [0x0185e000] >>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> at org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> at org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> at org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> at org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> at org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> at org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> at org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> at org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> at java.lang.System.checkIO(System.java:253) >>>> at java.lang.System.setErr(System.java:199) >>>> at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>> >>>> >>>> ...no mention of 0x040ebee8 in main thread...?! >>>> >>>> Peter >>>> >>>>> >>>>> The policy provider in use thread confines PermissionCollection instances, which never leave the cpu cache, they are discarded immediately after use. >>>>> >>>>> The problem here is that two different threads are attempting to load the same class at the same time. >>>>> >>>>> I'll have to make sure that all classes are loaded before the security manager becomes the system security manager, either that or eliminate the AccessControlContext check permission cache. >>>>> >>>>> While the standard java security manager and policy provider consume around 10% cpu, this consumes less than 1%. >>>>> >>>>> On 6/02/2016 9:57 PM, David Holmes wrote: >>>>>> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>>>>>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>>>>>> deadlock prone? Guess I was wrong. >>>>>> >>>>>> The deadlock is introduced by a custom security manager. SM's play a critical role in many parts of the core libraries so if they utilize synchronization then they can easily introduce deadlock. >>>>>> >>>>>>> Observation: On an unrelated occassion, I had a URLClassLoader >>>>>>> synchronization hotspot (well not standard URLClassLoader, but a high >>>>>>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>>>>>> instead of URL DNS lookup), I solved that problem by thread confining >>>>>>> class loading. I've never experienced deadlock using thread >>>>>>> confinement, initially I was worried that there would be >>>>>>> interdependencies, however I didn't experience a problem, as the single >>>>>>> thread would always load the classes it needed. >>>>>>> >>>>>>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>>>>>> strategy, I found thread confinement was very performant. >>>>>> >>>>>> Are you suggesting that a class-load request is actually transmitted to a single class-loading thread, while the initial thread blocks until the loading is complete? That would still seem prone to the same SM related deadlock scenario. And it would not generally be particularly performant. >>>>> >>>>> I thought that too, until I tried it. >>>>> >>>>> Would you like to see some performance figures? >>>>> >>>>> Cheers, >>>>> >>>>> Peter. >>>>> >>>>>> >>>>>> David >>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Peter. >>>>>>> >>>>>>> 2016-02-06 21:06:07 >>>>>>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>>>>>> >>>>>>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>>>>>> runnable [0x00000000] >>>>>>> java.lang.Thread.State: RUNNABLE >>>>>>> >>>>>>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>>>>>> nid=0x111c waiting on condition [0x00000000] >>>>>>> java.lang.Thread.State: RUNNABLE >>>>>>> >>>>>>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 >>>>>>> waiting on condition [0x00000000] >>>>>>> java.lang.Thread.State: RUNNABLE >>>>>>> >>>>>>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 nid=0x10b8 >>>>>>> runnable [0x00000000] >>>>>>> java.lang.Thread.State: RUNNABLE >>>>>>> >>>>>>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>>>>>> monitor entry [0x03ded000] >>>>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>>> >>>>>>> at >>>>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>>>> at >>>>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>>>> >>>>>>> at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>>>> - locked <0x03f624b8> (a java.lang.Object) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>>> at >>>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>>>> >>>>>>> at >>>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>>>> >>>>>>> - locked <0x03ef8b30> (a >>>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>>>> at >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>> at >>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>>>> >>>>>>> ava:180) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>>>> >>>>>>> ) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>>>> >>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>> >>>>>>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>>>>>> Object.wait() [0x03cff000] >>>>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>>>> at java.lang.Object.wait(Native Method) >>>>>>> - waiting on <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>>>>>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>>>>>> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>>>>>> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>>>>>> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>>>>>> >>>>>>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>>>>>> nid=0x1398 in Object.wait() [0x144cf000] >>>>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>>>> at java.lang.Object.wait(Native Method) >>>>>>> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >>>>>>> at java.lang.Object.wait(Object.java:502) >>>>>>> at >>>>>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>>>>>> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >>>>>>> >>>>>>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for monitor >>>>>>> entry [0x0185e000] >>>>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>>> >>>>>>> at java.lang.System.checkIO(System.java:253) >>>>>>> at java.lang.System.setErr(System.java:199) >>>>>>> at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>>>> >>>>>>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>>>>>> >>>>>>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 waiting on >>>>>>> condition >>>>>>> >>>>>>> JNI global references: 19 >>>>>>> >>>>>>> >>>>>>> Found one Java-level deadlock: >>>>>>> ============================= >>>>>>> "Thread-1": >>>>>>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>>>>>> which is held by "main" >>>>>>> "main": >>>>>>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>>>>>> java.lang.Object), >>>>>>> which is held by "Thread-1" >>>>>>> >>>>>>> Java stack information for the threads listed above: >>>>>>> =================================================== >>>>>>> "Thread-1": >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>>> >>>>>>> at >>>>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>>>> at >>>>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>>>> at >>>>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>>>> >>>>>>> at java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>>>> - locked <0x03f624b8> (a java.lang.Object) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>>> at >>>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>>>> >>>>>>> at >>>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>>>> >>>>>>> - locked <0x03ef8b30> (a >>>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>>>> at >>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>>> at >>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>>>> >>>>>>> ava:180) >>>>>>> at >>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>>>> >>>>>>> ) >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>>>> >>>>>>> at >>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>>>> >>>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>>> "main": >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>>> at >>>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>>> at >>>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>>> >>>>>>> at >>>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>>> >>>>>>> at java.lang.System.checkIO(System.java:253) >>>>>>> at java.lang.System.setErr(System.java:199) >>>>>>> at org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>>>> >>>>>>> Found 1 deadlock. >>>>> >>>> >>>> >>>> _______________________________________________ >>>> Concurrency-interest mailing list >>>> Concurrency-interest at cs.oswego.edu >>>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest >>> >> > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest From martinrb at google.com Mon Feb 8 22:29:01 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 8 Feb 2016 14:29:01 -0800 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: Message-ID: On Mon, Feb 8, 2016 at 2:09 AM, Paul Sandoz wrote: > >> On 1 Feb 2016, at 19:45, Martin Buchholz wrote: >> >> After much debate on what to do when CompleteableFuture.whenComplete >> encounters an exception in both the source and the action, we chose >> what was acceptable to the most people - add the action's exception to >> the source exception as a suppressed exception. And added usage >> guidelines. And gave handle "top billing" over whenComplete. >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ >> >> and the usual miscellaneous improvements. > > > +1 > > Looks like the diffs got swapped for EmptyNavigableSet.java in the misc patch: Thanks, fixed. This is a result of having bidirectional changes flowing between different source code control systems and no merge bridging tool. (I don't think I've regressed an upstream change yet, but I'm careful before doing any hg push. I've modified my upstreaming script to warn me about such conflicts in the future) From david.holmes at oracle.com Tue Feb 9 04:11:47 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 9 Feb 2016 14:11:47 +1000 Subject: [DONG] Re: [DING] Re: [PING] Potential infinite waiting at JMXConnection#createConnection In-Reply-To: References: Message-ID: <56B96703.6050509@oracle.com> Hi Yuji, Not sure who would look at this so cc'ing net-dev. Also note that contributions can only be accepted if presented via OpenJKDK infrastructure. Links to patches on http://icedtea.classpath.org are not acceptable. The patch needs to be included in the email (beware stripped attachments) if you can't get it hosted on cr.openjdk.java.net. Sorry. David On 9/02/2016 12:10 AM, KUBOTA Yuji wrote: > Hi all, > > Could someone review this fix? > > Thanks, > Yuji > > 2016-02-04 2:27 GMT+09:00 KUBOTA Yuji : >> Hi all, >> >> Could someone please review and sponsor this fix ? >> I write the details of this issue again. Please review it. >> >> =Problem= >> Potential infinite waiting at TCPChannel#createConnection. >> >> This method flushes the DataOutputStream without the socket >> timeout settings when choose stream protocol [1]. If connection lost >> or the destination server do not return response during the flush, >> this method wait forever because the timeout settings is set the >> default value of SO_TIMEOUT, i.e., infinite. >> >> [1]: http://hg.openjdk.java.net/jdk9/dev/jdk/file/7adef1c3afd5/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java#l227 >> >> I think this issue is rarely, however serious. >> >> =Reproduce= >> I write a test program to reproduce. You can reproduce by the below. >> >> * hg clone http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/ >> * cd fixLoopAtJMXConnectorFactory; mvn package >> * setting "stop_time" at debugcontrol.properties if you need. >> * java -cp .:target/debugcontrol-1.0-SNAPSHOT.jar debugcontrol.DebugController >> >> This program keep to wait at TCPChannel#createConnection due to >> this issue. After "debugcontroltest.stop_time" ms, this program release >> the waiting by sending quit to jdb which is stopping the destination >> server. Finally, return 2. >> >> =Solution= >> Set timeout by using property-configured value: >> sun.rmi.transport.tcp.responseTimeout. >> >> My patch is below. >> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch >> >> If you run the test program with modified JDK9 by my patch, the test >> program will get java.net.SocketTimeoutException after the connection >> timeout happen, then return 0. >> >> Thanks, >> Yuji. >> >> >> 2016-01-13 23:31 GMT+09:00 KUBOTA Yuji : >>> Hi all, >>> >>> Can somebody please review and sponsor this fix ? >>> >>> Thanks, >>> Yuji >>> >>> 2016-01-05 17:56 GMT+09:00 KUBOTA Yuji : >>>> Hi Jaroslav and core-libs-dev, >>>> >>>> Thank Jaroslav for your kindness! >>>> >>>> For core-libs-dev members, links the information about this issue. >>>> >>>> * details of problem >>>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002152.html >>>> >>>> * patch >>>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch >>>> >>>> * testcase for reproduce >>>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/testProgram >>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-December/018415.html >>>> >>>> Could you please review these reports? >>>> Hope this patch helps to community. >>>> >>>> Thanks, >>>> Yuji >>>> >>>> 2016-01-04 23:51 GMT+09:00 Jaroslav Bachorik : >>>>> Hi Yuji, >>>>> >>>>> On 4.1.2016 15:14, KUBOTA Yuji wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> Could you please review this patch? >>>>> >>>>> >>>>> Sorry for the long delay. Shanliang has not been present for some time and >>>>> probably this slipped the attention of the others. >>>>> >>>>> However, core-libs mailing list might be more appropriate place to review >>>>> this change since you are dealing with s.r.t.t.TCPChannel >>>>> (http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch) >>>>> >>>>> Regards, >>>>> >>>>> -JB- From peter.firmstone at zeus.net.au Tue Feb 9 04:19:54 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Tue, 9 Feb 2016 14:19:54 +1000 (AEST) Subject: DeserializationPermission Proposal Message-ID: <1b9fb4632fa0b17ad2b3ca7c824f9188@org.tizen.email> Why not, just prior to instantiating an object just prior to deserializing, add each class' ProtectionDomain in the objects hierarchy to an AccessControlContext and pass this to the SecurityManager's two argument checkPermission call? This permission could never be granted to a principal, it is only ever a code trust concern. ?This would allow an administrator to minimise the attack surface of Serializable classes. Just a thought, Peter. Sent from my Samsung device. ? From kubota.yuji at gmail.com Tue Feb 9 06:50:15 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Tue, 9 Feb 2016 15:50:15 +0900 Subject: [DONG] Re: [DING] Re: [PING] Potential infinite waiting at JMXConnection#createConnection In-Reply-To: <56B96703.6050509@oracle.com> References: <56B96703.6050509@oracle.com> Message-ID: Hi David, Thank you for your advice and cc-ing! I do not have any role yet, so I paste my patches as below. diff --git a/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java b/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java --- a/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java +++ b/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java @@ -222,20 +222,34 @@ // choose protocol (single op if not reusable socket) if (!conn.isReusable()) { out.writeByte(TransportConstants.SingleOpProtocol); } else { out.writeByte(TransportConstants.StreamProtocol); + + int usableSoTimeout = 0; + try { + /* + * If socket factory had set a non-zero timeout on its + * own, then restore it instead of using the property- + * configured value. + */ + usableSoTimeout = sock.getSoTimeout(); + if (usableSoTimeout == 0) { + usableSoTimeout = responseTimeout; + } + sock.setSoTimeout(usableSoTimeout); + } catch (Exception e) { + // if we fail to set this, ignore and proceed anyway + } out.flush(); /* * Set socket read timeout to configured value for JRMP * connection handshake; this also serves to guard against * non-JRMP servers that do not respond (see 4322806). */ - int originalSoTimeout = 0; try { - originalSoTimeout = sock.getSoTimeout(); sock.setSoTimeout(handshakeTimeout); } catch (Exception e) { // if we fail to set this, ignore and proceed anyway } @@ -279,18 +293,11 @@ * connection. NOTE: this timeout, if configured to a * finite duration, places an upper bound on the time * that a remote method call is permitted to execute. */ try { - /* - * If socket factory had set a non-zero timeout on its - * own, then restore it instead of using the property- - * configured value. - */ - sock.setSoTimeout((originalSoTimeout != 0 ? - originalSoTimeout : - responseTimeout)); + sock.setSoTimeout(usableSoTimeout); } catch (Exception e) { // if we fail to set this, ignore and proceed anyway } out.flush(); Thanks, Yuji 2016-02-09 13:11 GMT+09:00 David Holmes : > Hi Yuji, > > Not sure who would look at this so cc'ing net-dev. > > Also note that contributions can only be accepted if presented via OpenJKDK > infrastructure. Links to patches on http://icedtea.classpath.org are not > acceptable. The patch needs to be included in the email (beware stripped > attachments) if you can't get it hosted on cr.openjdk.java.net. Sorry. > > David > > > On 9/02/2016 12:10 AM, KUBOTA Yuji wrote: >> >> Hi all, >> >> Could someone review this fix? >> >> Thanks, >> Yuji >> >> 2016-02-04 2:27 GMT+09:00 KUBOTA Yuji : >>> >>> Hi all, >>> >>> Could someone please review and sponsor this fix ? >>> I write the details of this issue again. Please review it. >>> >>> =Problem= >>> Potential infinite waiting at TCPChannel#createConnection. >>> >>> This method flushes the DataOutputStream without the socket >>> timeout settings when choose stream protocol [1]. If connection lost >>> or the destination server do not return response during the flush, >>> this method wait forever because the timeout settings is set the >>> default value of SO_TIMEOUT, i.e., infinite. >>> >>> [1]: >>> http://hg.openjdk.java.net/jdk9/dev/jdk/file/7adef1c3afd5/src/java.rmi/share/classes/sun/rmi/transport/tcp/TCPChannel.java#l227 >>> >>> I think this issue is rarely, however serious. >>> >>> =Reproduce= >>> I write a test program to reproduce. You can reproduce by the below. >>> >>> * hg clone >>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/ >>> * cd fixLoopAtJMXConnectorFactory; mvn package >>> * setting "stop_time" at debugcontrol.properties if you need. >>> * java -cp .:target/debugcontrol-1.0-SNAPSHOT.jar >>> debugcontrol.DebugController >>> >>> This program keep to wait at TCPChannel#createConnection due to >>> this issue. After "debugcontroltest.stop_time" ms, this program release >>> the waiting by sending quit to jdb which is stopping the destination >>> server. Finally, return 2. >>> >>> =Solution= >>> Set timeout by using property-configured value: >>> sun.rmi.transport.tcp.responseTimeout. >>> >>> My patch is below. >>> >>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch >>> >>> If you run the test program with modified JDK9 by my patch, the test >>> program will get java.net.SocketTimeoutException after the connection >>> timeout happen, then return 0. >>> >>> Thanks, >>> Yuji. >>> >>> >>> 2016-01-13 23:31 GMT+09:00 KUBOTA Yuji : >>>> >>>> Hi all, >>>> >>>> Can somebody please review and sponsor this fix ? >>>> >>>> Thanks, >>>> Yuji >>>> >>>> 2016-01-05 17:56 GMT+09:00 KUBOTA Yuji : >>>>> >>>>> Hi Jaroslav and core-libs-dev, >>>>> >>>>> Thank Jaroslav for your kindness! >>>>> >>>>> For core-libs-dev members, links the information about this issue. >>>>> >>>>> * details of problem >>>>> >>>>> http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-April/002152.html >>>>> >>>>> * patch >>>>> >>>>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch >>>>> >>>>> * testcase for reproduce >>>>> >>>>> http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/testProgram >>>>> >>>>> http://mail.openjdk.java.net/pipermail/serviceability-dev/2015-December/018415.html >>>>> >>>>> Could you please review these reports? >>>>> Hope this patch helps to community. >>>>> >>>>> Thanks, >>>>> Yuji >>>>> >>>>> 2016-01-04 23:51 GMT+09:00 Jaroslav Bachorik >>>>> : >>>>>> >>>>>> Hi Yuji, >>>>>> >>>>>> On 4.1.2016 15:14, KUBOTA Yuji wrote: >>>>>>> >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> Could you please review this patch? >>>>>> >>>>>> >>>>>> >>>>>> Sorry for the long delay. Shanliang has not been present for some time >>>>>> and >>>>>> probably this slipped the attention of the others. >>>>>> >>>>>> However, core-libs mailing list might be more appropriate place to >>>>>> review >>>>>> this change since you are dealing with s.r.t.t.TCPChannel >>>>>> >>>>>> (http://icedtea.classpath.org/people/ykubota/fixLoopAtJMXConnectorFactory/file/e31044f0804f/jdk9.patch) >>>>>> >>>>>> Regards, >>>>>> >>>>>> -JB- From ivan.gerasimov at oracle.com Tue Feb 9 07:17:21 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Tue, 9 Feb 2016 10:17:21 +0300 Subject: RFR 8046339: sun.rmi.transport.DGCAckHandler leaks memory Message-ID: <56B99281.605@oracle.com> Hello! This is a request to help review a fix for the problem which can lead to memory leakage under some rare circumstances. The fix itself is quite simple - it just makes sure the handler being released is also removed from the set of tracked handlers. I've also augmented an existing regression test to make sure we're not observing the memory leak anymore. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8046339 WEBREV: http://cr.openjdk.java.net/~igerasim/8046339/00/webrev/ Would you please help review it? Sincerely yours, Ivan From paul.sandoz at oracle.com Tue Feb 9 11:12:26 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 9 Feb 2016 12:12:26 +0100 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> Message-ID: <4EC42B69-2603-43BC-AB29-BAD8EF53F90E@oracle.com> Hi Jeremy, Sorry for the late reply. Catching after some distractions and being away. > On 22 Jan 2016, at 19:03, Jeremy Manson wrote: > > Couple of thoughts: > The Java Language Specification permits operations to be executed in orders different than are apparent in program source code, subject to constraints arising, for example, from the use of locks, volatile fields or VarHandles. The static methods, fullFence, acquireFence, releaseFence, loadLoadFence andstoreStoreFence, can also be used to impose constraints. Their specifications are phrased in terms of the lack of "reorderings" -- observable ordering effects that might otherwise occur if the fence were not present. > > > There is no notion of reordering (per se) in the JLS; the fact that reorderings can occur is just implied by the way programs can be observed to behave. So, these fence operations don't map to anything non-implementation dependent. I don't think it would be impossible to fix the spec to define something that works the way you want (similar to the final field semantics), but it isn't in there already. You might want to call that out (either by saying this behavior is best effort implementation-dependent or by fixing the spec). > Until we update the JMM spec, pushed out beyond 9, we are in a bit of a bind and have to hand wave a bit for both access mode methods and fences, both of which talk about reorderings (see getAcquire/setRelease for example). There is the following at the end, but it?s easy to loose sight of with all the other sections: * @apiNote * More precise phrasing of the specification of access mode methods and memory * fence methods may accompany future updates of the Java Language * Specification. I moved that up into the fences paragraph. *

In addition to supporting access to variables under various access modes, * a set of static methods, referred to as memory fence methods, is also * provided for fine-grained control of memory ordering. * * The Java Language Specification permits operations to be executed in * orders different than are apparent in program source code, subject to * constraints arising, for example, from the use of locks, {@code volatile} * fields or VarHandles. The static methods, {@link #fullFence fullFence}, * {@link #acquireFence acquireFence}, {@link #releaseFence releaseFence}, * {@link #loadLoadFence loadLoadFence} and {@link #storeStoreFence * storeStoreFence}, can also be used to impose constraints. Their * specifications, as is the case for certain access modes, are phrased in terms * of the lack of "reorderings" -- observable ordering effects that might * otherwise occur if the fence was not present. More precise phrasing of the * specification of access mode methods and memory fence methods may accompany * future updates of the Java Language Specification. I could append another sentence: Until then, such reordering behaviour is considered implementation-dependent on a best-effort basis. But perhaps the less we say the better? > If a VarHandle references a read-only variable (for example a final field) then write, atomic update and numeric atomic update access modes are not supported and corresponding methods throw UnsupportedOperationException. > > Are you sure you want to limit it in this way? There are an awful lot of calls to setAccessible in the world of reflection. And writing to final fields *does* map to something sensible in the spec. In fact, it would be great to have something that wasn't quite such a blunt instrument as setAccessible. Indeed, very much a blunt instrument. Note that setAccessible would otherwise be required to be called on the Field before being unreflected (either for unreflectVarHandle or, as is the case today, unreflectSetter/Getter). I don?t wanna widen the accessibility hole of this blunt instrument right now; it?s easy to widen rather than shrink later on if need be. The motivation behind this is we want to move towards a world "final really means final" and thus lots of nice optimisations may ensue. It may be a long and windy road to get there but we have some plans and are starting to work some routes towards that goal. > > getVolatile > public final Object getVolatile(Object... args) > Returns the value of a variable, with memory semantics of reading a volatile variable. > > Reading *which* volatile variable? You probably mean that all getVolatiles and setVolatiles provide semantics that behave as if the variable being written / read was declared volatile in the first place, but it is worth calling out. > /** * Returns the value of a variable, with memory semantics of reading as if * the variable was declared {@code volatile}. ... Object getVolatile(Object... args); /** * Sets the value of a variable to the {@code newValue}, with memory * semantics of setting as if the variable was declared {@code volatile}. ? void setVolatile(Object... args); I also updated the plain accessors using similar ?as if? language. > Nits: > The following nits are copied and translated from documentation on MethodHandle. > As is usual with virtual methods, source-level calls to access mode methods compile to an invokevirtual instruction. More unusually, the compiler must record the actual argument types, and may not perform method invocation conversions on the arguments. Instead, it must push them on the stack according to their own unconverted types. The VarHandle object itself is pushed on the stack before the arguments. The compiler then calls the VarHandle with a symbolic type descriptor which describes the argument and return types. > > This is somewhat oddly worded. The compiler doesn't push arguments on a stack or call anything - it generates instructions that do that. > I updated to: * As is usual with virtual methods, source-level calls to access mode methods * compile to an {@code invokevirtual} instruction. More unusually, the * compiler must record the actual argument types, and may not perform method * invocation conversions on the arguments. Instead, it must generate * instructions to push them on the stack according to their own unconverted * types. The VarHandle object itself will be pushed on the stack before the * arguments. The compiler then generates an {@code invokevirtual} instruction * that invokes the access mode method with a symbolic type descriptor which * describes the argument and return types. > The first time a invokevirtual instruction is executed it is linked, by symbolically resolving the names in the instruction and verifying that the method call is statically legal > > I keep trying not to call this out as a nit, but there should be no comma between "linked" and "by?. Updated. In both cases I also updated the relevant areas in MethodHandle. Thanks, Paul. From chris.hegarty at oracle.com Tue Feb 9 12:20:38 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 9 Feb 2016 12:20:38 +0000 Subject: ObjectInputStream SPI In-Reply-To: <2a7859285494d908effbc200502cbd52@org.tizen.email> References: <2a7859285494d908effbc200502cbd52@org.tizen.email> Message-ID: <56B9D996.204@oracle.com> Peter, I, along with others within Oracle, are interested in this general area. We are tied up with other issues at the moment, but I hope to get this within the next couple of weeks. -Chris. On 04/02/16 00:40, Peter Firmstone wrote: > In light of recent examples of gadget deserialization attacks, I believe we need an OIS SPI. > > While OIS functionality can be overridden, there's no way to ensure this can be done for all uses of OIS. > > I believe this is necessary for security reasons, to allow Serialization to be completely disabled or restricted to only those classes in use by an application or reimplemented to allow input validation. > > An OIS SPI would be a very simple straightforward solution. > > Regards, > > Peter Firmstone. > > Sent from my Samsung device. > > From miroslav.kos at oracle.com Tue Feb 9 13:09:59 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Tue, 9 Feb 2016 14:09:59 +0100 Subject: RFR [9] 8148861: Remove sun.misc.ManagedLocalsThread from jaxws In-Reply-To: <31B1F016-7C24-4C1E-AA0F-D06DFF95C160@oracle.com> References: <31B1F016-7C24-4C1E-AA0F-D06DFF95C160@oracle.com> Message-ID: <56B9E527.8070800@oracle.com> Looks ok to me, will take care of it in standalone project. Thanks Miran On 08/02/16 17:20, Chris Hegarty wrote: > Receiving some feedback off list, this code needs to be able to run on > older releases, so the new Thread constructor, added in JDK 9, should > be accessed reflectively. Webrev updated in-place: > > http://cr.openjdk.java.net/~chegar/8148861/ > > -Chris. > > On 8 Feb 2016, at 10:52, Chris Hegarty wrote: > >> Miran, >> >> JDK-8056152 added a new constructor to java.lang.Thread for constructing >> Threads that do not inherit inheritable thread-local initial values. All usages >> of sun.misc.ManagedLocalsThread in the base module were also replaced >> with this new constructor. Given there is now a supported API for creating such >> threads, other areas of the JDK should be updated to use it, where applicable. >> If all usages are replaced, then sun.misc.ManagedLocalsThread can be removed. >> >> >> The following patch, along with 'hg rm ThreadHelper.java', should be sufficient. >> It will remove the dependency on an internal JDK API, but adds a new dependency >> on Java SE 9. Is this OK, or does this code need to access the new Java SE 9 >> constructor using refection? >> >> diff --git a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java >> --- a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java >> +++ b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/api/pipe/Engine.java >> @@ -108,7 +108,8 @@ >> } >> >> public Thread newThread(Runnable r) { >> - Thread t = ThreadHelper.createNewThread(r); >> - t.setName(namePrefix + threadNumber.getAndIncrement()); >> + Thread t = new Thread(null, r, namePrefix + threadNumber.getAndIncrement(), 0, false); >> t.setName(namePrefix + threadNumber.getAndIncrement()); >> if (!t.isDaemon()) { >> t.setDaemon(true); >> >> Complete webrev: >> http://cr.openjdk.java.net/~chegar/8148861/ >> >> Is is possible, when agreed, to get this small change pushed upstream ? >> >> -Chris. From chris.hegarty at oracle.com Tue Feb 9 13:35:19 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 9 Feb 2016 13:35:19 +0000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B874DA.80001@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> Message-ID: <56B9EB17.7020303@oracle.com> Peter, On 08/02/16 10:58, Peter Levart wrote: > On 02/08/2016 10:33 AM, Chris Hegarty wrote: >> On 8 Feb 2016, at 06:27, Alan Bateman wrote: >> >>> On 07/02/2016 22:20, Peter Levart wrote: >>>> : >>>> >>>> If the decision to remove sun.misc.Cleaner was partly influenced by the desire to maintain just 2 instead of 3 Cleaner(s), then my proposal to migrate JDK code to the public API might enable Oracle to reconsider keeping sun.misc.Cleaner. >>> The main issue driving this is the design principles that we have listed in JEP 200. We don't want a standard module (java.base in this case) exporting a non-standard API. This means surgery to jettison the sun.misc package from java.base and move it to its own module (jdk.unsupported is the proposal in JEP 260). This is painful of course but we are at least in good shape for the most critical internal API, sun.misc.Unsafe. >>> >>> For sun.misc.Cleaner then the original proposal was for it to be a critical internal API too but it become clear that changing the type of internal/private fields in the NIO buffer and channel classes would break libraries that have been hacking into those fields. If they are changing away then there seems little motive to keep sun.misc.Cleaner so Chris moved into into jdk.internal.ref for now. As to whether we even need to keep jdk.internal.ref.Cleaner then I think the only remaining question was whether the special handling of Cleaner in the Reference implementation was worth it or not. It may not be, in which case your current proposal to just remove seems right. >> Alan?s summary of the situation is spot on. >> >> Before moving sun.misc.Cleaner to jdk.internal.ref, I did enquire if it would be >> possible to just remove it and use the new public Cleaner, but I got feedback >> that there were some issues with failing NIO tests ( it appeared that Cleaners >> were not running as quickly as possible ). I didn?t look further into that at the >> time, since the VM still had special knowledge of these cleaners, but as you >> say this is now removed. It is probably a good time to revisit this. >> >> -Chris. > > Hi Chris, > > Are you referring to the following test: > > test/java/nio/Buffer/DirectBufferAllocTest.java ? > > This test was written specifically for the following issue: > > https://bugs.openjdk.java.net/browse/JDK-6857566 > > ...which exercises multi-threaded allocation of direct buffers. Even > sun.misc.Cleaner was not quick enough to promptly deallocate them, so > allocation path was patched to help ReferenceHandler thread while > re-attempting to allocate new direct buffers. Transitioning to > java.lang.ref.Cleaner which uses additional thread to process > Cleanable(s), direct-buffer allocation path must help the Cleaner thread > too. See: Ah, I was not aware of this. So additional threads attempting to allocate already help out with cleaning. So moving away from the ReferenceHandler Thread is not such a big deal as I was thinking. > http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/src/java.base/share/classes/java/nio/Bits.java.sdiff.html > > ...for the place where this is performed... I did look through your changes and I think they are good. I'd like to spend a little more time on the details. -Chris. > Regards, Peter > > > From david.lloyd at redhat.com Tue Feb 9 15:04:07 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Tue, 9 Feb 2016 09:04:07 -0600 Subject: DeserializationPermission Proposal In-Reply-To: <1b9fb4632fa0b17ad2b3ca7c824f9188@org.tizen.email> References: <1b9fb4632fa0b17ad2b3ca7c824f9188@org.tizen.email> Message-ID: <56B9FFE7.8070703@redhat.com> On 02/08/2016 10:19 PM, Peter Firmstone wrote: > Why not, just prior to instantiating an object just prior to deserializing, add each class' ProtectionDomain in the objects hierarchy to an AccessControlContext and pass this to the SecurityManager's two argument checkPermission call? > > This permission could never be granted to a principal, it is only ever a code trust concern. This would allow an administrator to minimise the attack surface of Serializable classes. I think rather than using the ProtectionDomain of the objects in the serialization stream, would it not make more sense to capture and use (exclusively) the access control context of the entity which is constructing the stream? The reason I say this is that it's very possible for a less-privileged object to have references to more-privileged objects and vice-versa; also, in some cases you're predicating the success of deserialization upon the order in which objects are deserialized. For example, if I have a four-object graph of A, B, C, and D, in a diamond like this: A->B A->C B->D C->D If B's class does not have privileges to construct D, deserialization will fail, even if C does. On the other hand, if B has permission to construct D, but C doesn't, C can escape its restriction by relying on B to have already deserialized the object. But by using one permission set - the captured context of the creator of the stream, or perhaps the captured context of the "root" readObject() call, you cannot change the authorization outcome of the deserialization just by fiddling with the bits. A graph in this case is either valid or not. -- - DML From mr.chrisvest at gmail.com Tue Feb 9 15:20:27 2016 From: mr.chrisvest at gmail.com (Chris Vest) Date: Tue, 9 Feb 2016 16:20:27 +0100 Subject: Compact Strings and APIs for fast decoding of string data Message-ID: Hi, Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string data in Latin1, which is one of the two special encodings for compact strings, we?d ideally like to produce the String object with just the two necessary object allocations and a single, vectorised memory copy. Our use case is that we are a database and we do our own file paging, effectively having file data in a large set of DirectByteBuffers. We have string data in our files in a number of different encodings, a popular one being Latin1. Occasionally these String values span multiple buffers. We often need to expose this data as String objects, in which case decoding the bytes and turning them into a String is often very performance sensitive - to the point of being one of our top bottlenecks for the given queries. Part of the story is that in the case of Latin1, I?ll know up front exactly how many bytes my string data takes up, though I might not know how many buffers are going to be involved. As far as I can tell, this is currently not possible using public APIs. Using private APIs it may be possible, but will be relying on the JIT for vectorising the memory copying. From an API standpoint, CharsetDecoder is close to home, but is not quite there. It?s stateful and not thread-safe, so I either have to allocate new ones every time or cache them in thread-locals. I?m also required to allocate the receiving CharBuffer. Since I may need to decode from multiple buffers, I realise that I might not be able to get away from allocating at least one extra object to keep track of intermediate decoding state. The CharsetDecoder does not have a method where I can specify the offset and length for the desired part of the ByteBuffer I want to decode, which forces be to allocate views instead. The CharBuffers are allocated with a length up front, which is nice, but I can?t restrict its encoding so it has to allocate a char array instead of the byte array that I really want. Even if it did allocate a byte array, the CharBuffer is mutable, which would force String do a defensive copy anyway. One way I imagine this could be solved would be with a less dynamic kind of decoder, where the target length is given upfront to the decoder. Buffers are then consumed one by one, and a terminal method performs finishing sanity checks (did we get all the bytes we were promised?) and returns the result. StringDecoder decoder = Charset.forName(?latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, len2).done(); This will in principle allow the string decoding to be 2 small allocations, an array allocation without zeroing, and a sequence of potentially vectorised memcpys. I don?t see any potentially troubling interactions with fused Strings either, since all the knowledge (except for the string data itself) needed to allocate the String objects are available from the get-go. What do you guys think? Btw, Richard Warburton has already done some work in this area, and made a patch that adds a constructor to String that takes a buffer, offset, length, and charset. This work now at least needs rebasing: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ It doesn?t solve the case where multiple buffers are used to build the string, but does remove the need for a separate intermediate state-holding object when a single buffer is enough. It?d be a nice addition if possible, but I (for one) can tolerate a small object allocation otherwise. Cheers, Chris From jini at zeus.net.au Sat Feb 6 21:32:58 2016 From: jini at zeus.net.au (Peter) Date: Sun, 07 Feb 2016 07:32:58 +1000 Subject: [concurrency-interest] ClassLoader deadlock In-Reply-To: <56B63B11.20903@gmail.com> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> <56B5E446.9020205@zeus.net.au> <56B63B11.20903@gmail.com> Message-ID: <56B6668A.3030106@zeus.net.au> The 0x040ebee8 monitor is most likely being held by native code. Regards, Peter Firmstone. On 7/02/2016 4:27 AM, Peter Levart wrote: > > > On 02/06/2016 01:17 PM, Peter Firmstone wrote: >> The security manager is non blocking, unfortunately java system >> classes aren't. > > It doesn't seem so. At least, I can't find where main thread locks the > 0x040ebee8 monitor: > > "Thread-1": > waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), > which is held by "main" > > > "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for > monitor entry [0x0185e000] > java.lang.Thread.State: BLOCKED (on object monitor) > at java.lang.ClassLoader.loadClass(ClassLoader.java:406) > - waiting to lock <0x03f624b8> (a java.lang.Object) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > at > org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:44) > at > org.apache.river.concurrent.StrongReference.(StrongReference.java:57) > at > org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) > at > org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) > at > org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) > at > org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > at java.lang.System.checkIO(System.java:253) > at java.lang.System.setErr(System.java:199) > at > org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) > > > ...no mention of 0x040ebee8 in main thread...?! > > Peter > >> >> The policy provider in use thread confines PermissionCollection >> instances, which never leave the cpu cache, they are discarded >> immediately after use. >> >> The problem here is that two different threads are attempting to load >> the same class at the same time. >> >> I'll have to make sure that all classes are loaded before the >> security manager becomes the system security manager, either that or >> eliminate the AccessControlContext check permission cache. >> >> While the standard java security manager and policy provider consume >> around 10% cpu, this consumes less than 1%. >> >> On 6/02/2016 9:57 PM, David Holmes wrote: >>> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>>> deadlock prone? Guess I was wrong. >>> >>> The deadlock is introduced by a custom security manager. SM's play a >>> critical role in many parts of the core libraries so if they utilize >>> synchronization then they can easily introduce deadlock. >>> >>>> Observation: On an unrelated occassion, I had a URLClassLoader >>>> synchronization hotspot (well not standard URLClassLoader, but a high >>>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>>> instead of URL DNS lookup), I solved that problem by thread confining >>>> class loading. I've never experienced deadlock using thread >>>> confinement, initially I was worried that there would be >>>> interdependencies, however I didn't experience a problem, as the >>>> single >>>> thread would always load the classes it needed. >>>> >>>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>>> strategy, I found thread confinement was very performant. >>> >>> Are you suggesting that a class-load request is actually transmitted >>> to a single class-loading thread, while the initial thread blocks >>> until the loading is complete? That would still seem prone to the >>> same SM related deadlock scenario. And it would not generally be >>> particularly performant. >> >> I thought that too, until I tried it. >> >> Would you like to see some performance figures? >> >> Cheers, >> >> Peter. >> >>> >>> David >>> >>>> Regards, >>>> >>>> Peter. >>>> >>>> 2016-02-06 21:06:07 >>>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>>> >>>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>>> runnable [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>>> nid=0x111c waiting on condition [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 nid=0x13d8 >>>> waiting on condition [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 >>>> nid=0x10b8 >>>> runnable [0x00000000] >>>> java.lang.Thread.State: RUNNABLE >>>> >>>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>>> monitor entry [0x03ded000] >>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at >>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>> >>>> at >>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>> >>>> >>>> at >>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>> - locked <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>> >>>> >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>> >>>> >>>> - locked <0x03ef8b30> (a >>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> >>>> at >>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>> >>>> >>>> ava:180) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>> >>>> >>>> ) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> >>>> >>>> at java.lang.Thread.run(Thread.java:744) >>>> >>>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>>> Object.wait() [0x03cff000] >>>> java.lang.Thread.State: WAITING (on object monitor) >>>> at java.lang.Object.wait(Native Method) >>>> - waiting on <0x03e056d8> (a >>>> java.lang.ref.ReferenceQueue$Lock) >>>> at >>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>>> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>>> at >>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>>> at >>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>>> >>>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>>> nid=0x1398 in Object.wait() [0x144cf000] >>>> java.lang.Thread.State: WAITING (on object monitor) >>>> at java.lang.Object.wait(Native Method) >>>> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >>>> at java.lang.Object.wait(Object.java:502) >>>> at >>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>>> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >>>> >>>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>>> monitor >>>> entry [0x0185e000] >>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at java.lang.System.checkIO(System.java:253) >>>> at java.lang.System.setErr(System.java:199) >>>> at >>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>> >>>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>>> >>>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 >>>> waiting on >>>> condition >>>> >>>> JNI global references: 19 >>>> >>>> >>>> Found one Java-level deadlock: >>>> ============================= >>>> "Thread-1": >>>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>>> which is held by "main" >>>> "main": >>>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>>> java.lang.Object), >>>> which is held by "Thread-1" >>>> >>>> Java stack information for the threads listed above: >>>> =================================================== >>>> "Thread-1": >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at >>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>> >>>> at >>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>> >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>> >>>> at >>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>> >>>> >>>> at >>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>> - locked <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>> >>>> >>>> at >>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>> >>>> >>>> - locked <0x03ef8b30> (a >>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>> at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>> >>>> at >>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>> >>>> >>>> ava:180) >>>> at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>> >>>> >>>> ) >>>> at >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>> >>>> >>>> at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>> >>>> >>>> at java.lang.Thread.run(Thread.java:744) >>>> "main": >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>> at >>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at >>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>> >>>> at >>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>> >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>> >>>> at >>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>> >>>> >>>> at >>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>> >>>> >>>> at java.lang.System.checkIO(System.java:253) >>>> at java.lang.System.setErr(System.java:199) >>>> at >>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>> >>>> Found 1 deadlock. >> > > > _______________________________________________ > Concurrency-interest mailing list > Concurrency-interest at cs.oswego.edu > http://cs.oswego.edu/mailman/listinfo/concurrency-interest From jini at zeus.net.au Sun Feb 7 09:54:08 2016 From: jini at zeus.net.au (Peter) Date: Sun, 07 Feb 2016 19:54:08 +1000 Subject: [concurrency-interest] ClassLoader deadlock In-Reply-To: <56B7023C.5030708@gmail.com> References: <56B5DB78.1070502@zeus.net.au> <56B5DF8C.5010201@oracle.com> <56B5E446.9020205@zeus.net.au> <56B63B11.20903@gmail.com> <56B6668A.3030106@zeus.net.au> <56B7023C.5030708@gmail.com> Message-ID: <56B71440.5000108@zeus.net.au> Thanks Peter & David, That's good to know. I've altered the SecurityManager to perform a permission check prior to becoming the security manager, this ensures the cache has been created, to avoid any recursive deadlock prone calls. The class that both threads are attempting to load and init is called org.apache.river.concurrent.ReferenceIterator. The code can be found here: https://github.com/pfirmstone/river-internet/tree/Input-validation-for-Serialization Cheers, Peter. On 7/02/2016 6:37 PM, Peter Levart wrote: > > > On 02/06/2016 10:32 PM, Peter wrote: >> The 0x040ebee8 monitor is most likely being held by native code. >> >> Regards, >> >> Peter Firmstone. > > Ok, but where? The deadlock report says it is held by main thread. > Somewhere between the following two java frames? > > "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for > monitor entry [0x0185e000] > ... > ... > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > --here-- > at > org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) > > > As I understand this is where JVM resolves some class that for the 1st > time while ReferenceCollection.iterator is being executed. So before > calling-back into ClassLoader.loadClass, it acquires a monitor lock on > some int[] object? > > > Anyway. Looking at the following part of Thread-1 stacktrace: > > at > org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) > at > java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) > at > java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) > at > java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) > at > java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) > at > java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) > at > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) > at > java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) > > ...I can see you are using some early release of JDK 8 where > ThreadLocalRandom initialization involved obtaining a hardware address > of some interface. This in turn checks for security manager. Since JDK > 8u45 this code has been removed and only current time is used to > derive initial seed, so you might not see this deadlock any more with > JDK 8u45 and later. > > Regards, Peter > >> >> >> On 7/02/2016 4:27 AM, Peter Levart wrote: >>> >>> >>> On 02/06/2016 01:17 PM, Peter Firmstone wrote: >>>> The security manager is non blocking, unfortunately java system >>>> classes aren't. >>> >>> It doesn't seem so. At least, I can't find where main thread locks >>> the 0x040ebee8 monitor: >>> >>> "Thread-1": >>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>> which is held by "main" >>> >>> >>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>> monitor entry [0x0185e000] >>> java.lang.Thread.State: BLOCKED (on object monitor) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>> at >>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at >>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>> at >>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>> at >>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>> at >>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>> at >>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>> at >>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>> at >>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>> at >>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>> at java.lang.System.checkIO(System.java:253) >>> at java.lang.System.setErr(System.java:199) >>> at >>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>> >>> >>> ...no mention of 0x040ebee8 in main thread...?! >>> >>> Peter >>> >>>> >>>> The policy provider in use thread confines PermissionCollection >>>> instances, which never leave the cpu cache, they are discarded >>>> immediately after use. >>>> >>>> The problem here is that two different threads are attempting to >>>> load the same class at the same time. >>>> >>>> I'll have to make sure that all classes are loaded before the >>>> security manager becomes the system security manager, either that >>>> or eliminate the AccessControlContext check permission cache. >>>> >>>> While the standard java security manager and policy provider >>>> consume around 10% cpu, this consumes less than 1%. >>>> >>>> On 6/02/2016 9:57 PM, David Holmes wrote: >>>>> On 6/02/2016 9:39 PM, Peter Firmstone wrote: >>>>>> Hmm, thought the new parallel lock stategy in ClassLoader wasn't >>>>>> deadlock prone? Guess I was wrong. >>>>> >>>>> The deadlock is introduced by a custom security manager. SM's play >>>>> a critical role in many parts of the core libraries so if they >>>>> utilize synchronization then they can easily introduce deadlock. >>>>> >>>>>> Observation: On an unrelated occassion, I had a URLClassLoader >>>>>> synchronization hotspot (well not standard URLClassLoader, but a >>>>>> high >>>>>> performance RFC3986 URL ClassLoader, that use normalized RFC3986 URI >>>>>> instead of URL DNS lookup), I solved that problem by thread >>>>>> confining >>>>>> class loading. I've never experienced deadlock using thread >>>>>> confinement, initially I was worried that there would be >>>>>> interdependencies, however I didn't experience a problem, as the >>>>>> single >>>>>> thread would always load the classes it needed. >>>>>> >>>>>> I'm wondering if multiple locks for ClassLoader's might be the wrong >>>>>> strategy, I found thread confinement was very performant. >>>>> >>>>> Are you suggesting that a class-load request is actually >>>>> transmitted to a single class-loading thread, while the initial >>>>> thread blocks until the loading is complete? That would still seem >>>>> prone to the same SM related deadlock scenario. And it would not >>>>> generally be particularly performant. >>>> >>>> I thought that too, until I tried it. >>>> >>>> Would you like to see some performance figures? >>>> >>>> Cheers, >>>> >>>> Peter. >>>> >>>>> >>>>> David >>>>> >>>>>> Regards, >>>>>> >>>>>> Peter. >>>>>> >>>>>> 2016-02-06 21:06:07 >>>>>> Full thread dump Java HotSpot(TM) Client VM (25.0-b70 mixed mode): >>>>>> >>>>>> "Service Thread" #9 daemon prio=9 os_prio=0 tid=0x14387400 nid=0xc54 >>>>>> runnable [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "C1 CompilerThread0" #8 daemon prio=9 os_prio=2 tid=0x14332c00 >>>>>> nid=0x111c waiting on condition [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "Attach Listener" #7 daemon prio=5 os_prio=2 tid=0x14331c00 >>>>>> nid=0x13d8 >>>>>> waiting on condition [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "Signal Dispatcher" #6 daemon prio=9 os_prio=2 tid=0x14331400 >>>>>> nid=0x10b8 >>>>>> runnable [0x00000000] >>>>>> java.lang.Thread.State: RUNNABLE >>>>>> >>>>>> "Thread-1" #5 prio=10 os_prio=2 tid=0x14318800 nid=0x4a8 waiting for >>>>>> monitor entry [0x03ded000] >>>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at >>>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>>> >>>>>> >>>>>> at >>>>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>>> - locked <0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>>> >>>>>> >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>>> >>>>>> >>>>>> - locked <0x03ef8b30> (a >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>>> at >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>>> >>>>>> >>>>>> ava:180) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>>> >>>>>> >>>>>> ) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>>> >>>>>> >>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>> >>>>>> "Finalizer" #3 daemon prio=8 os_prio=1 tid=0x14277800 nid=0x15cc in >>>>>> Object.wait() [0x03cff000] >>>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>>> at java.lang.Object.wait(Native Method) >>>>>> - waiting on <0x03e056d8> (a >>>>>> java.lang.ref.ReferenceQueue$Lock) >>>>>> at >>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:142) >>>>>> - locked <0x03e056d8> (a java.lang.ref.ReferenceQueue$Lock) >>>>>> at >>>>>> java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:158) >>>>>> at >>>>>> java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209) >>>>>> >>>>>> "Reference Handler" #2 daemon prio=10 os_prio=2 tid=0x14271c00 >>>>>> nid=0x1398 in Object.wait() [0x144cf000] >>>>>> java.lang.Thread.State: WAITING (on object monitor) >>>>>> at java.lang.Object.wait(Native Method) >>>>>> - waiting on <0x03e05200> (a java.lang.ref.Reference$Lock) >>>>>> at java.lang.Object.wait(Object.java:502) >>>>>> at >>>>>> java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157) >>>>>> - locked <0x03e05200> (a java.lang.ref.Reference$Lock) >>>>>> >>>>>> "main" #1 prio=5 os_prio=0 tid=0x017cf400 nid=0x1284 waiting for >>>>>> monitor >>>>>> entry [0x0185e000] >>>>>> java.lang.Thread.State: BLOCKED (on object monitor) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at java.lang.System.checkIO(System.java:253) >>>>>> at java.lang.System.setErr(System.java:199) >>>>>> at >>>>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>>> >>>>>> "VM Thread" os_prio=2 tid=0x1426e400 nid=0x16a8 runnable >>>>>> >>>>>> "VM Periodic Task Thread" os_prio=2 tid=0x14388400 nid=0x17a8 >>>>>> waiting on >>>>>> condition >>>>>> >>>>>> JNI global references: 19 >>>>>> >>>>>> >>>>>> Found one Java-level deadlock: >>>>>> ============================= >>>>>> "Thread-1": >>>>>> waiting to lock monitor 0x142766ac (object 0x040ebee8, a [I), >>>>>> which is held by "main" >>>>>> "main": >>>>>> waiting to lock monitor 0x14274a3c (object 0x03f624b8, a >>>>>> java.lang.Object), >>>>>> which is held by "Thread-1" >>>>>> >>>>>> Java stack information for the threads listed above: >>>>>> =================================================== >>>>>> "Thread-1": >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at >>>>>> java.net.NetworkInterface.getHardwareAddress(NetworkInterface.java:447) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.initialSeed(ThreadLocalRandom.java:158) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadLocalRandom.(ThreadLocalRandom.java:137) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.fullAddCount(ConcurrentHashMap.java:2526) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2266) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) >>>>>> >>>>>> at >>>>>> java.util.concurrent.ConcurrentHashMap.putIfAbsent(ConcurrentHashMap.java:1535) >>>>>> >>>>>> >>>>>> at >>>>>> java.lang.ClassLoader.getClassLoadingLock(ClassLoader.java:463) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:404) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:411) >>>>>> - locked <0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$SnapshotK.(NonBlockingHashMap.java:1167) >>>>>> >>>>>> >>>>>> at >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap$2.iterator(NonBlockingHashMap.java:1200) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor$EnqueGarbageTask.run(ReferenceProcessor.java:166) >>>>>> >>>>>> >>>>>> - locked <0x03ef8b30> (a >>>>>> org.cliffc.high_scale_lib.NonBlockingHashMap) >>>>>> at >>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) >>>>>> >>>>>> at >>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j >>>>>> >>>>>> >>>>>> ava:180) >>>>>> at >>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294 >>>>>> >>>>>> >>>>>> ) >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >>>>>> >>>>>> >>>>>> at >>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >>>>>> >>>>>> >>>>>> at java.lang.Thread.run(Thread.java:744) >>>>>> "main": >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:406) >>>>>> - waiting to lock <0x03f624b8> (a java.lang.Object) >>>>>> at >>>>>> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceCollection.iterator(ReferenceCollection.java:124) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceSet.hashCode(ReferenceSet.java:65) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:44) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.StrongReference.(StrongReference.java:57) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceFactory.create(ReferenceFactory.java:64) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:128) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceProcessor.referenced(ReferenceProcessor.java:44) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceMap.wrapVal(ReferenceMap.java:244) >>>>>> >>>>>> at >>>>>> org.apache.river.concurrent.ReferenceConcurrentMap.putIfAbsent(ReferenceConcurrentMap.java:68) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:261) >>>>>> >>>>>> >>>>>> at >>>>>> org.apache.river.api.security.CombinerSecurityManager.checkPermission(CombinerSecurityManager.java:202) >>>>>> >>>>>> >>>>>> at java.lang.System.checkIO(System.java:253) >>>>>> at java.lang.System.setErr(System.java:199) >>>>>> at >>>>>> org.apache.river.qa.harness.MasterTest.main(MasterTest.java:84) >>>>>> >>>>>> Found 1 deadlock. >>>> >>> >>> >>> _______________________________________________ >>> Concurrency-interest mailing list >>> Concurrency-interest at cs.oswego.edu >>> http://cs.oswego.edu/mailman/listinfo/concurrency-interest >> > From anubis1 at linux-ecke.de Mon Feb 8 19:29:56 2016 From: anubis1 at linux-ecke.de (Thomas Kamps) Date: Mon, 08 Feb 2016 20:29:56 +0100 Subject: New Collection framework? Message-ID: <2485208.rZGvsojIaB@linux-vgv1> Hi everyone, Did you ever think about a new Collection framework for Java? My main complains about the java.util Collections are: * No co/contra variance for assigning. Which forbids assigments like List foo = new List(); Currently it can be bypassed when using wildcards, but then you cannot add items to foo. Even more problematic are interfaces with some kind of List> as parameter * The main Collection interface allowes too much. If you get a Collection and you should not modify it, your programm will either behave stragly with a hard to detect bug, or crash with some kind of UnsupportedOperationException, when you do it. I have my problems with UnsupprtedOperationException, because when i get a Collection, i have to read the documentation (if existing) to know what i am allowed to do, or run unexpectedly into an exception. Would it be great to hava an "readonly" interface, which simply does not have and modifing methods? The other possible reason could be the Valhalla project, which tries to introduce value types, to enable Collections with primitive type parameters. There is (or was?) a discussion how to handle the remove methods on List: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2015-December/000028.html There is a suggestion for partial methods to solve the problem, but i think partial methods makes the language even more complicated to compile and more difficult to understand. All theese problems could be resolved by just letting the java.util Collection Framework as it is. And make a new one. I have created a proposal for it, which can be easily (modulo class names) used together with the java.util Collections. This is achieved by having wrapper Collections and converter in both directions. https://github.com/Blahord/BetterCollections Note: This is the first release with the target to have a clean and small as possible API. Since it makes heavily use of wildcards, it is some kind of bureaucratic. This could be reduced, by * Adding a shortcur for wildcards like "?Type" or "? extends Type" and "!Type" for "? super Type" or * Allowing co and contra variance for type parameters. From dhoffer6 at gmail.com Tue Feb 9 18:51:11 2016 From: dhoffer6 at gmail.com (David Hoffer) Date: Tue, 9 Feb 2016 11:51:11 -0700 Subject: OpenJDK bug question (JDK-6519463) Message-ID: We have a production environment where we are getting errors similar to what was reported in JDK-6519463. I would like to test our code with the test file (buggedfile.bin) used in this bug report but the file is posted on an ftp server that is not publicly available. Can this file be attached to the bug report or just email me a copy? Thanks, -Dave From joe.darcy at oracle.com Tue Feb 9 19:34:04 2016 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 9 Feb 2016 11:34:04 -0800 Subject: JDK 9 RFR of JDK-8149492: Problem list CheckEncodings.sh Message-ID: <56BA3F2C.1040507@oracle.com> Hello, As noted in bug JDK-7008363, the test test/java/lang/StringCoding/CheckEncodings.sh "does nothing and is very slow at that" Until and unless the problems noted in JDK-7008363 are addressed, there is little point in running the test and it should be put on the problem list. Please review the patch below which does this. Thanks, -Joe --- a/test/ProblemList.txt Mon Feb 08 17:37:46 2016 +0100 +++ b/test/ProblemList.txt Tue Feb 09 11:33:23 2016 -0800 @@ -133,6 +133,9 @@ # 8029891 java/lang/ClassLoader/deadlock/GetResource.java generic-all +# 7008363 +java/lang/StringCoding/CheckEncodings.sh generic-all + ############################################################################ # jdk_instrument From roger.riggs at oracle.com Tue Feb 9 19:39:02 2016 From: roger.riggs at oracle.com (Roger Riggs) Date: Tue, 9 Feb 2016 14:39:02 -0500 Subject: JDK 9 RFR of JDK-8149492: Problem list CheckEncodings.sh In-Reply-To: <56BA3F2C.1040507@oracle.com> References: <56BA3F2C.1040507@oracle.com> Message-ID: <56BA4056.5050309@oracle.com> +1 On 2/9/16 2:34 PM, joe darcy wrote: > Hello, > > As noted in bug JDK-7008363, the test > > test/java/lang/StringCoding/CheckEncodings.sh > > "does nothing and is very slow at that" > > Until and unless the problems noted in JDK-7008363 are addressed, > there is little point in running the test and it should be put on the > problem list. > > Please review the patch below which does this. > > Thanks, > > -Joe > > --- a/test/ProblemList.txt Mon Feb 08 17:37:46 2016 +0100 > +++ b/test/ProblemList.txt Tue Feb 09 11:33:23 2016 -0800 > @@ -133,6 +133,9 @@ > # 8029891 > java/lang/ClassLoader/deadlock/GetResource.java generic-all > > +# 7008363 > +java/lang/StringCoding/CheckEncodings.sh generic-all > + > ############################################################################ > > > # jdk_instrument > > From stuart.marks at oracle.com Tue Feb 9 21:57:46 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 9 Feb 2016 13:57:46 -0800 Subject: RFR 8046339: sun.rmi.transport.DGCAckHandler leaks memory In-Reply-To: <56B99281.605@oracle.com> References: <56B99281.605@oracle.com> Message-ID: <56BA60DA.1050601@oracle.com> Hi Ivan, Thanks for picking up this old bug. The fix and the change to the regression test look good. s'marks On 2/8/16 11:17 PM, Ivan Gerasimov wrote: > Hello! > > This is a request to help review a fix for the problem which can lead to memory > leakage under some rare circumstances. > The fix itself is quite simple - it just makes sure the handler being released > is also removed from the set of tracked handlers. > > I've also augmented an existing regression test to make sure we're not observing > the memory leak anymore. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8046339 > WEBREV: http://cr.openjdk.java.net/~igerasim/8046339/00/webrev/ > > Would you please help review it? > > Sincerely yours, > Ivan From stuart.marks at oracle.com Tue Feb 9 22:04:31 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Tue, 9 Feb 2016 14:04:31 -0800 Subject: JDK 9 RFR of JDK-8149492: Problem list CheckEncodings.sh In-Reply-To: <56BA3F2C.1040507@oracle.com> References: <56BA3F2C.1040507@oracle.com> Message-ID: <56BA626F.1010100@oracle.com> Hi Joe, Adding this test to the problem list looks fine to me. s'marks On 2/9/16 11:34 AM, joe darcy wrote: > Hello, > > As noted in bug JDK-7008363, the test > > test/java/lang/StringCoding/CheckEncodings.sh > > "does nothing and is very slow at that" > > Until and unless the problems noted in JDK-7008363 are addressed, there is > little point in running the test and it should be put on the problem list. > > Please review the patch below which does this. > > Thanks, > > -Joe > > --- a/test/ProblemList.txt Mon Feb 08 17:37:46 2016 +0100 > +++ b/test/ProblemList.txt Tue Feb 09 11:33:23 2016 -0800 > @@ -133,6 +133,9 @@ > # 8029891 > java/lang/ClassLoader/deadlock/GetResource.java generic-all > > +# 7008363 > +java/lang/StringCoding/CheckEncodings.sh generic-all > + > ############################################################################ > > # jdk_instrument > > From jeremymanson at google.com Wed Feb 10 00:09:06 2016 From: jeremymanson at google.com (Jeremy Manson) Date: Tue, 9 Feb 2016 16:09:06 -0800 Subject: API review of VarHandles In-Reply-To: <4EC42B69-2603-43BC-AB29-BAD8EF53F90E@oracle.com> References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <4EC42B69-2603-43BC-AB29-BAD8EF53F90E@oracle.com> Message-ID: Thanks, Paul! I agree that the wording about reorderings is probably okay. I probably would have added the sentence about implementation-dependence, but I can see why you might want to avoid it. Jeremy On Tue, Feb 9, 2016 at 3:12 AM, Paul Sandoz wrote: > Hi Jeremy, > > Sorry for the late reply. Catching after some distractions and being away. > > > > On 22 Jan 2016, at 19:03, Jeremy Manson wrote: > > > > Couple of thoughts: > > The Java Language Specification permits operations to be executed in > orders different than are apparent in program source code, subject to > constraints arising, for example, from the use of locks, volatile fields or > VarHandles. The static methods, fullFence, acquireFence, releaseFence, > loadLoadFence andstoreStoreFence, can also be used to impose constraints. > Their specifications are phrased in terms of the lack of "reorderings" -- > observable ordering effects that might otherwise occur if the fence were > not present. > > > > > > There is no notion of reordering (per se) in the JLS; the fact that > reorderings can occur is just implied by the way programs can be observed > to behave. So, these fence operations don't map to anything > non-implementation dependent. I don't think it would be impossible to fix > the spec to define something that works the way you want (similar to the > final field semantics), but it isn't in there already. You might want to > call that out (either by saying this behavior is best effort > implementation-dependent or by fixing the spec). > > > > Until we update the JMM spec, pushed out beyond 9, we are in a bit of a > bind and have to hand wave a bit for both access mode methods and fences, > both of which talk about reorderings (see getAcquire/setRelease for > example). > > There is the following at the end, but it?s easy to loose sight of with > all the other sections: > > * @apiNote > * More precise phrasing of the specification of access mode methods and > memory > * fence methods may accompany future updates of the Java Language > * Specification. > > I moved that up into the fences paragraph. > > *

In addition to supporting access to variables under various access > modes, > * a set of static methods, referred to as memory fence methods, is also > * provided for fine-grained control of memory ordering. > * > * The Java Language Specification permits operations to be executed in > * orders different than are apparent in program source code, subject to > * constraints arising, for example, from the use of locks, {@code volatile} > * fields or VarHandles. The static methods, {@link #fullFence fullFence}, > * {@link #acquireFence acquireFence}, {@link #releaseFence releaseFence}, > * {@link #loadLoadFence loadLoadFence} and {@link #storeStoreFence > * storeStoreFence}, can also be used to impose constraints. Their > * specifications, as is the case for certain access modes, are phrased in > terms > * of the lack of "reorderings" -- observable ordering effects that might > * otherwise occur if the fence was not present. More precise phrasing of > the > * specification of access mode methods and memory fence methods may > accompany > * future updates of the Java Language Specification. > > > I could append another sentence: > > Until then, such reordering behaviour is considered > implementation-dependent on a best-effort basis. > > But perhaps the less we say the better? > > > > If a VarHandle references a read-only variable (for example a final > field) then write, atomic update and numeric atomic update access modes are > not supported and corresponding methods throw UnsupportedOperationException. > > > > Are you sure you want to limit it in this way? There are an awful lot > of calls to setAccessible in the world of reflection. And writing to final > fields *does* map to something sensible in the spec. In fact, it would be > great to have something that wasn't quite such a blunt instrument as > setAccessible. > > Indeed, very much a blunt instrument. > > Note that setAccessible would otherwise be required to be called on the > Field before being unreflected (either for unreflectVarHandle or, as is the > case today, unreflectSetter/Getter). > > I don?t wanna widen the accessibility hole of this blunt instrument right > now; it?s easy to widen rather than shrink later on if need be. > > The motivation behind this is we want to move towards a world "final > really means final" and thus lots of nice optimisations may ensue. It may > be a long and windy road to get there but we have some plans and are > starting to work some routes towards that goal. > > > > > > getVolatile > > public final Object getVolatile(Object... args) > > Returns the value of a variable, with memory semantics of reading a > volatile variable. > > > > Reading *which* volatile variable? You probably mean that all > getVolatiles and setVolatiles provide semantics that behave as if the > variable being written / read was declared volatile in the first place, but > it is worth calling out. > > > > /** > * Returns the value of a variable, with memory semantics of reading as if > * the variable was declared {@code volatile}. > ... > Object getVolatile(Object... args); > > /** > * Sets the value of a variable to the {@code newValue}, with memory > * semantics of setting as if the variable was declared {@code volatile}. > ? > void setVolatile(Object... args); > > I also updated the plain accessors using similar ?as if? language. > > > > Nits: > > > > The following nits are copied and translated from documentation on > MethodHandle. > > > > As is usual with virtual methods, source-level calls to access mode > methods compile to an invokevirtual instruction. More unusually, the > compiler must record the actual argument types, and may not perform method > invocation conversions on the arguments. Instead, it must push them on the > stack according to their own unconverted types. The VarHandle object itself > is pushed on the stack before the arguments. The compiler then calls the > VarHandle with a symbolic type descriptor which describes the argument and > return types. > > > > This is somewhat oddly worded. The compiler doesn't push arguments on a > stack or call anything - it generates instructions that do that. > > > > I updated to: > > * As is usual with virtual methods, source-level calls to access mode > methods > * compile to an {@code invokevirtual} instruction. More unusually, the > * compiler must record the actual argument types, and may not perform > method > * invocation conversions on the arguments. Instead, it must generate > * instructions to push them on the stack according to their own unconverted > * types. The VarHandle object itself will be pushed on the stack before > the > * arguments. The compiler then generates an {@code invokevirtual} > instruction > * that invokes the access mode method with a symbolic type descriptor which > * describes the argument and return types. > > > > The first time a invokevirtual instruction is executed it is linked, by > symbolically resolving the names in the instruction and verifying that the > method call is statically legal > > > > I keep trying not to call this out as a nit, but there should be no > comma between "linked" and "by?. > > Updated. > > In both cases I also updated the relevant areas in MethodHandle. > > Thanks, > Paul. > From hboehm at google.com Wed Feb 10 01:03:22 2016 From: hboehm at google.com (Hans Boehm) Date: Tue, 9 Feb 2016 17:03:22 -0800 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <4EC42B69-2603-43BC-AB29-BAD8EF53F90E@oracle.com> Message-ID: I found this a bit scary for naive readers: * The Java Language Specification permits operations to be executed in * orders different than are apparent in program source code, subject to * constraints arising, for example, from the use of locks, {@code volatile} * fields or VarHandles. I would change it to * The Java Language Specification permits other threads to observe operations * as if they were executed in orders different than are apparent in program source code, * subject to constraints arising, for example, from the use of locks, {@code volatile} * fields or VarHandles. to make it clear that we are not talking about intra-thread semantics here. That way it will hopefully only be scary to experts, as intended. On Tue, Feb 9, 2016 at 4:09 PM, Jeremy Manson wrote: > Thanks, Paul! I agree that the wording about reorderings is probably > okay. I probably would have added the sentence about > implementation-dependence, but I can see why you might want to avoid it. > > Jeremy > > On Tue, Feb 9, 2016 at 3:12 AM, Paul Sandoz > wrote: > > > Hi Jeremy, > > > > Sorry for the late reply. Catching after some distractions and being > away. > > > > > > > On 22 Jan 2016, at 19:03, Jeremy Manson > wrote: > > > > > > Couple of thoughts: > > > The Java Language Specification permits operations to be executed in > > orders different than are apparent in program source code, subject to > > constraints arising, for example, from the use of locks, volatile fields > or > > VarHandles. The static methods, fullFence, acquireFence, releaseFence, > > loadLoadFence andstoreStoreFence, can also be used to impose constraints. > > Their specifications are phrased in terms of the lack of "reorderings" -- > > observable ordering effects that might otherwise occur if the fence were > > not present. > > > > > > > > > There is no notion of reordering (per se) in the JLS; the fact that > > reorderings can occur is just implied by the way programs can be observed > > to behave. So, these fence operations don't map to anything > > non-implementation dependent. I don't think it would be impossible to > fix > > the spec to define something that works the way you want (similar to the > > final field semantics), but it isn't in there already. You might want to > > call that out (either by saying this behavior is best effort > > implementation-dependent or by fixing the spec). > > > > > > > Until we update the JMM spec, pushed out beyond 9, we are in a bit of a > > bind and have to hand wave a bit for both access mode methods and fences, > > both of which talk about reorderings (see getAcquire/setRelease for > > example). > > > > There is the following at the end, but it?s easy to loose sight of with > > all the other sections: > > > > * @apiNote > > * More precise phrasing of the specification of access mode methods and > > memory > > * fence methods may accompany future updates of the Java Language > > * Specification. > > > > I moved that up into the fences paragraph. > > > > *

In addition to supporting access to variables under various access > > modes, > > * a set of static methods, referred to as memory fence methods, is also > > * provided for fine-grained control of memory ordering. > > * > > * The Java Language Specification permits operations to be executed in > > * orders different than are apparent in program source code, subject to > > * constraints arising, for example, from the use of locks, {@code > volatile} > > * fields or VarHandles. The static methods, {@link #fullFence > fullFence}, > > * {@link #acquireFence acquireFence}, {@link #releaseFence releaseFence}, > > * {@link #loadLoadFence loadLoadFence} and {@link #storeStoreFence > > * storeStoreFence}, can also be used to impose constraints. Their > > * specifications, as is the case for certain access modes, are phrased in > > terms > > * of the lack of "reorderings" -- observable ordering effects that might > > * otherwise occur if the fence was not present. More precise phrasing of > > the > > * specification of access mode methods and memory fence methods may > > accompany > > * future updates of the Java Language Specification. > > > > > > I could append another sentence: > > > > Until then, such reordering behaviour is considered > > implementation-dependent on a best-effort basis. > > > > But perhaps the less we say the better? > > > > > > > If a VarHandle references a read-only variable (for example a final > > field) then write, atomic update and numeric atomic update access modes > are > > not supported and corresponding methods throw > UnsupportedOperationException. > > > > > > Are you sure you want to limit it in this way? There are an awful lot > > of calls to setAccessible in the world of reflection. And writing to > final > > fields *does* map to something sensible in the spec. In fact, it would > be > > great to have something that wasn't quite such a blunt instrument as > > setAccessible. > > > > Indeed, very much a blunt instrument. > > > > Note that setAccessible would otherwise be required to be called on the > > Field before being unreflected (either for unreflectVarHandle or, as is > the > > case today, unreflectSetter/Getter). > > > > I don?t wanna widen the accessibility hole of this blunt instrument right > > now; it?s easy to widen rather than shrink later on if need be. > > > > The motivation behind this is we want to move towards a world "final > > really means final" and thus lots of nice optimisations may ensue. It may > > be a long and windy road to get there but we have some plans and are > > starting to work some routes towards that goal. > > > > > > > > > > getVolatile > > > public final Object getVolatile(Object... args) > > > Returns the value of a variable, with memory semantics of reading a > > volatile variable. > > > > > > Reading *which* volatile variable? You probably mean that all > > getVolatiles and setVolatiles provide semantics that behave as if the > > variable being written / read was declared volatile in the first place, > but > > it is worth calling out. > > > > > > > /** > > * Returns the value of a variable, with memory semantics of reading as > if > > * the variable was declared {@code volatile}. > > ... > > Object getVolatile(Object... args); > > > > /** > > * Sets the value of a variable to the {@code newValue}, with memory > > * semantics of setting as if the variable was declared {@code volatile}. > > ? > > void setVolatile(Object... args); > > > > I also updated the plain accessors using similar ?as if? language. > > > > > > > Nits: > > > > > > > The following nits are copied and translated from documentation on > > MethodHandle. > > > > > > > As is usual with virtual methods, source-level calls to access mode > > methods compile to an invokevirtual instruction. More unusually, the > > compiler must record the actual argument types, and may not perform > method > > invocation conversions on the arguments. Instead, it must push them on > the > > stack according to their own unconverted types. The VarHandle object > itself > > is pushed on the stack before the arguments. The compiler then calls the > > VarHandle with a symbolic type descriptor which describes the argument > and > > return types. > > > > > > This is somewhat oddly worded. The compiler doesn't push arguments on > a > > stack or call anything - it generates instructions that do that. > > > > > > > I updated to: > > > > * As is usual with virtual methods, source-level calls to access mode > > methods > > * compile to an {@code invokevirtual} instruction. More unusually, the > > * compiler must record the actual argument types, and may not perform > > method > > * invocation conversions on the arguments. Instead, it must generate > > * instructions to push them on the stack according to their own > unconverted > > * types. The VarHandle object itself will be pushed on the stack before > > the > > * arguments. The compiler then generates an {@code invokevirtual} > > instruction > > * that invokes the access mode method with a symbolic type descriptor > which > > * describes the argument and return types. > > > > > > > The first time a invokevirtual instruction is executed it is linked, by > > symbolically resolving the names in the instruction and verifying that > the > > method call is statically legal > > > > > > I keep trying not to call this out as a nit, but there should be no > > comma between "linked" and "by?. > > > > Updated. > > > > In both cases I also updated the relevant areas in MethodHandle. > > > > Thanks, > > Paul. > > > From steve.drach at oracle.com Wed Feb 10 01:04:01 2016 From: steve.drach at oracle.com (Steve Drach) Date: Tue, 9 Feb 2016 17:04:01 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56B1C2CC.7080102@oracle.com> References: <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC! .7080102@oracle.com> Message-ID: <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> Hi, Yet another webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.06/ , with a change to JarEntryIterator to fix a problem discovered by performance tests ? calling hasNext() twice as often as needed. I also removed the @since 9 tags from the methods entries() and stream(), and added an additional sentence to the spec for each of those methods that clarifies what a base entry is (actually is not). > I think having stream and entries do this is right although I would like to see some performance data if possible. See http://cr.openjdk.java.net/~sdrach/8132734/JarFile%20Performance.pdf I used JMH to run the benchmark. See http://cr.openjdk.java.net/~sdrach/8132734/MyBenchmark.java . I used two jar files, the rt.jar file from JDK 8 that has 20653 entries and the multi-release.jar found in the test directory with 14 entries. Obviously rt.jar is not a multi-release jar file. The first two tables (1 and 2) are comparable and the second two tables are somewhat comparable (3 and 4). Tables 1 and 2 have 4 sections that show the results of tests on the two jar files in 4 configurations of JarFile. The tests were done with a JarFile object constructed without the Release object argument, essentially the legacy constructor. The section labeled "JDK 8 JarFile? was done with JDK 8u66. The section labeled ?JDK 9 JarFile? was done with the latest build of openjdk/jdk9/dev without any changes in my 8132734 changeset. I chose this section as the reference, so the last column shows the values normalized to 1 micro/milli second per operation (rt.jar times are in milliseconds and multi-release.jar times are in microseconds). It should be obvious that JDK 9 is much faster than JDK 8, somewhere on the order of 5-6 times faster. I think that is because ZipFile doesn?t use JNI in JDK 9. Of the two remaining sections in Tables 1 and 2, the section labeled ?MultiRelease JarFile? differs from the section labeled ?MultiRelease JarFile, new iterator? only in the JarEntryIterator class. The first one uses the original iterator in JarFile.java that can be seen starting with line 551 of webrev.04 , and the new iterator starts with line 553 of webrev.06 . The results are strange. The new, more complex, iterator appears to be faster than the old, simpler, iterator. I double, and triple, checked it, but it was always faster. I used jitwatch to look at the hotspot logs generated during compilation and neither method was compiled. I suppose I could dig into it further but decided not to. Consider it good news. The results do show that the multi-release enhancement slows JarFile entries/stream down by 2-18% depending on the size of the jar file. But they are still much better than the JDK 8 values. > Also I would expect that if a JAR file is not mult-release but the library opens it with Release.RUNTIME to perform the same as opening the JAR file with the Release-less constructors. The results in Table 3 attempts to answer this question since rt.jar is not a multi-release jar file. This tells me that if one opens the JarFile with Release.RUNTIME, that there is a performance penalty of 2-6% on this very large jar file. Finally, the results in Table 4 tell me that processing a true multi-release jar file takes about 80% more time per entries() or stream() operation. I?ve looked at this in a profiler and there is no particular area that stands out to me, it?s just more complicated to process a multi-release jar file, as would be expected. > I think the javadoc will need to also need to make it clear whether entries with names starting with META-INF/versions/ are returned. Fixed. > > I see you've added @since 9 to the existing methods, I assume you didn't mean to do this. Fixed. > > At some point then we need to discuss how RUNTIME_VERSION is computed. Iris (via Mandy) has pushed jdk.Version to jdk9/dev but having it exported by java.base conflicts with the design principles in JEP 200. Moving it to another module means that code in java.base cannot use it and thus the JAR file can't use it. Left as is. From kubota.yuji at gmail.com Wed Feb 10 01:17:19 2016 From: kubota.yuji at gmail.com (KUBOTA Yuji) Date: Wed, 10 Feb 2016 10:17:19 +0900 Subject: Unexpected BindException in Endpoint.publish In-Reply-To: <565EF499.6040709@oracle.com> References: <565EF499.6040709@oracle.com> Message-ID: Hi Miroslav, Thank you for your sponsor! : https://bugs.openjdk.java.net/browse/JDK-8146086 Can I ask the schedule when does this fix backport to JDK8 ? Thanks, Yuji 2015-12-02 22:39 GMT+09:00 Miroslav Kos : > Hi Yuji, > thanks for the patch - it fixes the issue and looks ok to me. I'll integrate > it to standalone JAX-WS repo and it will be integrated into openjdk during > next syncup. > > Thanks > Miran > > > > > On 01/12/15 11:11, KUBOTA Yuji wrote: >> >> Hi Miroslav and all, >> >> Could you please review the below issue and patch? >> >> I got the advice by Alan at net-dev. So I want to ask you. >> http://mail.openjdk.java.net/pipermail/net-dev/2015-December/009361.html >> >> ---- >> I'm at the HackerGarten @ JavaOne15, and write a patch for OpenJDK >> community. This's second times from JavaOne14. :) >> >> We find an unexpected exception in JAX-WS, so I write a patch to fix it. >> We think that this issue may block the migration to JDK9 from JDK7. >> >> If we bind 0.0.0.0 ( using as wildcard ) to publish multiple as the >> following test code, JDK9 (and JDK8) returns "java.net.BindException: >> Address already in use.? as the below. But JDK7 does NOT return the >> exception. >> >> - Test code for reproduce >> --- >> import javax.jws.*; >> import javax.xml.ws.*; >> >> public class WSTest{ >> >> @WebService >> public static class Method1{ >> @WebMethod >> public String getMethod1Value(){ >> return "from Method1"; >> } >> } >> >> @WebService >> public static class Method2{ >> @WebMethod >> public String getMethod2Value(){ >> return "from Method2"; >> } >> } >> >> public static void main(String[] args) throws Exception{ >> Endpoint endPoint1 = Endpoint.publish("http://0.0.0.0:8081/method1", >> new >> Method1()); >> Endpoint endPoint2 = Endpoint.publish("http://0.0.0.0:8081/method2", >> new >> Method2()); >> >> System.out.println("Sleep 3 secs..."); >> >> Thread.sleep(3000); >> >> endPoint2.stop(); >> endPoint1.stop(); >> } >> >> } >> --- >> >> - StackTrace >> --- >> Exception in thread "main" >> com.sun.xml.internal.ws.server.ServerRtException: Server Runtime >> Error: java.net.BindException: Address already in use >> at >> com.sun.xml.internal.ws.transport.http.server.ServerMgr.createContext(ServerMgr.java:117) >> at >> com.sun.xml.internal.ws.transport.http.server.HttpEndpoint.publish(HttpEndpoint.java:64) >> at >> com.sun.xml.internal.ws.transport.http.server.EndpointImpl.publish(EndpointImpl.java:232) >> at >> com.sun.xml.internal.ws.spi.ProviderImpl.createAndPublishEndpoint(ProviderImpl.java:126) >> at javax.xml.ws.Endpoint.publish(Endpoint.java:240) >> at wstest.WSTest.main(WSTest.java:27) >> Caused by: java.net.BindException: Address already in use >> at sun.nio.ch.Net.bind0(Native Method) >> at sun.nio.ch.Net.bind(Net.java:432) >> at sun.nio.ch.Net.bind(Net.java:424) >> at >> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) >> at >> sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) >> at sun.net.httpserver.ServerImpl.(ServerImpl.java:102) >> at >> sun.net.httpserver.HttpServerImpl.(HttpServerImpl.java:50) >> at >> sun.net.httpserver.DefaultHttpServerProvider.createHttpServer(DefaultHttpServerProvider.java:35) >> at com.sun.net.httpserver.HttpServer.create(HttpServer.java:130) >> at >> com.sun.xml.internal.ws.transport.http.server.ServerMgr.createContext(ServerMgr.java:86) >> ... 5 more >> ----- >> >> To publishes the Endpoint, JAX-WS checks whether the HttpContext has >> been created by given address, then creates a HttpContext if do not >> exist. >> If we sets 0.0.0.0 as given address, JAX-WS checks by >> ServerSocket#getLocalSocketAddress() (server local address), so >> returns BindException when 0.0.0.0 has been blinded already. >> >> Why so? JAX_WS-941[1] fixes NPE in Endpoint.stop but do not think >> about above situation. And JAX_WS-941 does not back port to JDK7. >> >> So I write a patch which is based jdk9/dev/jaxws >> (changeset:637:2d84c6f4cbba) to fix the BindException with JAX_WS-941. >> >> Please review this patch :) >> >> [1]: https://java.net/jira/browse/JAX_WS-941 >> >> - Patch >> --- >> diff -r 2d84c6f4cbba >> >> src/java.xml.ws/share/classes/com/sun/xml/internal/ws/transport/http/server/ServerMgr.java >> --- >> a/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/transport/http/server/ServerMgr.java >> Thu Oct 22 08:47:47 2015 -0700 >> +++ >> b/src/java.xml.ws/share/classes/com/sun/xml/internal/ws/transport/http/server/ServerMgr.java >> Tue Oct 27 19:48:35 2015 +0900 >> @@ -38,6 +38,7 @@ >> import java.util.concurrent.ExecutorService; >> import java.util.concurrent.Executors; >> import java.util.logging.Logger; >> +import java.util.Optional; >> >> /** >> * Manages all the WebService HTTP servers created by JAXWS runtime. >> @@ -81,24 +82,38 @@ >> synchronized(servers) { >> state = servers.get(inetAddress); >> if (state == null) { >> - logger.fine("Creating new HTTP Server at >> "+inetAddress); >> - // Creates server with default socket backlog >> - server = HttpServer.create(inetAddress, 0); >> - server.setExecutor(Executors.newCachedThreadPool()); >> - String path = url.toURI().getPath(); >> - logger.fine("Creating HTTP Context at = "+path); >> - HttpContext context = server.createContext(path); >> - server.start(); >> + final int finalPortNum = port; >> + Optional stateOpt = >> + servers.values() >> + .stream() >> + .filter(s -> s.getServer() >> + .getAddress() >> + .getPort() == >> finalPortNum) >> + .findAny(); >> >> - // we have to get actual inetAddress from server, >> which can differ from the original in some cases. >> - // e.g. A port number of zero will let the system >> pick up an ephemeral port in a bind operation, >> - // or IP: 0.0.0.0 - which is used to monitor >> network traffic from any valid IP address >> - inetAddress = server.getAddress(); >> + if (inetAddress.getAddress().isAnyLocalAddress() && >> + stateOpt.isPresent()) { >> + state = stateOpt.get(); >> + } else { >> + logger.fine("Creating new HTTP Server at >> "+inetAddress); >> + // Creates server with default socket backlog >> + server = HttpServer.create(inetAddress, 0); >> + >> server.setExecutor(Executors.newCachedThreadPool()); >> + String path = url.toURI().getPath(); >> + logger.fine("Creating HTTP Context at = "+path); >> + HttpContext context = server.createContext(path); >> + server.start(); >> >> - logger.fine("HTTP server started = "+inetAddress); >> - state = new ServerState(server, path); >> - servers.put(inetAddress, state); >> - return context; >> + // we have to get actual inetAddress from >> server, which can differ from the original in some cases. >> + // e.g. A port number of zero will let the >> system pick up an ephemeral port in a bind operation, >> + // or IP: 0.0.0.0 - which is used to monitor >> network traffic from any valid IP address >> + inetAddress = server.getAddress(); >> + >> + logger.fine("HTTP server started = >> "+inetAddress); >> + state = new ServerState(server, path); >> + servers.put(inetAddress, state); >> + return context; >> + } >> } >> } >> server = state.getServer(); >> --- >> >> Thanks, >> Yuji > > From mikael.vidstedt at oracle.com Wed Feb 10 03:42:03 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Tue, 9 Feb 2016 19:42:03 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56B7DDD8.2050701@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> <56B5207B.8010107@oracle.com> <56B7DDD8.2050701@oracle.com> Message-ID: <56BAB18B.3070707@oracle.com> Can I please get a quick review of these updated webrevs: hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/hotspot/webrev/ jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/jdk/webrev/ incremental webrevs: hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/hotspot/webrev/ jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/jdk/webrev/ Changes: * Added asserts in copy.cpp/conjoint_swap * Correctness: Moved offset sign checks to only be performed if corresponding base object is null, and added corresponding tests I'm about to make additional changes in this same area, so unless this last change is horribly broken I'm planning on pushing this and doing any additional cleanup in the upcoming change(s). Cheers, Mikael On 2016-02-07 16:14, David Holmes wrote: > On 6/02/2016 8:21 AM, Mikael Vidstedt wrote: >> >> I fully agree that moving the arguments checking up to Java makes more >> sense, and I've prepared new webrevs which do exactly that, including >> changes to address the other feedback from David, John and others: > > Shouldn't the lowest-level do_conjoint_swap routines at least check > preconditions with asserts to catch the cases where the calling Java > code has failed to do the right thing? The other Copy methods seem to > do this. > > David > ----- > >> hotspot: >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/hotspot/webrev/ >> >> >> jdk: >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/jdk/webrev/ >> >> Incremental webrevs for your convenience: >> >> hotspot: >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/hotspot/webrev/ >> >> >> jdk: >> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/jdk/webrev/ >> >> >> >> >> I have done some benchmarking of this code and for large copies (16MB+) >> this outperforms the old Bits.c implementation by *30-100%* depending on >> platform and exact element sizes! For smaller copies the additional >> checks which are now performed hurt performance on client VMs (80-90% of >> old impl), but with the server VMs I see performance on par with, or in >> most cases 5-10% better than the old implementation. There's a >> potentially statistically significant regression of ~3-4% for >> elemSize=2, but for now I'm going to declare success. There's certainly >> room for further improvements here, but this should at least do for >> addressing the original problem. >> >> >> I filed https://bugs.openjdk.java.net/browse/JDK-8149159 for moving the >> checks for Unsafe.copyMemory to Java, and will work on that next. I also >> filed https://bugs.openjdk.java.net/browse/JDK-8149162 to cover the >> potential renaming of the Bits methods to have more informative names. >> Finally, I filed https://bugs.openjdk.java.net/browse/JDK-8149163 to >> look at improving the behavior of Unsafe.addressSize(), after having >> spent too much time trying to understand why the performance of the new >> U.copySwapMemory Java checks wasn't quite living up to my expectations >> (spoiler alert: Unsafe.addressSize() is not intrinsified, so will always >> result in a call into the VM/unsafe.cpp). >> >> >> Finally, I - too - would like to see the copy-swap logic moved into >> Java, and as I mentioned I played around with that first before I >> decided to do the native implementation to address the immediate >> problem. Looking forward to what you find Paul! >> >> Cheers, >> Mikael >> >> On 2016-02-05 05:00, Paul Sandoz wrote: >>> Hi, >>> >>> Nice use of C++ templates :-) >>> >>> Overall looks good. >>> >>> I too would prefer if we could move the argument checking out, perhaps >>> even to the point of requiring callers do that rather than providing >>> another method, for example for Buffer i think the arguments are known >>> to be valid? I think in either case it is important to improve the >>> documentation on the method stating the constraints on arguments, >>> atomicity guarantees etc. >>> >>> I have a hunch that for the particular case of copying-with-swap for >>> buffers i could get this to work work efficiently using Unsafe (three >>> separate methods for each unit type of 2, 4 and 8 bytes), since IIUC >>> the range is bounded to be less than Integer.MAX_VALUE so an int loop >>> rather than a long loop can be used and therefore safe points checks >>> will not be placed within the loop. >>> >>> However, i think what you have done is more generally applicable and >>> could be made intrinsic. It would be a nice at some future point if it >>> could be made a pure Java implementation and intrinsified where >>> appropriate. >>> >>> ? >>> >>> John, regarding array mismatch there were issues with the efficiency >>> of the unrolled loops with Unsafe access. (Since the loops were int >>> bases there were no issues with safe point checks.) Roland recently >>> fixed that so now code is generated that is competitive with direct >>> array accesses. We drop into the stub intrinsic and leverage 128bits >>> or 256bits where supported. Interestingly it seems the unrolled loop >>> using Unsafe is now slightly faster than the stub using 128bit >>> registers. I don?t know if that is due to unluckly alignment, and/or >>> the stub needs to do some manual unrolling. In terms of code-cache >>> efficiency the intrinsic is better. >>> >>> Paul. >>> >>> >>> >>> >>> >>>> On 4 Feb 2016, at 06:27, John Rose wrote: >>>> >>>> On Feb 2, 2016, at 11:25 AM, Mikael Vidstedt >>>> wrote: >>>>> Please review this change which introduces a Copy::conjoint_swap and >>>>> an Unsafe.copySwapMemory method to call it from Java, along with the >>>>> necessary changes to have java.nio.Bits call it instead of the >>>>> Bits.c code. >>>>> >>>>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ >>>>> >>>>> >>>> This is very good. >>>> >>>> I have some nit-picks: >>>> >>>> These days, when we introduce a new intrinsic (@HSIntrCand), >>>> we write the argument checking code separately in a non-intrinsic >>>> bytecode method. In this case, we don't (yet) have an intrinsic >>>> binding for U.copy*, but we might in the future. (C intrinsifies >>>> memcpy, which is a precedent.) In any case, I would prefer >>>> if we could structure the argument checking code in a similar >>>> way, with Unsafe.java containing both copySwapMemory >>>> and a private copySwapMemory0. Then we can JIT-optimize >>>> the safety checks. >>>> >>>> You might as well extend the same treatment to the pre-existing >>>> copyMemory call. The most important check (and the only one >>>> in U.copyMemory) is to ensure that the size_t operand has not >>>> wrapped around from a Java negative value to a crazy-large >>>> size_t value. That's the low-hanging fruit. Checking the pointers >>>> (for null or oob) is more problematic, of course. Checking >>>> consistency >>>> around elemSize is cheap and easy, so I agree that the U.copySM >>>> should do that work also. Basically, Unsafe can do very basic >>>> checks if there is a tricky user model to enforce, but it mustn't >>>> "sign up" to guard the user against all errors. >>>> >>>> Rule of thumb: Unsafe calls don't throw NPEs, they just SEGV. >>>> And the rare bit that *does* throw (IAE usually) should be placed >>>> into Unsafe.java, not unsafe.cpp. (The best-practice rule for putting >>>> argument checking code outside of the intrinsic is a newer one, >>>> so Unsafe code might not always do this.) >>>> >>>> The comment "Generalizing it would be reasonable, but requires >>>> card marking" is bogus, since we never byte-swap managed pointers. >>>> >>>> The test logic will flow a little smoother if your GenericPointer guy, >>>> the onHeap version, stores the appropriate array base offset in his >>>> offset field. >>>> You won't have to mention p.isOnHeap nearly so much, and the code will >>>> set a slightly better example. >>>> >>>> The VM_ENTRY_BASE_FROM_LEAF macro is really cool. >>>> >>>> The C++ template code is cool also. It reminds me of the kind >>>> of work Gosling's "Ace" processor could do, but now it's mainstreamed >>>> for all to use in C++. We're going to get some of that goodness >>>> in Project Valhalla with specialization logic. >>>> >>>> I find it amazing that the right way to code this in C is to >>>> use memcpy for unaligned accesses and byte peek/poke >>>> into registers for byte-swapping operators. I'm glad we >>>> can write this code *once* for the JVM and JDK. >>>> >>>> Possible future work: If we can get a better handle on >>>> writing vectorizable loops from Java, including Unsafe-based >>>> ones, we can move some of the C code back up to Java. >>>> Perhaps U.copy* calls for very short lengths deserved to >>>> be broken out into small loops of U.get/put* (with alignment). >>>> I think you experimented with this, and there were problems >>>> with the JIT putting fail-safe memory barriers between >>>> U.get/put* calls. Paul's work on Array.mismatch ran into >>>> similar issues, with the right answer being to write manual >>>> vector code in assembly. >>>> >>>> Anyway, you can count me as a reviewer. >>>> >>>> Thanks, >>>> >>>> ? John >> From david.holmes at oracle.com Wed Feb 10 04:03:44 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Feb 2016 14:03:44 +1000 Subject: New Collection framework? In-Reply-To: <2485208.rZGvsojIaB@linux-vgv1> References: <2485208.rZGvsojIaB@linux-vgv1> Message-ID: <56BAB6A0.7070402@oracle.com> On 9/02/2016 5:29 AM, Thomas Kamps wrote: > Hi everyone, > > Did you ever think about a new Collection framework for Java? A new framework would need to be significantly better than the existing one to ever consider any kind of replacement. It would need to be compatible in such a way that users can transition over time, and that time would be quite lengthy - at least 2 major releases, more likely 3 IMHO. The Java 8 Streams provide a new way for working with collections that can avoid problems/limitations in the base API's. > My main complains about the java.util Collections are: > > * No co/contra variance for assigning. Which forbids assigments like > List foo = new List(); > Currently it can be bypassed when using wildcards, but then you cannot add > items to foo. Even more problematic are interfaces with some kind of > List> as parameter That's a language issue not a j.u.Collection issue per-se. But of course if generics had been around at the beginning there would likely have been numerous API differences in the Collection framework. > * The main Collection interface allowes too much. If you get a Collection and > you should not modify it, your programm will either behave stragly with a hard > to detect bug, or crash with some kind of UnsupportedOperationException, when > you do it. > I have my problems with UnsupprtedOperationException, because when i get a > Collection, i have to read the documentation (if existing) to know what i am > allowed to do, or run unexpectedly into an exception. Would it be great to > hava an "readonly" interface, which simply does not have and modifing methods? I don't understand your problem. If you write a method that takes a collection then your method has a specification as to what it does that involves the collection that is passed in. If the passed in collection doesn't in fact support that then the UnsupportedOperationException propagates back to the caller that passed you an inappropriate collection - and that's exactly as it should be. If you take a collection and then try to do something to it that you didn't specify you were going to do, and get UnsupportedOperationException then that is a bug in your code - if you let it propagate then your caller gets it and the user of your code submits a bug report. In case you have not seen it there is a document that discusses the design of the Collections framework and why the existing approach was taken: https://docs.oracle.com/javase/8/docs/technotes/guides/collections/designfaq.html > The other possible reason could be the Valhalla project, which tries to > introduce value types, to enable Collections with primitive type parameters. > There is (or was?) a discussion how to handle the remove methods on List: > http://mail.openjdk.java.net/pipermail/valhalla-spec-experts/2015-December/000028.html > There is a suggestion for partial methods to solve the problem, but i think > partial methods makes the language even more complicated to compile and > more difficult to understand. Yes value types are increasing the strain on the API, but I have confidence that the Valhalla folk will find acceptable ways to resolve that. > All theese problems could be resolved by just letting the java.util Collection > Framework as it is. And make a new one. > I have created a proposal for it, which can be easily (modulo class names) > used together with the java.util Collections. This is achieved by having > wrapper Collections and converter in both directions. > https://github.com/Blahord/BetterCollections I have not looked at this - sorry, I simply do not have time right now. Cheers, David ------ > Note: This is the first release with the target to have a clean and small as > possible API. > Since it makes heavily use of wildcards, it is some kind of bureaucratic. > This could be reduced, by > > * Adding a shortcur for wildcards like "?Type" or "? extends Type" and "!Type" > for "? super Type" or > * Allowing co and contra variance for type parameters. > From david.holmes at oracle.com Wed Feb 10 04:10:39 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Feb 2016 14:10:39 +1000 Subject: OpenJDK bug question (JDK-6519463) In-Reply-To: References: Message-ID: <56BAB83F.3010109@oracle.com> On 10/02/2016 4:51 AM, David Hoffer wrote: > We have a production environment where we are getting errors similar to > what was reported in JDK-6519463. I would like to test our code with the > test file (buggedfile.bin) used in this bug report but the file is posted > on an ftp server that is not publicly available. Can this file be attached > to the bug report or just email me a copy? Will the test file in: https://bugs.openjdk.java.net/browse/JDK-4040920 suffice, as it was reported to be the underlying cause. David > Thanks, > -Dave > From dhoffer6 at gmail.com Wed Feb 10 04:20:49 2016 From: dhoffer6 at gmail.com (David Hoffer) Date: Tue, 9 Feb 2016 21:20:49 -0700 Subject: OpenJDK bug question (JDK-6519463) In-Reply-To: <56BAB83F.3010109@oracle.com> References: <56BAB83F.3010109@oracle.com> Message-ID: That issue has the same problem, it references files that are not in the bug report so it won't help, really need all files to be attached to the bug report. That being said we think we have tracked our problem down to a file transfer step after compression so I don't think our issue pertains to JDK/compression but just in case we are wrong I'd like to see the test files attached to the bug report...in case we need them. Thanks, -Dave On Tue, Feb 9, 2016 at 9:10 PM, David Holmes wrote: > On 10/02/2016 4:51 AM, David Hoffer wrote: > >> We have a production environment where we are getting errors similar to >> what was reported in JDK-6519463. I would like to test our code with the >> test file (buggedfile.bin) used in this bug report but the file is posted >> on an ftp server that is not publicly available. Can this file be >> attached >> to the bug report or just email me a copy? >> > > Will the test file in: > > https://bugs.openjdk.java.net/browse/JDK-4040920 > > suffice, as it was reported to be the underlying cause. > > David > > Thanks, >> -Dave >> >> From xueming.shen at oracle.com Wed Feb 10 04:38:16 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 9 Feb 2016 20:38:16 -0800 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: References: Message-ID: <56BABEB8.2050505@oracle.com> Hi Chris, I think basically you are asking a String constructor that takes a ByteBuffer. StringCoding then can take advantage of the current CompactString design to optimize the decoding operation by just a single byte[]/vectorized memory copy from the ByteBuffer to the String's internal byte[], WHEN the charset is 8859-1. String(ByteBuffer src, String charset); Further we will need a "buffer gathering" style constructor String(ByteBuffer[] srcs, String charset); (or more generally, String(ByteBuffer[] srcs, int off, int len, String charset) to create a String object from a sequence of ByteBuffers, if it's really desired. And then I would also assume it will also be desired to extend the current CharsetDecoder/Encoder class as well to add a pair of the "gathering" style coding methods CharBuffer CharsetDecoder.decode(ByteBuffer... ins); ByteBuffer CharsetEncoder.encode(CharBuffer... ins); Though the implementation might have to deal with the tricky "splitting byte/char" issue, in which part of the "byte/char sequence" is in the previous buffer and the continuing byte/chars are in the next following buffer ... -Sherman On 2/9/16 7:20 AM, Chris Vest wrote: > Hi, > > Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string data in Latin1, which is one of the two special encodings for compact strings, we?d ideally like to produce the String object with just the two necessary object allocations and a single, vectorised memory copy. > > Our use case is that we are a database and we do our own file paging, effectively having file data in a large set of DirectByteBuffers. We have string data in our files in a number of different encodings, a popular one being Latin1. Occasionally these String values span multiple buffers. We often need to expose this data as String objects, in which case decoding the bytes and turning them into a String is often very performance sensitive - to the point of being one of our top bottlenecks for the given queries. Part of the story is that in the case of Latin1, I?ll know up front exactly how many bytes my string data takes up, though I might not know how many buffers are going to be involved. > > As far as I can tell, this is currently not possible using public APIs. Using private APIs it may be possible, but will be relying on the JIT for vectorising the memory copying. > > From an API standpoint, CharsetDecoder is close to home, but is not quite there. It?s stateful and not thread-safe, so I either have to allocate new ones every time or cache them in thread-locals. I?m also required to allocate the receiving CharBuffer. Since I may need to decode from multiple buffers, I realise that I might not be able to get away from allocating at least one extra object to keep track of intermediate decoding state. The CharsetDecoder does not have a method where I can specify the offset and length for the desired part of the ByteBuffer I want to decode, which forces be to allocate views instead. > > The CharBuffers are allocated with a length up front, which is nice, but I can?t restrict its encoding so it has to allocate a char array instead of the byte array that I really want. Even if it did allocate a byte array, the CharBuffer is mutable, which would force String do a defensive copy anyway. > > One way I imagine this could be solved would be with a less dynamic kind of decoder, where the target length is given upfront to the decoder. Buffers are then consumed one by one, and a terminal method performs finishing sanity checks (did we get all the bytes we were promised?) and returns the result. > > StringDecoder decoder = Charset.forName(?latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); > String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, len2).done(); > > This will in principle allow the string decoding to be 2 small allocations, an array allocation without zeroing, and a sequence of potentially vectorised memcpys. I don?t see any potentially troubling interactions with fused Strings either, since all the knowledge (except for the string data itself) needed to allocate the String objects are available from the get-go. > > What do you guys think? > > Btw, Richard Warburton has already done some work in this area, and made a patch that adds a constructor to String that takes a buffer, offset, length, and charset. This work now at least needs rebasing: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ > It doesn?t solve the case where multiple buffers are used to build the string, but does remove the need for a separate intermediate state-holding object when a single buffer is enough. It?d be a nice addition if possible, but I (for one) can tolerate a small object allocation otherwise. > > Cheers, > Chris > From david.holmes at oracle.com Wed Feb 10 04:38:39 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Feb 2016 14:38:39 +1000 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56BAB18B.3070707@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> <56B5207B.8010107@oracle.com> <56B7DDD8.2050701@oracle.com> <56BAB18B.3070707@oracle.com> Message-ID: <56BABECF.4050505@oracle.com> On 10/02/2016 1:42 PM, Mikael Vidstedt wrote: > > Can I please get a quick review of these updated webrevs: In terms of the incremental changes this looks fine. If you consider it all reviewed then nothing in the increments should change that. But looking at the JDK code I have some follow up suggestions for copySwapMemory: - document all parameters with @param and describe constraints on values/relationships - specify @throws for all IllegalArgumentException and NullPointerException conditions - add a descriptive error message when throwing IllegalArgumentException - not sure NullPointerException is correct, rather than IllegalArgumentException. for null base with zero offset cases Thanks, David > hotspot: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/hotspot/webrev/ > > jdk: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/jdk/webrev/ > > incremental webrevs: > > hotspot: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/hotspot/webrev/ > > jdk: > http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/jdk/webrev/ > > > Changes: > > * Added asserts in copy.cpp/conjoint_swap > * Correctness: Moved offset sign checks to only be performed if > corresponding base object is null, and added corresponding tests > > I'm about to make additional changes in this same area, so unless this > last change is horribly broken I'm planning on pushing this and doing > any additional cleanup in the upcoming change(s). > > Cheers, > Mikael > > > On 2016-02-07 16:14, David Holmes wrote: >> On 6/02/2016 8:21 AM, Mikael Vidstedt wrote: >>> >>> I fully agree that moving the arguments checking up to Java makes more >>> sense, and I've prepared new webrevs which do exactly that, including >>> changes to address the other feedback from David, John and others: >> >> Shouldn't the lowest-level do_conjoint_swap routines at least check >> preconditions with asserts to catch the cases where the calling Java >> code has failed to do the right thing? The other Copy methods seem to >> do this. >> >> David >> ----- >> >>> hotspot: >>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/hotspot/webrev/ >>> >>> >>> jdk: >>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04/jdk/webrev/ >>> >>> Incremental webrevs for your convenience: >>> >>> hotspot: >>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/hotspot/webrev/ >>> >>> >>> jdk: >>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.04.incr/jdk/webrev/ >>> >>> >>> >>> >>> I have done some benchmarking of this code and for large copies (16MB+) >>> this outperforms the old Bits.c implementation by *30-100%* depending on >>> platform and exact element sizes! For smaller copies the additional >>> checks which are now performed hurt performance on client VMs (80-90% of >>> old impl), but with the server VMs I see performance on par with, or in >>> most cases 5-10% better than the old implementation. There's a >>> potentially statistically significant regression of ~3-4% for >>> elemSize=2, but for now I'm going to declare success. There's certainly >>> room for further improvements here, but this should at least do for >>> addressing the original problem. >>> >>> >>> I filed https://bugs.openjdk.java.net/browse/JDK-8149159 for moving the >>> checks for Unsafe.copyMemory to Java, and will work on that next. I also >>> filed https://bugs.openjdk.java.net/browse/JDK-8149162 to cover the >>> potential renaming of the Bits methods to have more informative names. >>> Finally, I filed https://bugs.openjdk.java.net/browse/JDK-8149163 to >>> look at improving the behavior of Unsafe.addressSize(), after having >>> spent too much time trying to understand why the performance of the new >>> U.copySwapMemory Java checks wasn't quite living up to my expectations >>> (spoiler alert: Unsafe.addressSize() is not intrinsified, so will always >>> result in a call into the VM/unsafe.cpp). >>> >>> >>> Finally, I - too - would like to see the copy-swap logic moved into >>> Java, and as I mentioned I played around with that first before I >>> decided to do the native implementation to address the immediate >>> problem. Looking forward to what you find Paul! >>> >>> Cheers, >>> Mikael >>> >>> On 2016-02-05 05:00, Paul Sandoz wrote: >>>> Hi, >>>> >>>> Nice use of C++ templates :-) >>>> >>>> Overall looks good. >>>> >>>> I too would prefer if we could move the argument checking out, perhaps >>>> even to the point of requiring callers do that rather than providing >>>> another method, for example for Buffer i think the arguments are known >>>> to be valid? I think in either case it is important to improve the >>>> documentation on the method stating the constraints on arguments, >>>> atomicity guarantees etc. >>>> >>>> I have a hunch that for the particular case of copying-with-swap for >>>> buffers i could get this to work work efficiently using Unsafe (three >>>> separate methods for each unit type of 2, 4 and 8 bytes), since IIUC >>>> the range is bounded to be less than Integer.MAX_VALUE so an int loop >>>> rather than a long loop can be used and therefore safe points checks >>>> will not be placed within the loop. >>>> >>>> However, i think what you have done is more generally applicable and >>>> could be made intrinsic. It would be a nice at some future point if it >>>> could be made a pure Java implementation and intrinsified where >>>> appropriate. >>>> >>>> ? >>>> >>>> John, regarding array mismatch there were issues with the efficiency >>>> of the unrolled loops with Unsafe access. (Since the loops were int >>>> bases there were no issues with safe point checks.) Roland recently >>>> fixed that so now code is generated that is competitive with direct >>>> array accesses. We drop into the stub intrinsic and leverage 128bits >>>> or 256bits where supported. Interestingly it seems the unrolled loop >>>> using Unsafe is now slightly faster than the stub using 128bit >>>> registers. I don?t know if that is due to unluckly alignment, and/or >>>> the stub needs to do some manual unrolling. In terms of code-cache >>>> efficiency the intrinsic is better. >>>> >>>> Paul. >>>> >>>> >>>> >>>> >>>> >>>>> On 4 Feb 2016, at 06:27, John Rose wrote: >>>>> >>>>> On Feb 2, 2016, at 11:25 AM, Mikael Vidstedt >>>>> wrote: >>>>>> Please review this change which introduces a Copy::conjoint_swap and >>>>>> an Unsafe.copySwapMemory method to call it from Java, along with the >>>>>> necessary changes to have java.nio.Bits call it instead of the >>>>>> Bits.c code. >>>>>> >>>>>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/hotspot/webrev/ >>>>>> >>>>>> >>>>>> http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.03/jdk/webrev/ >>>>>> >>>>>> >>>>> This is very good. >>>>> >>>>> I have some nit-picks: >>>>> >>>>> These days, when we introduce a new intrinsic (@HSIntrCand), >>>>> we write the argument checking code separately in a non-intrinsic >>>>> bytecode method. In this case, we don't (yet) have an intrinsic >>>>> binding for U.copy*, but we might in the future. (C intrinsifies >>>>> memcpy, which is a precedent.) In any case, I would prefer >>>>> if we could structure the argument checking code in a similar >>>>> way, with Unsafe.java containing both copySwapMemory >>>>> and a private copySwapMemory0. Then we can JIT-optimize >>>>> the safety checks. >>>>> >>>>> You might as well extend the same treatment to the pre-existing >>>>> copyMemory call. The most important check (and the only one >>>>> in U.copyMemory) is to ensure that the size_t operand has not >>>>> wrapped around from a Java negative value to a crazy-large >>>>> size_t value. That's the low-hanging fruit. Checking the pointers >>>>> (for null or oob) is more problematic, of course. Checking >>>>> consistency >>>>> around elemSize is cheap and easy, so I agree that the U.copySM >>>>> should do that work also. Basically, Unsafe can do very basic >>>>> checks if there is a tricky user model to enforce, but it mustn't >>>>> "sign up" to guard the user against all errors. >>>>> >>>>> Rule of thumb: Unsafe calls don't throw NPEs, they just SEGV. >>>>> And the rare bit that *does* throw (IAE usually) should be placed >>>>> into Unsafe.java, not unsafe.cpp. (The best-practice rule for putting >>>>> argument checking code outside of the intrinsic is a newer one, >>>>> so Unsafe code might not always do this.) >>>>> >>>>> The comment "Generalizing it would be reasonable, but requires >>>>> card marking" is bogus, since we never byte-swap managed pointers. >>>>> >>>>> The test logic will flow a little smoother if your GenericPointer guy, >>>>> the onHeap version, stores the appropriate array base offset in his >>>>> offset field. >>>>> You won't have to mention p.isOnHeap nearly so much, and the code will >>>>> set a slightly better example. >>>>> >>>>> The VM_ENTRY_BASE_FROM_LEAF macro is really cool. >>>>> >>>>> The C++ template code is cool also. It reminds me of the kind >>>>> of work Gosling's "Ace" processor could do, but now it's mainstreamed >>>>> for all to use in C++. We're going to get some of that goodness >>>>> in Project Valhalla with specialization logic. >>>>> >>>>> I find it amazing that the right way to code this in C is to >>>>> use memcpy for unaligned accesses and byte peek/poke >>>>> into registers for byte-swapping operators. I'm glad we >>>>> can write this code *once* for the JVM and JDK. >>>>> >>>>> Possible future work: If we can get a better handle on >>>>> writing vectorizable loops from Java, including Unsafe-based >>>>> ones, we can move some of the C code back up to Java. >>>>> Perhaps U.copy* calls for very short lengths deserved to >>>>> be broken out into small loops of U.get/put* (with alignment). >>>>> I think you experimented with this, and there were problems >>>>> with the JIT putting fail-safe memory barriers between >>>>> U.get/put* calls. Paul's work on Array.mismatch ran into >>>>> similar issues, with the right answer being to write manual >>>>> vector code in assembly. >>>>> >>>>> Anyway, you can count me as a reviewer. >>>>> >>>>> Thanks, >>>>> >>>>> ? John >>> > From david.holmes at oracle.com Wed Feb 10 04:56:26 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 10 Feb 2016 14:56:26 +1000 Subject: OpenJDK bug question (JDK-6519463) In-Reply-To: References: <56BAB83F.3010109@oracle.com> Message-ID: <56BAC2FA.6040509@oracle.com> On 10/02/2016 2:20 PM, David Hoffer wrote: > That issue has the same problem, it references files that are not in the > bug report so it won't help, really need all files to be attached to the > bug report. Is the attachment a.zip.Z not visible to you on the bug report? David > That being said we think we have tracked our problem down > to a file transfer step after compression so I don't think our issue > pertains to JDK/compression but just in case we are wrong I'd like to > see the test files attached to the bug report...in case we need them. > > Thanks, > -Dave > > On Tue, Feb 9, 2016 at 9:10 PM, David Holmes > wrote: > > On 10/02/2016 4:51 AM, David Hoffer wrote: > > We have a production environment where we are getting errors > similar to > what was reported in JDK-6519463. I would like to test our code > with the > test file (buggedfile.bin) used in this bug report but the file > is posted > on an ftp server that is not publicly available. Can this file > be attached > to the bug report or just email me a copy? > > > Will the test file in: > > https://bugs.openjdk.java.net/browse/JDK-4040920 > > suffice, as it was reported to be the underlying cause. > > David > > Thanks, > -Dave > > From shilpi.rastogi at oracle.com Wed Feb 10 07:00:19 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Wed, 10 Feb 2016 12:30:19 +0530 Subject: RFR 8138884:MethodHandles.Lookup.findVirtual() Javadoc fails to consider private interface methods In-Reply-To: <56A1F4AE.4050201@oracle.com> References: <569F80B0.3040103@oracle.com> <56A1F4AE.4050201@oracle.com> Message-ID: <56BAE003.7000602@oracle.com> Hi All, Please review my updated patch http://cr.openjdk.java.net/~srastogi/8138884/webrev.01/ Same is not applicable for "Lookup.unreflect()". Thanks, Shilpi > > Hi All, > > Please review my fix related to java doc. > > MethodHandles.Lookup.findVirtual() Javadoc fails to consider private > methods of an interface. > > MethodHandles.Lookup.findVirtual() throws IllegalAccessException if > the method MH is being created for is private and declared in an > interface. > > Bug link- https://bugs.openjdk.java.net/browse/JDK-8138884 > > Webrev link- http://cr.openjdk.java.net/~srastogi/8138884/webrev.00/ > > > > Thanks, > Shilpi > > From shilpi.rastogi at oracle.com Wed Feb 10 07:01:16 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Wed, 10 Feb 2016 12:31:16 +0530 Subject: RFR: 8071368 Use more concrete types for NamedFunction constants in the code In-Reply-To: <56B895B9.40401@oracle.com> References: <56B34F40.1010800@oracle.com> <6804C2B3-DE2F-4DCF-B70D-6F3BC846B403@oracle.com> <56B4C467.7040300@oracle.com> <56B895B9.40401@oracle.com> Message-ID: <56BAE03C.2070406@oracle.com> Gentle Reminder! > > On 2/5/2016 9:18 PM, Vladimir Ivanov wrote: >> Proposed fix looks good. >> >> Quoting John: "The use of erased types (any ref => Object) in the MH >> runtime is an artifact of bootstrapping difficulties, early in the >> project. I hope it is not necessary any more." >> >> Best regards, >> Vladimir Ivanov >> >> On 2/5/16 4:29 PM, Paul Sandoz wrote: >>> Hi Shilpi, >>> >>> In principle this change is ok, however these methods are kind of >>> special because they are called by code generated from LambdaForms >>> with erased types. >>> >>> IIUC the casts can serve as belts-and-braces checks just in case >>> that generated code passes something that is not of the correct >>> type. By removing the casts it?s possible to erroneously pass in >>> something of another type that the runtime treats erroneously as if >>> a MethodHandle or MethodType, and then strange things happen. The >>> casts will ordinary be removed by the runtime compiler. By removing >>> them from the Java code it might slightly improve C2 compilation time. >>> >>> Approval from Vladimir/John would be good. >>> >>> Paul. >>> >>>> On 4 Feb 2016, at 13:16, shilpi rastogi >>>> wrote: >>>> >>>> Hi All, >>>> >>>> Please review the following- >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8071368 >>>> http://cr.openjdk.java.net/~srastogi/8071368/webrev.02/ >>>> >>>> Thanks, >>>> Shilpi >>> > From ramanand.patil at oracle.com Wed Feb 10 07:08:08 2016 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Tue, 9 Feb 2016 23:08:08 -0800 (PST) Subject: [PING] RFR: JDK-8135259: InetAddress.getAllByName only reports "unknown error" instead of actual cause In-Reply-To: <48fd26eb-350e-4166-8fe7-6883f8562cd5@default> References: <48fd26eb-350e-4166-8fe7-6883f8562cd5@default> Message-ID: <3c428763-825c-40c9-b4d5-bffcd766baea@default> Hi all, Please help me in getting this review done. Regards, Ramanand. -----Original Message----- From: Ramanand Patil Sent: Friday, February 05, 2016 10:46 PM To: core-libs-dev at openjdk.java.net Subject: RFR: JDK-8135259: InetAddress.getAllByName only reports "unknown error" instead of actual cause Hi all Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8135259 Bug Description: Attempts to resolve a host unknown to the DNS server cause an UnknownHostException stating "unknown error" instead of "Name or service not known". Webrev: http://cr.openjdk.java.net/~rpatil/8135259/webrev.00/ Fix: Using a function call directly "gai_strerror(gai_error)" instead of using the function pointer which was declared but not initialized. Apart from this I have removed few other unused variables and function pointers. Brief description of how this bug was introduced is added to the bug comment. Regards, Ramanand. From paul.sandoz at oracle.com Wed Feb 10 08:03:43 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 09:03:43 +0100 Subject: RFR: 8071368 Use more concrete types for NamedFunction constants in the code In-Reply-To: <56BAE03C.2070406@oracle.com> References: <56B34F40.1010800@oracle.com> <6804C2B3-DE2F-4DCF-B70D-6F3BC846B403@oracle.com> <56B4C467.7040300@oracle.com> <56B895B9.40401@oracle.com> <56BAE03C.2070406@oracle.com> Message-ID: <86D5EA70-E7ED-42C2-AEC2-085733A69758@oracle.com> Hi Shilpi, I think you can consider this reviewed. It got the ok from Vladimir, with reference to John?s feedback in the bug. Pau. > On 10 Feb 2016, at 08:01, shilpi rastogi wrote: > > > Gentle Reminder! >> >> On 2/5/2016 9:18 PM, Vladimir Ivanov wrote: >>> Proposed fix looks good. >>> >>> Quoting John: "The use of erased types (any ref => Object) in the MH runtime is an artifact of bootstrapping difficulties, early in the project. I hope it is not necessary any more." >>> >>> Best regards, >>> Vladimir Ivanov >>> From paul.sandoz at oracle.com Wed Feb 10 08:06:21 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 09:06:21 +0100 Subject: RFR 8138884:MethodHandles.Lookup.findVirtual() Javadoc fails to consider private interface methods In-Reply-To: <56BAE003.7000602@oracle.com> References: <569F80B0.3040103@oracle.com> <56A1F4AE.4050201@oracle.com> <56BAE003.7000602@oracle.com> Message-ID: <3BA12D86-8A4B-438E-80F0-15F3FDBEE4B0@oracle.com> > On 10 Feb 2016, at 08:00, shilpi rastogi wrote: > > Hi All, > > Please review my updated patch > http://cr.openjdk.java.net/~srastogi/8138884/webrev.01/ > Just add a ?,? after the new statement .. {@code static}, No need for another review. > Same is not applicable for "Lookup.unreflect()". > Ok. Thanks for checking. Paul. From paul.sandoz at oracle.com Wed Feb 10 08:54:17 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 09:54:17 +0100 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <56BABEB8.2050505@oracle.com> References: <56BABEB8.2050505@oracle.com> Message-ID: <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> Hi, A more functional approach would be to compose a sequence buffers into one view, perhaps read-only. Then there would be no need to accept arrays of buffers. That should work well for bulk operations. That?s a non-trivial but not very difficult amount of work, and possibly simplified if restricted to read-only views. Thus i think we should focus Richard?s work with: String(ByteBuffer src, String charset) and perhaps a sub-range variant, if perturbing the position/limit of an existing buffer and/or slicing is too problematic. ? Zeroing memory and possibly avoiding it can be tricky. Any such optimisations have to be carefully performed otherwise uninitialised regions might leak and be accessed, nefariously or otherwise. I imagine it?s easier to contain/control within a constructor than say a builder. Paul. > On 10 Feb 2016, at 05:38, Xueming Shen wrote: > > Hi Chris, > > I think basically you are asking a String constructor that takes a ByteBuffer. StringCoding > then can take advantage of the current CompactString design to optimize the decoding > operation by just a single byte[]/vectorized memory copy from the ByteBuffer to the String's > internal byte[], WHEN the charset is 8859-1. > > String(ByteBuffer src, String charset); > > Further we will need a "buffer gathering" style constructor > > String(ByteBuffer[] srcs, String charset); > (or more generally, String(ByteBuffer[] srcs, int off, int len, String charset) > > to create a String object from a sequence of ByteBuffers, if it's really desired. > > And then I would also assume it will also be desired to extend the current > CharsetDecoder/Encoder class as well to add a pair of the "gathering" style coding > methods > > CharBuffer CharsetDecoder.decode(ByteBuffer... ins); > ByteBuffer CharsetEncoder.encode(CharBuffer... ins); > > Though the implementation might have to deal with the tricky "splitting > byte/char" issue, in which part of the "byte/char sequence" is in the previous > buffer and the continuing byte/chars are in the next following buffer ... > > -Sherman > > > On 2/9/16 7:20 AM, Chris Vest wrote: >> Hi, >> >> Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string data in Latin1, which is one of the two special encodings for compact strings, we?d ideally like to produce the String object with just the two necessary object allocations and a single, vectorised memory copy. >> >> Our use case is that we are a database and we do our own file paging, effectively having file data in a large set of DirectByteBuffers. We have string data in our files in a number of different encodings, a popular one being Latin1. Occasionally these String values span multiple buffers. We often need to expose this data as String objects, in which case decoding the bytes and turning them into a String is often very performance sensitive - to the point of being one of our top bottlenecks for the given queries. Part of the story is that in the case of Latin1, I?ll know up front exactly how many bytes my string data takes up, though I might not know how many buffers are going to be involved. >> >> As far as I can tell, this is currently not possible using public APIs. Using private APIs it may be possible, but will be relying on the JIT for vectorising the memory copying. >> >> From an API standpoint, CharsetDecoder is close to home, but is not quite there. It?s stateful and not thread-safe, so I either have to allocate new ones every time or cache them in thread-locals. I?m also required to allocate the receiving CharBuffer. Since I may need to decode from multiple buffers, I realise that I might not be able to get away from allocating at least one extra object to keep track of intermediate decoding state. The CharsetDecoder does not have a method where I can specify the offset and length for the desired part of the ByteBuffer I want to decode, which forces be to allocate views instead. >> >> The CharBuffers are allocated with a length up front, which is nice, but I can?t restrict its encoding so it has to allocate a char array instead of the byte array that I really want. Even if it did allocate a byte array, the CharBuffer is mutable, which would force String do a defensive copy anyway. >> >> One way I imagine this could be solved would be with a less dynamic kind of decoder, where the target length is given upfront to the decoder. Buffers are then consumed one by one, and a terminal method performs finishing sanity checks (did we get all the bytes we were promised?) and returns the result. >> >> StringDecoder decoder = Charset.forName(?latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); >> String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, len2).done(); >> >> This will in principle allow the string decoding to be 2 small allocations, an array allocation without zeroing, and a sequence of potentially vectorised memcpys. I don?t see any potentially troubling interactions with fused Strings either, since all the knowledge (except for the string data itself) needed to allocate the String objects are available from the get-go. >> >> What do you guys think? >> >> Btw, Richard Warburton has already done some work in this area, and made a patch that adds a constructor to String that takes a buffer, offset, length, and charset. This work now at least needs rebasing: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ >> It doesn?t solve the case where multiple buffers are used to build the string, but does remove the need for a separate intermediate state-holding object when a single buffer is enough. It?d be a nice addition if possible, but I (for one) can tolerate a small object allocation otherwise. >> >> Cheers, >> Chris >> > From michael.haupt at oracle.com Wed Feb 10 09:48:03 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Wed, 10 Feb 2016 10:48:03 +0100 Subject: RFR 8138884:MethodHandles.Lookup.findVirtual() Javadoc fails to consider private interface methods In-Reply-To: <3BA12D86-8A4B-438E-80F0-15F3FDBEE4B0@oracle.com> References: <569F80B0.3040103@oracle.com> <56A1F4AE.4050201@oracle.com> <56BAE003.7000602@oracle.com> <3BA12D86-8A4B-438E-80F0-15F3FDBEE4B0@oracle.com> Message-ID: <920BD333-2CE8-4E8C-B607-5330767B985F@oracle.com> Hi, I'll be happy to sponsor this. Best, Michael > Am 10.02.2016 um 09:06 schrieb Paul Sandoz : > > >> On 10 Feb 2016, at 08:00, shilpi rastogi wrote: >> >> Hi All, >> >> Please review my updated patch >> http://cr.openjdk.java.net/~srastogi/8138884/webrev.01/ >> > > Just add a ?,? after the new statement > > .. {@code static}, > > No need for another review. > > >> Same is not applicable for "Lookup.unreflect()". >> > > Ok. Thanks for checking. > > Paul. -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From michael.haupt at oracle.com Wed Feb 10 09:48:15 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Wed, 10 Feb 2016 10:48:15 +0100 Subject: RFR: 8071368 Use more concrete types for NamedFunction constants in the code In-Reply-To: <86D5EA70-E7ED-42C2-AEC2-085733A69758@oracle.com> References: <56B34F40.1010800@oracle.com> <6804C2B3-DE2F-4DCF-B70D-6F3BC846B403@oracle.com> <56B4C467.7040300@oracle.com> <56B895B9.40401@oracle.com> <56BAE03C.2070406@oracle.com> <86D5EA70-E7ED-42C2-AEC2-085733A69758@oracle.com> Message-ID: Hi, I'll be happy to sponsor this. Best, Michael > Am 10.02.2016 um 09:03 schrieb Paul Sandoz : > > Hi Shilpi, > > I think you can consider this reviewed. It got the ok from Vladimir, with reference to John?s feedback in the bug. > > Pau. > >> On 10 Feb 2016, at 08:01, shilpi rastogi wrote: >> >> >> Gentle Reminder! >>> >>> On 2/5/2016 9:18 PM, Vladimir Ivanov wrote: >>>> Proposed fix looks good. >>>> >>>> Quoting John: "The use of erased types (any ref => Object) in the MH runtime is an artifact of bootstrapping difficulties, early in the project. I hope it is not necessary any more." >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> > -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From paul.sandoz at oracle.com Wed Feb 10 09:52:27 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 10:52:27 +0100 Subject: API review of VarHandles In-Reply-To: References: <9461B198-5706-459C-A6BB-E8282AF9237A@oracle.com> <4EC42B69-2603-43BC-AB29-BAD8EF53F90E@oracle.com> Message-ID: <43E1CE7C-A367-4687-BD02-5A9ED39828AE@oracle.com> > On 10 Feb 2016, at 02:03, Hans Boehm wrote: > > I found this a bit scary for naive readers: > > * The Java Language Specification permits operations to be executed in > * orders different than are apparent in program source code, subject to > * constraints arising, for example, from the use of locks, {@code volatile} > * fields or VarHandles. > > I would change it to > > * The Java Language Specification permits other threads to observe operations > * as if they were executed in orders different than are apparent in program source code, > * subject to constraints arising, for example, from the use of locks, {@code volatile} > * fields or VarHandles. > > to make it clear that we are not talking about intra-thread semantics here. > > That way it will hopefully only be scary to experts, as intended. > Many thanks! updated to what you suggest, Paul. From aleksey.shipilev at oracle.com Wed Feb 10 09:58:21 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 10 Feb 2016 12:58:21 +0300 Subject: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory Message-ID: <56BB09BD.4000001@oracle.com> Hi, I would like to make StringConcatFactory more coherent with existing LambdaMetafactory: https://bugs.openjdk.java.net/browse/JDK-8149459 http://cr.openjdk.java.net/~shade/8149459/webrev.01/ Testing: jdk/test/java/String/concat jtreg, JPRT Cheers, -Aleksey From paul.sandoz at oracle.com Wed Feb 10 10:03:40 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 11:03:40 +0100 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: <56BAB18B.3070707@oracle.com> References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> <56B5207B.8010107@oracle.com> <56B7DDD8.2050701@oracle.com> <56BAB18B.3070707@oracle.com> Message-ID: > On 10 Feb 2016, at 04:42, Mikael Vidstedt wrote: > > > Can I please get a quick review of these updated webrevs: > > hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/hotspot/webrev/ > jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/jdk/webrev/ > > incremental webrevs: > > hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/hotspot/webrev/ > jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/jdk/webrev/ > +1 I agree with David on the JavaDoc, but that could be followed up with any future changes, including potentially the removal of wrapping methods in Bits.java, since buffers any way use Unsafe the wrappers now appear to offer little value. Paul. > Changes: > > * Added asserts in copy.cpp/conjoint_swap > * Correctness: Moved offset sign checks to only be performed if corresponding base object is null, and added corresponding tests > > I'm about to make additional changes in this same area, so unless this last change is horribly broken I'm planning on pushing this and doing any additional cleanup in the upcoming change(s). > > Cheers, > Mikael > From paul.sandoz at oracle.com Wed Feb 10 10:19:07 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 11:19:07 +0100 Subject: RFR 8138884:MethodHandles.Lookup.findVirtual() Javadoc fails to consider private interface methods In-Reply-To: <3BA12D86-8A4B-438E-80F0-15F3FDBEE4B0@oracle.com> References: <569F80B0.3040103@oracle.com> <56A1F4AE.4050201@oracle.com> <56BAE003.7000602@oracle.com> <3BA12D86-8A4B-438E-80F0-15F3FDBEE4B0@oracle.com> Message-ID: <26CAC34C-117F-4C43-8F98-2A6E5EC0258B@oracle.com> > On 10 Feb 2016, at 09:06, Paul Sandoz wrote: > > >> On 10 Feb 2016, at 08:00, shilpi rastogi wrote: >> >> Hi All, >> >> Please review my updated patch >> http://cr.openjdk.java.net/~srastogi/8138884/webrev.01/ >> > > Just add a ?,? after the new statement > > .. {@code static}, > Ignore that, i see Micheal did already :-) my eyes got crossed when reviewing side-by-side. Paul. From chris.hegarty at oracle.com Wed Feb 10 10:50:31 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 10 Feb 2016 10:50:31 +0000 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: Message-ID: <56BB15F7.1060606@oracle.com> On 02/02/16 15:23, Martin Buchholz wrote: > On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty wrote: >> >> On 1 Feb 2016, at 18:45, Martin Buchholz wrote: >> >>> After much debate on what to do when CompleteableFuture.whenComplete >>> encounters an exception in both the source and the action, we chose >>> what was acceptable to the most people - add the action's exception to >>> the source exception as a suppressed exception. And added usage >>> guidelines. And gave handle "top billing" over whenComplete. >>> >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ >> >> This all looks fine to me. >> >> So I assume you only need a small CCC request for CompletionStage, right? >> Everything else is implementation. > > If you squint you might argue that CompletionStage's contract hasn't > actually changed, > but yeah, go ahead and do a CCC for CompletionStage. Publishing a > specdiff would be nice - method reordering (for "top billing") has > made the diffs harder to review. Thanks. Here are the specdiffs that will be used for the CCC, unless there are any last minute changes. http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html -Chris. From forax at univ-mlv.fr Wed Feb 10 11:13:41 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 10 Feb 2016 12:13:41 +0100 (CET) Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> References: <56BABEB8.2050505@oracle.com> <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> Message-ID: <1948539740.1477006.1455102821585.JavaMail.zimbra@u-pem.fr> Or as Chris explain, having a string that span on more than one buffer is a corner case of this software, so for most of the strings, the constructor that takes a buffer is fine, and for the corner case, a constructor of String that takes a CharSequence seems to be easier to implement than creating a new kind of buffer that represents several buffers. and by the way, i would prefer to have static factory methods instead of constructors in String, there are already too many constructors. regards, R?mi ----- Mail original ----- > De: "Paul Sandoz" > Cc: core-libs-dev at openjdk.java.net > Envoy?: Mercredi 10 F?vrier 2016 09:54:17 > Objet: Re: Compact Strings and APIs for fast decoding of string data > > Hi, > > A more functional approach would be to compose a sequence buffers into one > view, perhaps read-only. Then there would be no need to accept arrays of > buffers. That should work well for bulk operations. That?s a non-trivial but > not very difficult amount of work, and possibly simplified if restricted to > read-only views. > > Thus i think we should focus Richard?s work with: > > String(ByteBuffer src, String charset) > > and perhaps a sub-range variant, if perturbing the position/limit of an > existing buffer and/or slicing is too problematic. > > ? > > Zeroing memory and possibly avoiding it can be tricky. Any such optimisations > have to be carefully performed otherwise uninitialised regions might leak > and be accessed, nefariously or otherwise. I imagine it?s easier to > contain/control within a constructor than say a builder. > > Paul. > > > On 10 Feb 2016, at 05:38, Xueming Shen wrote: > > > > Hi Chris, > > > > I think basically you are asking a String constructor that takes a > > ByteBuffer. StringCoding > > then can take advantage of the current CompactString design to optimize the > > decoding > > operation by just a single byte[]/vectorized memory copy from the > > ByteBuffer to the String's > > internal byte[], WHEN the charset is 8859-1. > > > > String(ByteBuffer src, String charset); > > > > Further we will need a "buffer gathering" style constructor > > > > String(ByteBuffer[] srcs, String charset); > > (or more generally, String(ByteBuffer[] srcs, int off, int len, String > > charset) > > > > to create a String object from a sequence of ByteBuffers, if it's really > > desired. > > > > And then I would also assume it will also be desired to extend the current > > CharsetDecoder/Encoder class as well to add a pair of the "gathering" style > > coding > > methods > > > > CharBuffer CharsetDecoder.decode(ByteBuffer... ins); > > ByteBuffer CharsetEncoder.encode(CharBuffer... ins); > > > > Though the implementation might have to deal with the tricky "splitting > > byte/char" issue, in which part of the "byte/char sequence" is in the > > previous > > buffer and the continuing byte/chars are in the next following buffer ... > > > > -Sherman > > > > > > On 2/9/16 7:20 AM, Chris Vest wrote: > >> Hi, > >> > >> Aleksey Shipilev did a talk on his journey to implement compact strings > >> and indified string concat at the JVM Tech Summit yesterday, and this > >> reminded me that we (Neo4j) have a need for turning segments of > >> DirectByteBuffers into Strings as fast as possible. If we already store > >> the string data in Latin1, which is one of the two special encodings for > >> compact strings, we?d ideally like to produce the String object with just > >> the two necessary object allocations and a single, vectorised memory > >> copy. > >> > >> Our use case is that we are a database and we do our own file paging, > >> effectively having file data in a large set of DirectByteBuffers. We have > >> string data in our files in a number of different encodings, a popular > >> one being Latin1. Occasionally these String values span multiple buffers. > >> We often need to expose this data as String objects, in which case > >> decoding the bytes and turning them into a String is often very > >> performance sensitive - to the point of being one of our top bottlenecks > >> for the given queries. Part of the story is that in the case of Latin1, > >> I?ll know up front exactly how many bytes my string data takes up, though > >> I might not know how many buffers are going to be involved. > >> > >> As far as I can tell, this is currently not possible using public APIs. > >> Using private APIs it may be possible, but will be relying on the JIT for > >> vectorising the memory copying. > >> > >> From an API standpoint, CharsetDecoder is close to home, but is not quite > >> there. It?s stateful and not thread-safe, so I either have to allocate > >> new ones every time or cache them in thread-locals. I?m also required to > >> allocate the receiving CharBuffer. Since I may need to decode from > >> multiple buffers, I realise that I might not be able to get away from > >> allocating at least one extra object to keep track of intermediate > >> decoding state. The CharsetDecoder does not have a method where I can > >> specify the offset and length for the desired part of the ByteBuffer I > >> want to decode, which forces be to allocate views instead. > >> > >> The CharBuffers are allocated with a length up front, which is nice, but I > >> can?t restrict its encoding so it has to allocate a char array instead of > >> the byte array that I really want. Even if it did allocate a byte array, > >> the CharBuffer is mutable, which would force String do a defensive copy > >> anyway. > >> > >> One way I imagine this could be solved would be with a less dynamic kind > >> of decoder, where the target length is given upfront to the decoder. > >> Buffers are then consumed one by one, and a terminal method performs > >> finishing sanity checks (did we get all the bytes we were promised?) and > >> returns the result. > >> > >> StringDecoder decoder = > >> Charset.forName(?latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); > >> String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, > >> len2).done(); > >> > >> This will in principle allow the string decoding to be 2 small > >> allocations, an array allocation without zeroing, and a sequence of > >> potentially vectorised memcpys. I don?t see any potentially troubling > >> interactions with fused Strings either, since all the knowledge (except > >> for the string data itself) needed to allocate the String objects are > >> available from the get-go. > >> > >> What do you guys think? > >> > >> Btw, Richard Warburton has already done some work in this area, and made a > >> patch that adds a constructor to String that takes a buffer, offset, > >> length, and charset. This work now at least needs rebasing: > >> http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ > >> > >> It doesn?t solve the case where multiple buffers are used to build the > >> string, but does remove the need for a separate intermediate > >> state-holding object when a single buffer is enough. It?d be a nice > >> addition if possible, but I (for one) can tolerate a small object > >> allocation otherwise. > >> > >> Cheers, > >> Chris > >> > > > > From volker.simonis at gmail.com Wed Feb 10 11:15:09 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 10 Feb 2016 12:15:09 +0100 Subject: RFR (S): 8149529: Adapt SAP copyrights to new company name in jdk repository Message-ID: Hi, can I please have a review for this boring change which simply changes all the SAP copyright messages from "SAP AG" to "SAP SE". The reason for this change is that SAP changed from a German stock corporation (AG) to a European one (SE). http://cr.openjdk.java.net/~simonis/webrevs/2016/8149529 https://bugs.openjdk.java.net/browse/JDK-8149529 Thank you and best regards, Volker From sbaiduzh at redhat.com Wed Feb 10 11:17:48 2016 From: sbaiduzh at redhat.com (Stanislav Baiduzhyi) Date: Wed, 10 Feb 2016 12:17:48 +0100 Subject: New Collection framework? In-Reply-To: <56BAB6A0.7070402@oracle.com> References: <2485208.rZGvsojIaB@linux-vgv1> <56BAB6A0.7070402@oracle.com> Message-ID: <56BB1C5C.9080404@redhat.com> On 10/02/16 05:03, David Holmes wrote: >> * The main Collection interface allowes too much. If you get a >> Collection and >> you should not modify it, your programm will either behave stragly >> with a hard >> to detect bug, or crash with some kind of >> UnsupportedOperationException, when >> you do it. >> I have my problems with UnsupprtedOperationException, because when i >> get a >> Collection, i have to read the documentation (if existing) to know >> what i am >> allowed to do, or run unexpectedly into an exception. Would it be >> great to >> hava an "readonly" interface, which simply does not have and modifing >> methods? > > I don't understand your problem. If you write a method that takes a > collection then your method has a specification as to what it does that > involves the collection that is passed in. If the passed in collection > doesn't in fact support that then the UnsupportedOperationException > propagates back to the caller that passed you an inappropriate > collection - and that's exactly as it should be. If you take a > collection and then try to do something to it that you didn't specify > you were going to do, and get UnsupportedOperationException then that is > a bug in your code - if you let it propagate then your caller gets it > and the user of your code submits a bug report. > > In case you have not seen it there is a document that discusses the > design of the Collections framework and why the existing approach was > taken: > > https://docs.oracle.com/javase/8/docs/technotes/guides/collections/designfaq.html No that's little different. I've had similar issues on multiple projects. When you want to keep your DTOs immutable, you have to either return a copy of collection in getter or wrap it into unmodifiable. Unmodifiable is not visible on API level but behaves differently on implementation level, making it harder to support. So many projects are just returning Guava ImmutableList or similar, to make it obvious on API level that this collection should not be modified. So yes, you are correct that it is documented and not an issue of JDK. Yet it is a valid issue of API design, having something like ImmutableList on API level would make it much simpler to follow self-documenting code principles and would improve maintenance cost in general. It would be very helpful to have a Collection extending some kind of ReadOnlyCollection (and similar to List and Set) that has only read operations, but probably that is not doable in backward-compatible manner. -- Regards, Stas From forax at univ-mlv.fr Wed Feb 10 11:25:49 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 10 Feb 2016 12:25:49 +0100 (CET) Subject: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory In-Reply-To: <56BB09BD.4000001@oracle.com> References: <56BB09BD.4000001@oracle.com> Message-ID: <1845801185.1485912.1455103549086.JavaMail.zimbra@u-pem.fr> Hi Aleksey, if you have a dumper, you should append a value at the end of the generated class otherwise if you have more than one string concatenation, you will see only one class on the disk. regards, R?mi ----- Mail original ----- > De: "Aleksey Shipilev" > ?: "Java Core Libs" > Envoy?: Mercredi 10 F?vrier 2016 10:58:21 > Objet: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory > > Hi, > > I would like to make StringConcatFactory more coherent with existing > LambdaMetafactory: > https://bugs.openjdk.java.net/browse/JDK-8149459 > http://cr.openjdk.java.net/~shade/8149459/webrev.01/ > > Testing: jdk/test/java/String/concat jtreg, JPRT > > Cheers, > -Aleksey > > From chris.hegarty at oracle.com Wed Feb 10 11:30:11 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 10 Feb 2016 11:30:11 +0000 Subject: RFR (S): 8149529: Adapt SAP copyrights to new company name in jdk repository In-Reply-To: References: Message-ID: <56BB1F43.4030800@oracle.com> On 10/02/16 11:15, Volker Simonis wrote: > Hi, > > can I please have a review for this boring change which simply changes > all the SAP copyright messages from "SAP AG" to "SAP SE". The reason > for this change is that SAP changed from a German stock corporation > (AG) to a European one (SE). > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8149529 > https://bugs.openjdk.java.net/browse/JDK-8149529 Skimming the patch file, it looks ok to me. -Chris From forax at univ-mlv.fr Wed Feb 10 11:55:14 2016 From: forax at univ-mlv.fr (Remi Forax) Date: Wed, 10 Feb 2016 12:55:14 +0100 (CET) Subject: New Collection framework? In-Reply-To: <56BB1C5C.9080404@redhat.com> References: <2485208.rZGvsojIaB@linux-vgv1> <56BAB6A0.7070402@oracle.com> <56BB1C5C.9080404@redhat.com> Message-ID: <1535226842.1492778.1455105314438.JavaMail.zimbra@u-pem.fr> ----- Mail original ----- > De: "Stanislav Baiduzhyi" > ?: core-libs-dev at openjdk.java.net > Envoy?: Mercredi 10 F?vrier 2016 12:17:48 > Objet: Re: New Collection framework? > > On 10/02/16 05:03, David Holmes wrote: > >> * The main Collection interface allowes too much. If you get a > >> Collection and > >> you should not modify it, your programm will either behave stragly > >> with a hard > >> to detect bug, or crash with some kind of > >> UnsupportedOperationException, when > >> you do it. > >> I have my problems with UnsupprtedOperationException, because when i > >> get a > >> Collection, i have to read the documentation (if existing) to know > >> what i am > >> allowed to do, or run unexpectedly into an exception. Would it be > >> great to > >> hava an "readonly" interface, which simply does not have and modifing > >> methods? > > > > I don't understand your problem. If you write a method that takes a > > collection then your method has a specification as to what it does that > > involves the collection that is passed in. If the passed in collection > > doesn't in fact support that then the UnsupportedOperationException > > propagates back to the caller that passed you an inappropriate > > collection - and that's exactly as it should be. If you take a > > collection and then try to do something to it that you didn't specify > > you were going to do, and get UnsupportedOperationException then that is > > a bug in your code - if you let it propagate then your caller gets it > > and the user of your code submits a bug report. > > > > In case you have not seen it there is a document that discusses the > > design of the Collections framework and why the existing approach was > > taken: > > > > https://docs.oracle.com/javase/8/docs/technotes/guides/collections/designfaq.html > > No that's little different. I've had similar issues on multiple > projects. When you want to keep your DTOs immutable, you have to either > return a copy of collection in getter or wrap it into unmodifiable. > Unmodifiable is not visible on API level but behaves differently on > implementation level, making it harder to support. So many projects are > just returning Guava ImmutableList or similar, to make it obvious on API > level that this collection should not be modified. > > So yes, you are correct that it is documented and not an issue of JDK. > Yet it is a valid issue of API design, having something like > ImmutableList on API level would make it much simpler to follow > self-documenting code principles and would improve maintenance cost in > general. > Java 8 to the rescue :) The problem you have is that you don't know if the list is the result of a calculation or if it reflect an internal state thus has to be immutable. The idea is not to use the interface List in both cases, the result of a calculation should be a Stream and not a List, a Stream can be seen as the result of a calculation without specifying the collection doing the actual storage (before 8, the JDK tends to use java.util.Enumeration and then Iterator to represent the result of a calculation). So you should never think that it's safe to mutate a List you get as a return value of a method. > It would be very helpful to have a Collection extending some kind of > ReadOnlyCollection (and similar to List and Set) that has only read > operations, but probably that is not doable in backward-compatible manner. as David says it was rule out when the collection API was created as too complex because you multiply the number of interfaces (not enough bangs for bucks), moreover, there is now the question of how to integrate persistent collections in that picture, something that should be considered now in 2016. > > -- > Regards, > Stas > regards, R?mi From goetz.lindenmaier at sap.com Wed Feb 10 11:57:43 2016 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 10 Feb 2016 11:57:43 +0000 Subject: RFR (S): 8149529: Adapt SAP copyrights to new company name in jdk repository In-Reply-To: References: Message-ID: Hi Volker, thanks for doing this tedious change :) It looks good, I checked these are all. Best regards, Goetz > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: Mittwoch, 10. Februar 2016 12:15 > To: Java Core Libs > Cc: Lindenmaier, Goetz > Subject: RFR (S): 8149529: Adapt SAP copyrights to new company name in jdk > repository > > Hi, > > can I please have a review for this boring change which simply changes > all the SAP copyright messages from "SAP AG" to "SAP SE". The reason > for this change is that SAP changed from a German stock corporation > (AG) to a European one (SE). > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8149529 > https://bugs.openjdk.java.net/browse/JDK-8149529 > > Thank you and best regards, > Volker From peter.firmstone at zeus.net.au Wed Feb 10 12:09:52 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Wed, 10 Feb 2016 22:09:52 +1000 (AEST) Subject: ObjectInputStream SPI Message-ID: <79f63ad7704ef2d3bdd486f0f8461aba@org.tizen.email> Thanks Chris, Over the last year I've been working on reimplimenting OIS, I've been able to retain serial form compatibility, but have made some changes to deserialization api. With the reimpl, existing Serializable objects that are stateless or those with only primitive fields are allowed to be deserialized. DeserializationPermission is required for classes with Object fields or that read input from the stream. ?A context is constructed from every class in the object's inheritance hierarchy and checked by the sm. I've chosen not to support private readObject methods and utilise annotations and constructor signatures instead, to allow failure atomicity. I've created a utility class with static methods that perform common invariant validation, such as null check, collection element type checks and defensive copying. ?These methods can be chained and throw InvalidObjectException. ?For most cases these utility methods allow the developer to simplify or eliminate the need to implement their own static validation methods. I've added two new public methods to OIS: T readObject(Class type) and another similar method for readUnshared. In this case the object is type checked for assignment compatibility before deserialization. ?This presents compatibility issues with the Serialization proxy pattern however. ?Solution: if a readResolve method is defined, its return argument type is checked, before deserialization occurs. ?Presently according to the serialization spec, the only allowed return type is object, any chance of allowing covariant return types for readResolve? Java collection classes are replaced in the stream with serializers for Map SortedMap, SortedSet, Set and List, this avoids array length issues or deliberate hash collisions. ?Client code must check all types within the collections when creating a defensive copy. At this time circular object graphs are not supported. I realise these changes aren't possible for standard java serialization as too many breakages would occur. As this OIS will be exposed to untrustred data, I must ensure it is not be subject to gadget or dos style attacks. Regards, Peter. Sent fro ?m my Samsung device. ? From sbaiduzh at redhat.com Wed Feb 10 12:18:33 2016 From: sbaiduzh at redhat.com (Stanislav Baiduzhyi) Date: Wed, 10 Feb 2016 13:18:33 +0100 Subject: New Collection framework? In-Reply-To: <1535226842.1492778.1455105314438.JavaMail.zimbra@u-pem.fr> References: <2485208.rZGvsojIaB@linux-vgv1> <56BAB6A0.7070402@oracle.com> <56BB1C5C.9080404@redhat.com> <1535226842.1492778.1455105314438.JavaMail.zimbra@u-pem.fr> Message-ID: <56BB2A99.30006@redhat.com> On 10/02/16 12:55, Remi Forax wrote: > Java 8 to the rescue :) > > The problem you have is that you don't know if the list is the result of a calculation or if it reflect an internal state thus has to be immutable. > The idea is not to use the interface List in both cases, the result of a calculation should be a Stream and not a List, a Stream can be seen as the result of a calculation without specifying the collection doing the actual storage (before 8, the JDK tends to use java.util.Enumeration and then Iterator to represent the result of a calculation). > So you should never think that it's safe to mutate a List you get as a return value of a method. In perfect world yes, but in our world we have to wait for JPA and other persistence frameworks to catch up with Java 8 and give us some Stream-friendly APIs. For now we're still stuck with Guava ImmutableLists. > as David says it was rule out when the collection API was created as too complex because you multiply the number of interfaces (not enough bangs for bucks), Pity. Now instead of nice and clean interfaces we have loads of libraries providing the same often incompatible functionality and even different naming for the same things. Remind of famous comics at xkcb about 14 competing standards. -- Regards, Stas From aleksey.shipilev at oracle.com Wed Feb 10 12:51:35 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 10 Feb 2016 15:51:35 +0300 Subject: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory In-Reply-To: <1845801185.1485912.1455103549086.JavaMail.zimbra@u-pem.fr> References: <56BB09BD.4000001@oracle.com> <1845801185.1485912.1455103549086.JavaMail.zimbra@u-pem.fr> Message-ID: <56BB3257.2020601@oracle.com> See new webrev that adds a few more tests: http://cr.openjdk.java.net/~shade/8149459/webrev.02/ On 10.02.2016 14:25, Remi Forax wrote: > Hi Aleksey, if you have a dumper, you should append a value at the > end of the generated class otherwise if you have more than one string > concatenation, you will see only one class on the disk. No, we are actually okay, because our class names are unique since Unsafe.defineAnonymousClass handles this: $ ls -lR dump/java.lang.String\$Concat/ dump/java.lang.String$Concat/2008017533.class dump/java.lang.String$Concat/665576141.class Cheers, -Aleksey From spliterator at gmail.com Wed Feb 10 13:04:20 2016 From: spliterator at gmail.com (Stefan Zobel) Date: Wed, 10 Feb 2016 14:04:20 +0100 Subject: RFR-8148838: Stream.flatMap(...).spliterator() cannot properly split after tryAdvance() In-Reply-To: <7C1931B7-8641-4DC3-82F9-F9F36287F339@oracle.com> References: <1591524905.20160202152420@gmail.com> <7286C7E7-C82B-4EA4-AAEB-208FA3E3C1E1@oracle.com> <179345382.20160203181946@gmail.com> <62E09116-D984-44E6-BEF6-CE2EB8A16297@oracle.com> <1489632171.20160206192945@gmail.com> <6A23376A-20E8-4717-B979-26D698E8DB29@oracle.com> <701936850.20160208195323@gmail.com> <7C1931B7-8641-4DC3-82F9-F9F36287F339@oracle.com> Message-ID: > 2016-02-08 15:05 GMT+01:00 Paul Sandoz : > >> On 8 Feb 2016, at 14:53, Tagir F. Valeev wrote: >> >> PS> Why don?t you check if "buffer == null? at #189? i.e. similar to forEachRemaining: >> >> That would make minimal behavioral change to fix this issue (fix >> flatMap keys only, but not affect other intermediate ops which were >> working correctly). Well, if buffer == null check is enough, here's >> update: >> > > Many thanks, yes it?s sufficient (finished == true when buffer != null && buffer.count() == 0). > Hi all, Question: will this get backported to Java 8 ? Regards, Stefan From paul.sandoz at oracle.com Wed Feb 10 13:16:00 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 14:16:00 +0100 Subject: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory In-Reply-To: <56BB3257.2020601@oracle.com> References: <56BB09BD.4000001@oracle.com> <1845801185.1485912.1455103549086.JavaMail.zimbra@u-pem.fr> <56BB3257.2020601@oracle.com> Message-ID: <763FAA77-D599-4489-B7E3-EAB6294539F5@oracle.com> > On 10 Feb 2016, at 13:51, Aleksey Shipilev wrote: > > See new webrev that adds a few more tests: > http://cr.openjdk.java.net/~shade/8149459/webrev.02/ > +1 Paul. From mr.chrisvest at gmail.com Wed Feb 10 13:16:55 2016 From: mr.chrisvest at gmail.com (Chris Vest) Date: Wed, 10 Feb 2016 14:16:55 +0100 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> References: <56BABEB8.2050505@oracle.com> <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> Message-ID: <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> It?d be a shame to loose the no-zeroing optimisation, so if that requires a constructor, then lets find a constructor. To recap, I have possibly multiple buffers and I only want to use parts of each. So with the composed buffer I?d have to first extract view buffers and then compose those, so I end up with 3 levels of buffers. Not ideal. A value type as a tuple of a buffer reference, an offset and a length would be better. We could make an array of those and pass that in. It could also find uses elsewhere in JDK where the buffer+offset+length pattern occurs. The array, a long with an encoding, could be passed up front to a String constructor (or a static method ? I don?t care) and my use case would be equally satisfied. If I?ve understood correctly, there?s a plan (or at least a great desire) to turn Optional into a value type once they become available. These ?buffer slice? objects could follow a similar path, and thus be made available independently of value types. Even if they can?t, the overhead they add would be small, I imagine. I?m slightly more worried about their array and what it implies, though. This API requires of me that I?m able to present all of the relevant buffers up front. In my case, these buffers are shared between multiple threads, so they are guarded by locks. I must take all the locks for all the buffers and hold them for the duration of the copy. _Maybe_ I have to do that anyway, but if there?s a way for me to only lock one buffer at a time, I?d like to be able to exploit this. The builder approach holds this door open for me. Cheers, Chris > On 10 Feb 2016, at 09:54, Paul Sandoz wrote: > > Hi, > > A more functional approach would be to compose a sequence buffers into one view, perhaps read-only. Then there would be no need to accept arrays of buffers. That should work well for bulk operations. That?s a non-trivial but not very difficult amount of work, and possibly simplified if restricted to read-only views. > > Thus i think we should focus Richard?s work with: > > String(ByteBuffer src, String charset) > > and perhaps a sub-range variant, if perturbing the position/limit of an existing buffer and/or slicing is too problematic. > > ? > > Zeroing memory and possibly avoiding it can be tricky. Any such optimisations have to be carefully performed otherwise uninitialised regions might leak and be accessed, nefariously or otherwise. I imagine it?s easier to contain/control within a constructor than say a builder. > > Paul. > >> On 10 Feb 2016, at 05:38, Xueming Shen wrote: >> >> Hi Chris, >> >> I think basically you are asking a String constructor that takes a ByteBuffer. StringCoding >> then can take advantage of the current CompactString design to optimize the decoding >> operation by just a single byte[]/vectorized memory copy from the ByteBuffer to the String's >> internal byte[], WHEN the charset is 8859-1. >> >> String(ByteBuffer src, String charset); >> >> Further we will need a "buffer gathering" style constructor >> >> String(ByteBuffer[] srcs, String charset); >> (or more generally, String(ByteBuffer[] srcs, int off, int len, String charset) >> >> to create a String object from a sequence of ByteBuffers, if it's really desired. >> >> And then I would also assume it will also be desired to extend the current >> CharsetDecoder/Encoder class as well to add a pair of the "gathering" style coding >> methods >> >> CharBuffer CharsetDecoder.decode(ByteBuffer... ins); >> ByteBuffer CharsetEncoder.encode(CharBuffer... ins); >> >> Though the implementation might have to deal with the tricky "splitting >> byte/char" issue, in which part of the "byte/char sequence" is in the previous >> buffer and the continuing byte/chars are in the next following buffer ... >> >> -Sherman >> >> >> On 2/9/16 7:20 AM, Chris Vest wrote: >>> Hi, >>> >>> Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string data in Latin1, which is one of the two special encodings for compact strings, we?d ideally like to produce the String object with just the two necessary object allocations and a single, vectorised memory copy. >>> >>> Our use case is that we are a database and we do our own file paging, effectively having file data in a large set of DirectByteBuffers. We have string data in our files in a number of different encodings, a popular one being Latin1. Occasionally these String values span multiple buffers. We often need to expose this data as String objects, in which case decoding the bytes and turning them into a String is often very performance sensitive - to the point of being one of our top bottlenecks for the given queries. Part of the story is that in the case of Latin1, I?ll know up front exactly how many bytes my string data takes up, though I might not know how many buffers are going to be involved. >>> >>> As far as I can tell, this is currently not possible using public APIs. Using private APIs it may be possible, but will be relying on the JIT for vectorising the memory copying. >>> >>> From an API standpoint, CharsetDecoder is close to home, but is not quite there. It?s stateful and not thread-safe, so I either have to allocate new ones every time or cache them in thread-locals. I?m also required to allocate the receiving CharBuffer. Since I may need to decode from multiple buffers, I realise that I might not be able to get away from allocating at least one extra object to keep track of intermediate decoding state. The CharsetDecoder does not have a method where I can specify the offset and length for the desired part of the ByteBuffer I want to decode, which forces be to allocate views instead. >>> >>> The CharBuffers are allocated with a length up front, which is nice, but I can?t restrict its encoding so it has to allocate a char array instead of the byte array that I really want. Even if it did allocate a byte array, the CharBuffer is mutable, which would force String do a defensive copy anyway. >>> >>> One way I imagine this could be solved would be with a less dynamic kind of decoder, where the target length is given upfront to the decoder. Buffers are then consumed one by one, and a terminal method performs finishing sanity checks (did we get all the bytes we were promised?) and returns the result. >>> >>> StringDecoder decoder = Charset.forName(?latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); >>> String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, len2).done(); >>> >>> This will in principle allow the string decoding to be 2 small allocations, an array allocation without zeroing, and a sequence of potentially vectorised memcpys. I don?t see any potentially troubling interactions with fused Strings either, since all the knowledge (except for the string data itself) needed to allocate the String objects are available from the get-go. >>> >>> What do you guys think? >>> >>> Btw, Richard Warburton has already done some work in this area, and made a patch that adds a constructor to String that takes a buffer, offset, length, and charset. This work now at least needs rebasing: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ >>> It doesn?t solve the case where multiple buffers are used to build the string, but does remove the need for a separate intermediate state-holding object when a single buffer is enough. It?d be a nice addition if possible, but I (for one) can tolerate a small object allocation otherwise. >>> >>> Cheers, >>> Chris >>> >> > From vladimir.x.ivanov at oracle.com Wed Feb 10 13:19:08 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Wed, 10 Feb 2016 16:19:08 +0300 Subject: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory In-Reply-To: <56BB3257.2020601@oracle.com> References: <56BB09BD.4000001@oracle.com> <1845801185.1485912.1455103549086.JavaMail.zimbra@u-pem.fr> <56BB3257.2020601@oracle.com> Message-ID: <56BB38CC.9030604@oracle.com> Looks good. Best regards, Vladimir Ivanov On 2/10/16 3:51 PM, Aleksey Shipilev wrote: > See new webrev that adds a few more tests: > http://cr.openjdk.java.net/~shade/8149459/webrev.02/ > > On 10.02.2016 14:25, Remi Forax wrote: >> Hi Aleksey, if you have a dumper, you should append a value at the >> end of the generated class otherwise if you have more than one string >> concatenation, you will see only one class on the disk. > > No, we are actually okay, because our class names are unique since > Unsafe.defineAnonymousClass handles this: > > $ ls -lR dump/java.lang.String\$Concat/ > dump/java.lang.String$Concat/2008017533.class > dump/java.lang.String$Concat/665576141.class > > Cheers, > -Aleksey > From volker.simonis at gmail.com Wed Feb 10 13:21:22 2016 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 10 Feb 2016 14:21:22 +0100 Subject: RFR (S): 8149529: Adapt SAP copyrights to new company name in jdk repository In-Reply-To: <56BB1F43.4030800@oracle.com> References: <56BB1F43.4030800@oracle.com> Message-ID: Thanks Chris! On Wed, Feb 10, 2016 at 12:30 PM, Chris Hegarty wrote: > On 10/02/16 11:15, Volker Simonis wrote: >> >> Hi, >> >> can I please have a review for this boring change which simply changes >> all the SAP copyright messages from "SAP AG" to "SAP SE". The reason >> for this change is that SAP changed from a German stock corporation >> (AG) to a European one (SE). >> >> http://cr.openjdk.java.net/~simonis/webrevs/2016/8149529 >> https://bugs.openjdk.java.net/browse/JDK-8149529 > > > Skimming the patch file, it looks ok to me. > > -Chris From ivan.gerasimov at oracle.com Wed Feb 10 13:44:43 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Wed, 10 Feb 2016 16:44:43 +0300 Subject: [PING] RFR: JDK-8135259: InetAddress.getAllByName only reports "unknown error" instead of actual cause In-Reply-To: <3c428763-825c-40c9-b4d5-bffcd766baea@default> References: <48fd26eb-350e-4166-8fe7-6883f8562cd5@default> <3c428763-825c-40c9-b4d5-bffcd766baea@default> Message-ID: <56BB3ECB.7070108@oracle.com> Hi Ramanand! Your fix looks good to me. I'm forwarding your request to net-dev at openjdk.java.net which is probably more appropriate to review this fix. It would be good, if net-dev people can confirm they're Okay with the fix. Sincerely yours, Ivan On 10.02.2016 10:08, Ramanand Patil wrote: > Hi all, > > Please help me in getting this review done. > > Regards, > Ramanand. > > -----Original Message----- > From: Ramanand Patil > Sent: Friday, February 05, 2016 10:46 PM > To: core-libs-dev at openjdk.java.net > Subject: RFR: JDK-8135259: InetAddress.getAllByName only reports "unknown error" instead of actual cause > > Hi all > > Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8135259 > > Bug Description: Attempts to resolve a host unknown to the DNS server cause an UnknownHostException stating "unknown error" instead of "Name or service not known". > > Webrev: http://cr.openjdk.java.net/~rpatil/8135259/webrev.00/ > > Fix: Using a function call directly "gai_strerror(gai_error)" instead of using the function pointer which was declared but not initialized. Apart from this I have removed few other unused variables and function pointers. Brief description of how this bug was introduced is added to the bug comment. > > > Regards, > > Ramanand. > > > From aleksey.shipilev at oracle.com Wed Feb 10 13:47:08 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 10 Feb 2016 16:47:08 +0300 Subject: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory In-Reply-To: <56BB38CC.9030604@oracle.com> References: <56BB09BD.4000001@oracle.com> <1845801185.1485912.1455103549086.JavaMail.zimbra@u-pem.fr> <56BB3257.2020601@oracle.com> <56BB38CC.9030604@oracle.com> Message-ID: <56BB3F5C.5060904@oracle.com> Thanks guys, pushed. -Aleksey On 10.02.2016 16:19, Vladimir Ivanov wrote: > Looks good. > > Best regards, > Vladimir Ivanov > > On 2/10/16 3:51 PM, Aleksey Shipilev wrote: >> See new webrev that adds a few more tests: >> http://cr.openjdk.java.net/~shade/8149459/webrev.02/ >> >> On 10.02.2016 14:25, Remi Forax wrote: >>> Hi Aleksey, if you have a dumper, you should append a value at the >>> end of the generated class otherwise if you have more than one string >>> concatenation, you will see only one class on the disk. >> >> No, we are actually okay, because our class names are unique since >> Unsafe.defineAnonymousClass handles this: >> >> $ ls -lR dump/java.lang.String\$Concat/ >> dump/java.lang.String$Concat/2008017533.class >> dump/java.lang.String$Concat/665576141.class >> >> Cheers, >> -Aleksey >> From chris.hegarty at oracle.com Wed Feb 10 13:51:01 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 10 Feb 2016 13:51:01 +0000 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <56B90EA6.7020406@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> Message-ID: <56BB4045.40405@oracle.com> On 08/02/16 21:54, Aleksey Shipilev wrote: > ... > Still, the corrupted stream may call readUTFBody(1L + > Integer.MAX_VALUE), that yields initial capacity of -2147483648, which > in turn fails StringBuilder(...) with NegativeArraySizeException. But I > wonder if that is actually a valid UTF8 input, because you cannot have > that large of a String... Right. To retain existing behavior (negative sizes => empty string) then we can just short-circuit this. diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java @@ -3144,7 +3144,12 @@ * utflen bytes. */ private String readUTFBody(long utflen) throws IOException { - StringBuilder sbuf = new StringBuilder(); + if (utflen < 0) + return ""; + + // a reasonable initial capacity based on the UTF length + int initialCapacity = Math.min((int)utflen, 16384); + StringBuilder sbuf = new StringBuilder(initialCapacity); if (!blkmode) { end = pos = 0; } -Chris. From dhoffer6 at gmail.com Wed Feb 10 13:53:11 2016 From: dhoffer6 at gmail.com (David Hoffer) Date: Wed, 10 Feb 2016 06:53:11 -0700 Subject: OpenJDK bug question (JDK-6519463) In-Reply-To: <56BAC2FA.6040509@oracle.com> References: <56BAB83F.3010109@oracle.com> <56BAC2FA.6040509@oracle.com> Message-ID: No that file is not visible to me either. -Dave On Tue, Feb 9, 2016 at 9:56 PM, David Holmes wrote: > On 10/02/2016 2:20 PM, David Hoffer wrote: > >> That issue has the same problem, it references files that are not in the >> bug report so it won't help, really need all files to be attached to the >> bug report. >> > > Is the attachment a.zip.Z not visible to you on the bug report? > > David > > That being said we think we have tracked our problem down >> to a file transfer step after compression so I don't think our issue >> pertains to JDK/compression but just in case we are wrong I'd like to >> see the test files attached to the bug report...in case we need them. >> >> Thanks, >> -Dave >> >> On Tue, Feb 9, 2016 at 9:10 PM, David Holmes > > wrote: >> >> On 10/02/2016 4:51 AM, David Hoffer wrote: >> >> We have a production environment where we are getting errors >> similar to >> what was reported in JDK-6519463. I would like to test our code >> with the >> test file (buggedfile.bin) used in this bug report but the file >> is posted >> on an ftp server that is not publicly available. Can this file >> be attached >> to the bug report or just email me a copy? >> >> >> Will the test file in: >> >> https://bugs.openjdk.java.net/browse/JDK-4040920 >> >> suffice, as it was reported to be the underlying cause. >> >> David >> >> Thanks, >> -Dave >> >> >> From aleksey.shipilev at oracle.com Wed Feb 10 13:57:33 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 10 Feb 2016 16:57:33 +0300 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <56BB4045.40405@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> Message-ID: <56BB41CD.2050801@oracle.com> On 10.02.2016 16:51, Chris Hegarty wrote: > On 08/02/16 21:54, Aleksey Shipilev wrote: >> ... >> Still, the corrupted stream may call readUTFBody(1L + >> Integer.MAX_VALUE), that yields initial capacity of -2147483648, which >> in turn fails StringBuilder(...) with NegativeArraySizeException. But I >> wonder if that is actually a valid UTF8 input, because you cannot have >> that large of a String... > > Right. To retain existing behavior (negative sizes => empty string) > then we can just short-circuit this. > > diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java > b/src/java.base/share/classes/java/io/ObjectInputStream.java > --- a/src/java.base/share/classes/java/io/ObjectInputStream.java > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java > @@ -3144,7 +3144,12 @@ > * utflen bytes. > */ > private String readUTFBody(long utflen) throws IOException { > - StringBuilder sbuf = new StringBuilder(); > + if (utflen < 0) > + return ""; > + > + // a reasonable initial capacity based on the UTF length > + int initialCapacity = Math.min((int)utflen, 16384); > + StringBuilder sbuf = new StringBuilder(initialCapacity); > if (!blkmode) { > end = pos = 0; > } Um (Integer.MAX_VALUE + 1L) is still a positive long, you probably want: int ilen = (int)utflen; if (ilen < 0) return ""; int initialCapacity = Math.min(ilen, 16384); ...or: if (utflen < 0 || utflen > Integer.MAX_INT) return ""; int initialCapacity = Math.min((int)utflen, 16384); Cheers, -Aleksey From sean.coffey at oracle.com Wed Feb 10 13:57:58 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 10 Feb 2016 13:57:58 +0000 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56B86623.4090602@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> Message-ID: <56BB41E6.3060102@oracle.com> On 08/02/16 09:55, Alan Bateman wrote: > > On 08/02/2016 10:42, Se?n Coffey wrote: >> Is there an option to fall back to the older v.1.2.8 implementation >> if necessary ? It would certainly alleviate issues for any >> applications that might run into issues with the latest and greatest >> zlib libraries. JDK-8133206 would be one such example. > Just at build time so - we introduce a runtime dependency on the underlying zlib libraries on the OS by default. I would be very concerned with this approach. We've run JDK 6 for 10+ years with zlib v1.1.3. It was consistent and reliable for the most part. When we moved JDK7/8 to zlib v1.2.5, we encountered an inflation issue[1]. When JDK was upgraded to zlib v1.2.8, we received reports of performance/OOM issues [2]. > but if the zlib on the platform is broken then it impacts tools and > probably lots of things. I would assume the OS would patch such issues > quickly. In the case of JDK-8133206 then was the issue addressed in > the libzip wrapper or in the zlib code? I thought it was the former. The code change is proposed in the libzip wrapper but the issue was triggered by the zlib library update. > > On a fallback or some way to configure at launch time then Sandhya > Viswanathan (Intel) has a proposal here last year. I think we mostly > agreed on that thread that switching the build to use the system zlib > by default would be better. I'm all for allowing one to introduce a new version of zlib to their JDK at runtime. I just don't think it's in the interests of enterprises and stability to introduce a dependency to the JDK on the underlying OS zlib libraries. Could we at least consider a runtime property to allow linking to the (currently bundled) zlib v.1.2.8 port in case issues arise ? regards, Sean. [1] https://bugs.openjdk.java.net/browse/JDK-8044725 [2] https://bugs.openjdk.java.net/browse/JDK-8133206 From chris.hegarty at oracle.com Wed Feb 10 14:07:51 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 10 Feb 2016 14:07:51 +0000 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <56BB41CD.2050801@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> <56BB41CD.2050801@oracle.com> Message-ID: <56BB4437.4000205@oracle.com> On 10/02/16 13:57, Aleksey Shipilev wrote: > .... > Um (Integer.MAX_VALUE + 1L) is still a positive long, you probably want: > > int ilen = (int)utflen; > if (ilen < 0) > return ""; > > int initialCapacity = Math.min(ilen, 16384); > > ...or: > > if (utflen < 0 || utflen > Integer.MAX_INT) > return ""; > > int initialCapacity = Math.min((int)utflen, 16384); Thanks Aleksey, your proposal is better. So the complete change is: diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java @@ -3144,7 +3144,12 @@ * utflen bytes. */ private String readUTFBody(long utflen) throws IOException { - StringBuilder sbuf = new StringBuilder(); + if (utflen < 0 || utflen > Integer.MAX_VALUE) + return ""; + + // a reasonable initial capacity based on the UTF length + int initialCapacity = Math.min((int)utflen, 16384); + StringBuilder sbuf = new StringBuilder(initialCapacity); if (!blkmode) { end = pos = 0; } -Chris. From paul.sandoz at oracle.com Wed Feb 10 14:19:50 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Feb 2016 15:19:50 +0100 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> References: <56BABEB8.2050505@oracle.com> <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> Message-ID: <737C9779-7970-4F46-934D-D129B7C68F6C@oracle.com> > On 10 Feb 2016, at 14:16, Chris Vest wrote: > > It?d be a shame to loose the no-zeroing optimisation, so if that requires a constructor, then lets find a constructor. > > To recap, I have possibly multiple buffers and I only want to use parts of each. So with the composed buffer I?d have to first extract view buffers and then compose those, so I end up with 3 levels of buffers. Not ideal. A value type as a tuple of a buffer reference, an offset and a length would be better. We could make an array of those and pass that in. It could also find uses elsewhere in JDK where the buffer+offset+length pattern occurs. The array, a long with an encoding, could be passed up front to a String constructor (or a static method ? I don?t care) and my use case would be equally satisfied. > Perhaps not necessarily 3, since ByteBuffer.slice does not wrap (IIRC wider views do, but that is an implementation detail to share code for heap and direct AFAICT [*]). An absolute slice(int pos, int limit) method might be useful. IMO the more we can simplify the consumer the better with regards to dealing with a sequence of regions. > If I?ve understood correctly, there?s a plan (or at least a great desire) to turn Optional into a value type once they become available. Yes, there is certainly a strong desire. The risk of course is we have no way of enforcing it be used under constraints of being value-based (or as if a value type). > These ?buffer slice? objects could follow a similar path, and thus be made available independently of value types. Even if they can?t, the overhead they add would be small, I imagine. > Project Panama will most likely require some form of memory region type, on or off-heap, and that would certainly be a value type (or value-based in the interim). So something may emerge, just not in the short term. Thus i think it would be premature to introduce something specific in Java 9. > I?m slightly more worried about their array and what it implies, though. This API requires of me that I?m able to present all of the relevant buffers up front. In my case, these buffers are shared between multiple threads, so they are guarded by locks. I must take all the locks for all the buffers and hold them for the duration of the copy. _Maybe_ I have to do that anyway, but if there?s a way for me to only lock one buffer at a time, I?d like to be able to exploit this. The builder approach holds this door open for me. > That?s true. Interesting requirements, thanks for sharing. Paul. [*] which i suspect could be made more efficient if we unified heap/direct access From aleksey.shipilev at oracle.com Wed Feb 10 14:25:37 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 10 Feb 2016 17:25:37 +0300 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <56BB4437.4000205@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> <56BB41CD.2050801@oracle.com> <56BB4437.4000205@oracle.com> Message-ID: <56BB4861.8000206@oracle.com> On 10.02.2016 17:07, Chris Hegarty wrote: > Thanks Aleksey, your proposal is better. So the complete change is: > > diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java > b/src/java.base/share/classes/java/io/ObjectInputStream.java > --- a/src/java.base/share/classes/java/io/ObjectInputStream.java > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java > @@ -3144,7 +3144,12 @@ > * utflen bytes. > */ > private String readUTFBody(long utflen) throws IOException { > - StringBuilder sbuf = new StringBuilder(); > + if (utflen < 0 || utflen > Integer.MAX_VALUE) > + return ""; > + > + // a reasonable initial capacity based on the UTF length > + int initialCapacity = Math.min((int)utflen, 16384); > + StringBuilder sbuf = new StringBuilder(initialCapacity); > if (!blkmode) { > end = pos = 0; > } +1 -Aleksey From Alan.Bateman at oracle.com Wed Feb 10 14:29:06 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 10 Feb 2016 14:29:06 +0000 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56BB41E6.3060102@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> Message-ID: <56BB4932.7060105@oracle.com> On 10/02/2016 13:57, Se?n Coffey wrote: > I'm all for allowing one to introduce a new version of zlib to their > JDK at runtime. I just don't think it's in the interests of > enterprises and stability to introduce a dependency to the JDK on the > underlying OS zlib libraries. Could we at least consider a runtime > property to allow linking to the (currently bundled) zlib v.1.2.8 port > in case issues arise ? Don't the LD_* environment variables serve this need already? Once the JDK is using the system zlib then this is the simplest way to get it to use a different libz library at runtime. -Alan From yasuenag at gmail.com Wed Feb 10 14:34:06 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 10 Feb 2016 23:34:06 +0900 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> <568D6553.4090108@oracle.com> Message-ID: <56BB4A5E.60801@gmail.com> Hi Sherman, I've refactored a patch for this enhancement: http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.04/ 1. I changed ZipCryption and implementation class to package private. 2. Encryption / Decryption key is allowed passphrase string. 3. I added passphrase and validation methods to ZipEntry. I would like to hear your comment. Thanks, Yasumasa On 2016/02/01 18:23, KUBOTA Yuji wrote: > Hi Sherman and all, > > Could you please let know your thought and the past case about AES? > > Thanks, > Yuji > > > 2016-01-08 0:01 GMT+09:00 KUBOTA Yuji : >> Hi Sherman, >> >> Thank you for sharing! >> >> 2016-01-07 4:04 GMT+09:00 Xueming Shen : >>> The reason that I'm not convinced that we really need a public interface of >>> ZipCryption here >>> is I don't know how useful/helpful/likely it would be going forward that >>> someone might really >>> use this interface to implement other encryption(s), especially the pkware >>> proprietary one, >>> I doubt it might be not that straightforward. >> >> In this proposal, we aim to support "traditional" because most people need it >> in secure environment. BTW, could you please share the reason why you did >> not support WinZip AES? Do you have a plan to support in the future? >> >> If you can share the reason, we want to decide the way of implementation with >> consideration for your information. I think we can implement by two >> way as below. >> >> 1. Implementing by reference to >> http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java >> This is good simply API. If we need to implement other encryption(s), >> try to refactor it. >> >> 2. Implementing with a package private interface of ZipCryption for next step. >> This has two problems as your advice. >> >> We agree with that the "encryption" and "compression" should be >> separated logically. >> However, current implementation compress the encrypted data, and buffering it. >> It is too tightly-coupled, so we need refactoring to separate the >> managing buffer >> processing and the stream processing of InflaterInputStream / >> DeflaterOutputStream. >> >> About "push back the bytes belong to next entry", we think >> InflaterInputStream.originBuf >> of our PoC do not required the needed info. Do this implements have problem? >> >> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/src/java.base/share/classes/java/util/zip/InflaterInputStream.java.cdiff.html >> >> Thanks, >> Yuji >> >>> In fact I did have a draft implementation that supports WinZip AES about 5-6 >>> years ago :-) >>> (which also supports compression methods bzip and lzma, btw) Here is the >>> top class, It appears >>> a general interface might not be that helpful and it might be ideal to >>> simply implement it inside >>> the JDK, as what is proposed here, when it's really desired. >>> >>> http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java >>> >>> It is a ZipFile based implementation, so it does not have the headache that >>> ZipInputStream has, >>> such as to push back the bytes belong to next entry, since the loc might not >>> have the needed >>> info regarding the size/csize in stream mode. >>> >>> From abstract point of view. The "encryption" and "compression" are >>> different layers, it would >>> be ideal to have them in separate classes logically, instead of mixing the >>> encryption into >>> compression. Sure, it might be convenient and probably may have better >>> performance to mix >>> them in certain use scenario, but the "encryption" should never appear in >>> the public interface >>> of those compression classes. Package private interface should be fine, if >>> have to. >>> >>> -Sherman >>> >>> >>>> >>>> 2016-01-06 7:10 GMT+09:00 Xueming Shen : >>>>> >>>>> it appears that instead of adding "password" specific method to these >>>>> classes directly, it might be more appropriate to extend the ZipEntry >>>>> class >>>>> for such "password" functionality. For example, with a pair of new >>>>> methods >>>>> >>>>> boolean ZipEntry.isTraditionalEncryption(). >>>>> void ZipEntry.setTraditionalEncryption(String password); >>>> >>>> Thanks advice, I agree. We should re-design the API to extend the >>>> ZipEntry class. >>>> >>>>> The encryption support should/can be added naturally/smoothly with >>>>> ZipFile.getInputStream(e), ZipInputstream and >>>>> ZipOutputStream.putNextEntry(e), >>>>> with no extra new method in these two classes. The implementation checks >>>>> the flag (bit0, no bit 6) first and then verifies the password, as an >>>>> implementation details. >>>> >>>> Agree. For this proposal, we aim to support only traditional >>>> encryption. So I think we should also check bit 6. >>>> >>>>> For ZipFile and ZipInputStream, we can add note to the api doc to force >>>>> the >>>>> invoker to check if the returned ZipEntry indicates it's an encrypted >>>>> entry. >>>>> If yes, it must to set the appropriate password to the returned ZipEntry >>>>> via >>>>> ZipEntry.setTraditionalEncryption(password); before reading any byte from >>>>> the input stream. >>>> >>>> Yes, we have to add note the flow of codes to the JavaDoc. >>>> >>>>> Again, we should not have any "encryption" related public field/method in >>>>> DeflaterOutputStream/InflaterInputStream. Ideally these two classes >>>>> really >>>>> should not be aware of "encryption" at all. >>>> >>>> Agree, but I think we might be faced technical difficulty about a >>>> processing between zlib and the internal buffer of InflaterInputStream >>>> / DeflaterOutputStream. Please give us time to implement. >>>> >>>>> -Sherman >>>> >>>> Thanks, >>>> Yuji >>>> >>>> >>>>> On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: >>>>>> >>>>>> Hi Sherman and all, >>>>>> >>>>>> Happy new year to everyone! >>>>>> >>>>>> Please let know your feedback about this proposal. :-) >>>>>> >>>>>> Thanks, >>>>>> Yuji >>>>>> >>>>>> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >>>>>>> >>>>>>> Hi Sherman, >>>>>>> >>>>>>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>>>>>>> >>>>>>>> It is no longer necessary to touch the native code (zip_util.c/h) >>>>>>>> after >>>>>>>> the >>>>>>>> native ZipFile implementation has been moved up to the java level. >>>>>>>> Those >>>>>>>> native code are for vm access only now, which I dont think care about >>>>>>>> the >>>>>>>> password support at all. >>>>>>> >>>>>>> Thanks for your information. We do not take care the native. >>>>>>> >>>>>>> I discussed with Yasumasa, and our thought is as below. >>>>>>> >>>>>>>> (1) what's the benefit of exposing the public interface ZipCryption? >>>>>>>> the >>>>>>>> real >>>>>>>> question is whether or not this interface is good enough for other >>>>>>>> encryption >>>>>>>> implementation to plugin their implementation to support the >>>>>>>> ZipFile/Input/ >>>>>>>> OutputStream to their encryption spec. >>>>>>> >>>>>>> We aimed that the public interface ZipCryption supports the >>>>>>> extensibillity for other encrypt engine. The JDK core libs developers >>>>>>> have to implementation ZipyCryption only. If not provide, the JDK >>>>>>> developers must implement ZipStream/Entry by JDK API to design the >>>>>>> data structure of entry. >>>>>>> If you want to use binary key data such as PKI, you can implement new >>>>>>> encrypt/decrypt engine by ZipCryption interface. >>>>>>> So we think we should provide this interface to be clearly how to >>>>>>> implement a new engine, e.g., cipher algorithm, cipher strength and >>>>>>> converting the header, etc. >>>>>>> >>>>>>>> (2) it seems like it might be possible to hide most of the >>>>>>>> implementation >>>>>>>> and only expose the "String password" (instead of the ZipCryption) as >>>>>>>> the >>>>>>>> public interface to support the "traditional" encryption. This depends >>>>>>>> on the >>>>>>>> result of (1) though. >>>>>>> >>>>>>> Thanks for your clues. We think the string password at first. However, >>>>>>> we should also create a new binary interface given we support PKI in >>>>>>> the future. >>>>>>> >>>>>>>> (3) I'm concerned of pushing ZipCryption into >>>>>>>> InflaterInputStream/DeflaterOutputStream. >>>>>>>> It might be worth considering to replace the ZipCryption >>>>>>>> implementation >>>>>>>> with >>>>>>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>>>>>> use >>>>>>>> the FilterOutputStream for the ZipOutputStream and the >>>>>>>> FilterInputStream >>>>>>>> for the >>>>>>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue >>>>>>>> ... >>>>>>> >>>>>>> Thanks for your clues, too. Honestly speaking, we think the current >>>>>>> zip implementation may break the data when used PushbackInputStream >>>>>>> for the following reasons. >>>>>>> >>>>>>> * PushbackInputStream uses an unique internal buffer for re-read >>>>>>> operation. >>>>>>> * But, InflaterInputStream provide date to Inflater per reads and >>>>>>> buffer by JNI (zlib). >>>>>>> * So we think PushbackInputStream is poor compatibility with >>>>>>> InflaterInputStream. >>>>>>> >>>>>>> We generally use InputStream through ZipEntry#getInputStream(). We do >>>>>>> not touch FileInputStream for reading ZIP data. If we call unread() >>>>>>> when we use PushbackInputStream as reading ZIP archive, we guess that >>>>>>> it will break the reading data. >>>>>>> So, our approach do not affect the PushbackInputStream. >>>>>>> What do you think about this? >>>>>>> >>>>>>>> (4) It seems the ZipOutputStream only supports the "stream based" >>>>>>>> password, while >>>>>>>> the ZipInputStream supports the "entry based" password. Do we really >>>>>>>> need >>>>>>>> "entry based" support here? >>>>>>> >>>>>>> As your suggestion, we should support "entry based". We will start to >>>>>>> implement "entry based" after this discussion is closed. >>>>>>> >>>>>>> Thanks, >>>>>>> Yuji >>>>>>> >>>>>>>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>>>>>>> >>>>>>>>> Hi Jason, >>>>>>>>> >>>>>>>>> Thank you for your comment. >>>>>>>>> I've fixed it in new webrev: >>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Yasumasa >>>>>>>>> >>>>>>>>> >>>>>>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>>>>>>> >>>>>>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is >>>>>>>>>> in >>>>>>>>>> conflict with line 350 and 348. >>>>>>>>>> >>>>>>>>>> ________________________________________ >>>>>>>>>> From: core-libs-dev on >>>>>>>>>> behalf >>>>>>>>>> of >>>>>>>>>> Yasumasa Suenaga >>>>>>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>>>>>> protection to Zip entries >>>>>>>>>> >>>>>>>>>> Hi Sergey, >>>>>>>>>> >>>>>>>>>> Thank you for your comment. >>>>>>>>>> >>>>>>>>>> I added that description in new webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>>>>>>> >>>>>>>>>>> Should the new methods describe how they will work in case of null >>>>>>>>>>> params? >>>>>>>>>>> >>>>>>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>>>>>>> >>>>>>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>>>>>> >>>>>>>>>>>> Could you review it? >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> >>>>>>>>>>>> Yasumasa >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> Hi Sherman, >>>>>>>>>>>>> >>>>>>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>>>>>> jdk9/dev >>>>>>>>>>>>> repos. >>>>>>>>>>>>> >>>>>>>>>>>>> Do you have any comments about current webrev? >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>> >>>>>>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Sherman, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks for your quick response :) >>>>>>>>>>>>>> >>>>>>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>>>>>> below >>>>>>>>>>>>>> reasons. >>>>>>>>>>>>>> >>>>>>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>>>>>> like a >>>>>>>>>>>>>> temporary file. >>>>>>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>>>>>> best >>>>>>>>>>>>>> for >>>>>>>>>>>>>> openjdk. >>>>>>>>>>>>>> * PKWare claims that they have patents about the >>>>>>>>>>>>>> "stronger" >>>>>>>>>>>>>> on >>>>>>>>>>>>>> Zip[1]. >>>>>>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>>>>>> "stronger" [2][3]. >>>>>>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>>>>>> interface >>>>>>>>>>>>>> to >>>>>>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>>>>>> easily >>>>>>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [1] >>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption >>>>>>>>>>>>>> Specification) >>>>>>>>>>>>>> >>>>>>>>>>>>>> [2] >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>>>>>> >>>>>>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming >>>>>>>>>>>>>> Shen: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Yuji, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>>>>>> bring >>>>>>>>>>>>>>> in the >>>>>>>>>>>>>>> security guy >>>>>>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>>>>>> in >>>>>>>>>>>>>>> the >>>>>>>>>>>>>>> "traditional PKWare >>>>>>>>>>>>>>> decryption", which is, based on the wiki, "known to be >>>>>>>>>>>>>>> seriously >>>>>>>>>>>>>>> flawed, and >>>>>>>>>>>>>>> in particular >>>>>>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>>>>>> support >>>>>>>>>>>>>>> "stronger" encryption >>>>>>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Sherman >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>>>>>> before >>>>>>>>>>>>>>>>> get >>>>>>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>>>>>> However, >>>>>>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we >>>>>>>>>>>>>>>>> must >>>>>>>>>>>>>>>>> use >>>>>>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has >>>>>>>>>>>>>>>>> been >>>>>>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>>>>>> zlib >>>>>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Test code is as below. This code will let you know >>>>>>>>>>>>>>>>> how >>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>> PoC >>>>>>>>>>>>>>>>> works. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are >>>>>>>>>>>>>>>>> providing >>>>>>>>>>>>>>>>> many >>>>>>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>>>>>> implement to >>>>>>>>>>>>>>>>> handle password-protected zip file. I guess that this >>>>>>>>>>>>>>>>> proposal >>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> [1]: >>>>>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>> From yasuenag at gmail.com Wed Feb 10 14:36:44 2016 From: yasuenag at gmail.com (Yasumasa Suenaga) Date: Wed, 10 Feb 2016 23:36:44 +0900 Subject: [PING] PoC for JDK-4347142: Need method to set Password protection to Zip entries In-Reply-To: <56BB4A5E.60801@gmail.com> References: <565FB732.4090401@oracle.com> <566C11A7.1070309@gmail.com> <56716147.20100@gmail.com> <567164D8.8040201@oracle.com> <5671796C.1030406@gmail.com> <5672BC96.3080301@gmail.com> <56765A42.3060307@oracle.com> <568C3F46.3040801@oracle.com> <568D6553.4090108@oracle.com> <56BB4A5E.60801@gmail.com> Message-ID: <56BB4AFC.3010403@gmail.com> I've uploaded testcase here: http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.04/Test.java Yasumasa On 2016/02/10 23:34, Yasumasa Suenaga wrote: > Hi Sherman, > > I've refactored a patch for this enhancement: > > http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.04/ > > 1. I changed ZipCryption and implementation class to package private. > 2. Encryption / Decryption key is allowed passphrase string. > 3. I added passphrase and validation methods to ZipEntry. > > I would like to hear your comment. > > > Thanks, > > Yasumasa > > > On 2016/02/01 18:23, KUBOTA Yuji wrote: >> Hi Sherman and all, >> >> Could you please let know your thought and the past case about AES? >> >> Thanks, >> Yuji >> >> >> 2016-01-08 0:01 GMT+09:00 KUBOTA Yuji : >>> Hi Sherman, >>> >>> Thank you for sharing! >>> >>> 2016-01-07 4:04 GMT+09:00 Xueming Shen : >>>> The reason that I'm not convinced that we really need a public interface of >>>> ZipCryption here >>>> is I don't know how useful/helpful/likely it would be going forward that >>>> someone might really >>>> use this interface to implement other encryption(s), especially the pkware >>>> proprietary one, >>>> I doubt it might be not that straightforward. >>> >>> In this proposal, we aim to support "traditional" because most people need it >>> in secure environment. BTW, could you please share the reason why you did >>> not support WinZip AES? Do you have a plan to support in the future? >>> >>> If you can share the reason, we want to decide the way of implementation with >>> consideration for your information. I think we can implement by two >>> way as below. >>> >>> 1. Implementing by reference to >>> http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java >>> This is good simply API. If we need to implement other encryption(s), >>> try to refactor it. >>> >>> 2. Implementing with a package private interface of ZipCryption for next step. >>> This has two problems as your advice. >>> >>> We agree with that the "encryption" and "compression" should be >>> separated logically. >>> However, current implementation compress the encrypted data, and buffering it. >>> It is too tightly-coupled, so we need refactoring to separate the >>> managing buffer >>> processing and the stream processing of InflaterInputStream / >>> DeflaterOutputStream. >>> >>> About "push back the bytes belong to next entry", we think >>> InflaterInputStream.originBuf >>> of our PoC do not required the needed info. Do this implements have problem? >>> >>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/src/java.base/share/classes/java/util/zip/InflaterInputStream.java.cdiff.html >>> >>> Thanks, >>> Yuji >>> >>>> In fact I did have a draft implementation that supports WinZip AES about 5-6 >>>> years ago :-) >>>> (which also supports compression methods bzip and lzma, btw) Here is the >>>> top class, It appears >>>> a general interface might not be that helpful and it might be ideal to >>>> simply implement it inside >>>> the JDK, as what is proposed here, when it's really desired. >>>> >>>> http://cr.openjdk.java.net/~sherman/zipmisc/ZipFile.java >>>> >>>> It is a ZipFile based implementation, so it does not have the headache that >>>> ZipInputStream has, >>>> such as to push back the bytes belong to next entry, since the loc might not >>>> have the needed >>>> info regarding the size/csize in stream mode. >>>> >>>> From abstract point of view. The "encryption" and "compression" are >>>> different layers, it would >>>> be ideal to have them in separate classes logically, instead of mixing the >>>> encryption into >>>> compression. Sure, it might be convenient and probably may have better >>>> performance to mix >>>> them in certain use scenario, but the "encryption" should never appear in >>>> the public interface >>>> of those compression classes. Package private interface should be fine, if >>>> have to. >>>> >>>> -Sherman >>>> >>>> >>>>> >>>>> 2016-01-06 7:10 GMT+09:00 Xueming Shen : >>>>>> >>>>>> it appears that instead of adding "password" specific method to these >>>>>> classes directly, it might be more appropriate to extend the ZipEntry >>>>>> class >>>>>> for such "password" functionality. For example, with a pair of new >>>>>> methods >>>>>> >>>>>> boolean ZipEntry.isTraditionalEncryption(). >>>>>> void ZipEntry.setTraditionalEncryption(String password); >>>>> >>>>> Thanks advice, I agree. We should re-design the API to extend the >>>>> ZipEntry class. >>>>> >>>>>> The encryption support should/can be added naturally/smoothly with >>>>>> ZipFile.getInputStream(e), ZipInputstream and >>>>>> ZipOutputStream.putNextEntry(e), >>>>>> with no extra new method in these two classes. The implementation checks >>>>>> the flag (bit0, no bit 6) first and then verifies the password, as an >>>>>> implementation details. >>>>> >>>>> Agree. For this proposal, we aim to support only traditional >>>>> encryption. So I think we should also check bit 6. >>>>> >>>>>> For ZipFile and ZipInputStream, we can add note to the api doc to force >>>>>> the >>>>>> invoker to check if the returned ZipEntry indicates it's an encrypted >>>>>> entry. >>>>>> If yes, it must to set the appropriate password to the returned ZipEntry >>>>>> via >>>>>> ZipEntry.setTraditionalEncryption(password); before reading any byte from >>>>>> the input stream. >>>>> >>>>> Yes, we have to add note the flow of codes to the JavaDoc. >>>>> >>>>>> Again, we should not have any "encryption" related public field/method in >>>>>> DeflaterOutputStream/InflaterInputStream. Ideally these two classes >>>>>> really >>>>>> should not be aware of "encryption" at all. >>>>> >>>>> Agree, but I think we might be faced technical difficulty about a >>>>> processing between zlib and the internal buffer of InflaterInputStream >>>>> / DeflaterOutputStream. Please give us time to implement. >>>>> >>>>>> -Sherman >>>>> >>>>> Thanks, >>>>> Yuji >>>>> >>>>> >>>>>> On 01/04/2016 06:26 AM, KUBOTA Yuji wrote: >>>>>>> >>>>>>> Hi Sherman and all, >>>>>>> >>>>>>> Happy new year to everyone! >>>>>>> >>>>>>> Please let know your feedback about this proposal. :-) >>>>>>> >>>>>>> Thanks, >>>>>>> Yuji >>>>>>> >>>>>>> 2015-12-21 22:38 GMT+09:00 KUBOTA Yuji: >>>>>>>> >>>>>>>> Hi Sherman, >>>>>>>> >>>>>>>> 2015-12-20 16:35 GMT+09:00 Xueming Shen: >>>>>>>>> >>>>>>>>> It is no longer necessary to touch the native code (zip_util.c/h) >>>>>>>>> after >>>>>>>>> the >>>>>>>>> native ZipFile implementation has been moved up to the java level. >>>>>>>>> Those >>>>>>>>> native code are for vm access only now, which I dont think care about >>>>>>>>> the >>>>>>>>> password support at all. >>>>>>>> >>>>>>>> Thanks for your information. We do not take care the native. >>>>>>>> >>>>>>>> I discussed with Yasumasa, and our thought is as below. >>>>>>>> >>>>>>>>> (1) what's the benefit of exposing the public interface ZipCryption? >>>>>>>>> the >>>>>>>>> real >>>>>>>>> question is whether or not this interface is good enough for other >>>>>>>>> encryption >>>>>>>>> implementation to plugin their implementation to support the >>>>>>>>> ZipFile/Input/ >>>>>>>>> OutputStream to their encryption spec. >>>>>>>> >>>>>>>> We aimed that the public interface ZipCryption supports the >>>>>>>> extensibillity for other encrypt engine. The JDK core libs developers >>>>>>>> have to implementation ZipyCryption only. If not provide, the JDK >>>>>>>> developers must implement ZipStream/Entry by JDK API to design the >>>>>>>> data structure of entry. >>>>>>>> If you want to use binary key data such as PKI, you can implement new >>>>>>>> encrypt/decrypt engine by ZipCryption interface. >>>>>>>> So we think we should provide this interface to be clearly how to >>>>>>>> implement a new engine, e.g., cipher algorithm, cipher strength and >>>>>>>> converting the header, etc. >>>>>>>> >>>>>>>>> (2) it seems like it might be possible to hide most of the >>>>>>>>> implementation >>>>>>>>> and only expose the "String password" (instead of the ZipCryption) as >>>>>>>>> the >>>>>>>>> public interface to support the "traditional" encryption. This depends >>>>>>>>> on the >>>>>>>>> result of (1) though. >>>>>>>> >>>>>>>> Thanks for your clues. We think the string password at first. However, >>>>>>>> we should also create a new binary interface given we support PKI in >>>>>>>> the future. >>>>>>>> >>>>>>>>> (3) I'm concerned of pushing ZipCryption into >>>>>>>>> InflaterInputStream/DeflaterOutputStream. >>>>>>>>> It might be worth considering to replace the ZipCryption >>>>>>>>> implementation >>>>>>>>> with >>>>>>>>> a pair of FilterOutput/InputStream. It would be easy and reasonable to >>>>>>>>> use >>>>>>>>> the FilterOutputStream for the ZipOutputStream and the >>>>>>>>> FilterInputStream >>>>>>>>> for the >>>>>>>>> ZipFile. The PushbackInputStream in ZipInputStream might be an issue >>>>>>>>> ... >>>>>>>> >>>>>>>> Thanks for your clues, too. Honestly speaking, we think the current >>>>>>>> zip implementation may break the data when used PushbackInputStream >>>>>>>> for the following reasons. >>>>>>>> >>>>>>>> * PushbackInputStream uses an unique internal buffer for re-read >>>>>>>> operation. >>>>>>>> * But, InflaterInputStream provide date to Inflater per reads and >>>>>>>> buffer by JNI (zlib). >>>>>>>> * So we think PushbackInputStream is poor compatibility with >>>>>>>> InflaterInputStream. >>>>>>>> >>>>>>>> We generally use InputStream through ZipEntry#getInputStream(). We do >>>>>>>> not touch FileInputStream for reading ZIP data. If we call unread() >>>>>>>> when we use PushbackInputStream as reading ZIP archive, we guess that >>>>>>>> it will break the reading data. >>>>>>>> So, our approach do not affect the PushbackInputStream. >>>>>>>> What do you think about this? >>>>>>>> >>>>>>>>> (4) It seems the ZipOutputStream only supports the "stream based" >>>>>>>>> password, while >>>>>>>>> the ZipInputStream supports the "entry based" password. Do we really >>>>>>>>> need >>>>>>>>> "entry based" support here? >>>>>>>> >>>>>>>> As your suggestion, we should support "entry based". We will start to >>>>>>>> implement "entry based" after this discussion is closed. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Yuji >>>>>>>> >>>>>>>>> On 12/17/15, 9:45 PM, Yasumasa Suenaga wrote: >>>>>>>>>> >>>>>>>>>> Hi Jason, >>>>>>>>>> >>>>>>>>>> Thank you for your comment. >>>>>>>>>> I've fixed it in new webrev: >>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.03/ >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> >>>>>>>>>> Yasumasa >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 2015/12/17 0:33, Jason Mehrens wrote: >>>>>>>>>>> >>>>>>>>>>> The null check of 'entry' at line 351 of ZipFile.getInputStream is >>>>>>>>>>> in >>>>>>>>>>> conflict with line 350 and 348. >>>>>>>>>>> >>>>>>>>>>> ________________________________________ >>>>>>>>>>> From: core-libs-dev on >>>>>>>>>>> behalf >>>>>>>>>>> of >>>>>>>>>>> Yasumasa Suenaga >>>>>>>>>>> Sent: Wednesday, December 16, 2015 8:47 AM >>>>>>>>>>> To: Sergey Bylokhov; Xueming Shen >>>>>>>>>>> Cc: core-libs-dev at openjdk.java.net >>>>>>>>>>> Subject: Re: [PING] PoC for JDK-4347142: Need method to set Password >>>>>>>>>>> protection to Zip entries >>>>>>>>>>> >>>>>>>>>>> Hi Sergey, >>>>>>>>>>> >>>>>>>>>>> Thank you for your comment. >>>>>>>>>>> >>>>>>>>>>> I added that description in new webrev: >>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.02/ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Thanks, >>>>>>>>>>> >>>>>>>>>>> Yasumasa >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 2015/12/16 22:19, Sergey Bylokhov wrote: >>>>>>>>>>>> >>>>>>>>>>>> Should the new methods describe how they will work in case of null >>>>>>>>>>>> params? >>>>>>>>>>>> >>>>>>>>>>>> On 16/12/15 16:04, Yasumasa Suenaga wrote: >>>>>>>>>>>>> >>>>>>>>>>>>> I adapted this enhancement after JDK-8145260: >>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.01/ >>>>>>>>>>>>> >>>>>>>>>>>>> Could you review it? >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> >>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On 2015/12/12 21:23, Yasumasa Suenaga wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>> Hi Sherman, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Our proposal is affected by JDK-8142508. >>>>>>>>>>>>>> We have to change ZipFile.java and and ZipFile.c . >>>>>>>>>>>>>> Thus we will create a new webrev for current (after 8142508) >>>>>>>>>>>>>> jdk9/dev >>>>>>>>>>>>>> repos. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Do you have any comments about current webrev? >>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> If you have comments, we will fix them in new webrev. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> >>>>>>>>>>>>>> Yasumasa >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On 2015/12/03 16:51, KUBOTA Yuji wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hi Sherman, >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks for your quick response :) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I aimed to implement the "traditional" at this proposal by the >>>>>>>>>>>>>>> below >>>>>>>>>>>>>>> reasons. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> * We want to prepare API for encrypted zip files at first. >>>>>>>>>>>>>>> * Many people use the "traditional" in problem-free scope >>>>>>>>>>>>>>> like a >>>>>>>>>>>>>>> temporary file. >>>>>>>>>>>>>>> * We do not know which implementation of the "stronger" is >>>>>>>>>>>>>>> best >>>>>>>>>>>>>>> for >>>>>>>>>>>>>>> openjdk. >>>>>>>>>>>>>>> * PKWare claims that they have patents about the >>>>>>>>>>>>>>> "stronger" >>>>>>>>>>>>>>> on >>>>>>>>>>>>>>> Zip[1]. >>>>>>>>>>>>>>> * OTOH, WinZip have the alternative implementation of the >>>>>>>>>>>>>>> "stronger" [2][3]. >>>>>>>>>>>>>>> * Instead, we prepared the extensibility by ZipCryption >>>>>>>>>>>>>>> interface >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>> implement other encrypt engine, such as the AES based. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thus, I think this PoC should support the "traditional" only. >>>>>>>>>>>>>>> In the future, anyone who want to implement the "stronger" can >>>>>>>>>>>>>>> easily >>>>>>>>>>>>>>> add their code by virtue of this proposal. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [1] >>>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>>> (1.4 Permitted Use& 7.0 Strong Encryption >>>>>>>>>>>>>>> Specification) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [2] >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> https://en.wikipedia.org/wiki/Zip_(file_format)#Strong_encryption_controversy >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> [3] http://www.winzip.com/aes_info.htm >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2015-12-03 12:29 GMT+09:00 Xueming >>>>>>>>>>>>>>> Shen: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi Yuji, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I will take a look at your PoC. Might need some time and even >>>>>>>>>>>>>>>> bring >>>>>>>>>>>>>>>> in the >>>>>>>>>>>>>>>> security guy >>>>>>>>>>>>>>>> to evaluate the proposal. It seems like you are only interested >>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>> "traditional PKWare >>>>>>>>>>>>>>>> decryption", which is, based on the wiki, "known to be >>>>>>>>>>>>>>>> seriously >>>>>>>>>>>>>>>> flawed, and >>>>>>>>>>>>>>>> in particular >>>>>>>>>>>>>>>> is vulnerable to known-plaintext attacks":-) Any request to >>>>>>>>>>>>>>>> support >>>>>>>>>>>>>>>> "stronger" encryption >>>>>>>>>>>>>>>> mechanism, such as the AES based? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>> Sherman >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On 12/2/15 6:48 PM, KUBOTA Yuji wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> We need reviewer(s) for this PoC. >>>>>>>>>>>>>>>>> Could you please review this proposal and PoC ? >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2015-11-26 13:22 GMT+09:00 KUBOTA Yuji: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi all, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> * Sorry for my mistake. I re-post this mail because I sent >>>>>>>>>>>>>>>>>> before >>>>>>>>>>>>>>>>>> get >>>>>>>>>>>>>>>>>> a response of subscription confirmation of core-libs-dev. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Our customers have to handle password-protected zip files. >>>>>>>>>>>>>>>>>> However, >>>>>>>>>>>>>>>>>> Java SE does not provide the APIs to handle it yet, so we >>>>>>>>>>>>>>>>>> must >>>>>>>>>>>>>>>>>> use >>>>>>>>>>>>>>>>>> third party library so far. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Recently, we found JDK-4347142: "Need method to set Password >>>>>>>>>>>>>>>>>> protection to Zip entries", and we tried to implement it. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> The current zlib in JDK is completely unaffected by this >>>>>>>>>>>>>>>>>> proposal. >>>>>>>>>>>>>>>>>> The >>>>>>>>>>>>>>>>>> traditional zip encryption encrypts a data after it is has >>>>>>>>>>>>>>>>>> been >>>>>>>>>>>>>>>>>> compressed by zlib.[1] So we do NOT need to change existing >>>>>>>>>>>>>>>>>> zlib >>>>>>>>>>>>>>>>>> implementation. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> We've created PoC and uploaded it as webrev: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/ >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Test code is as below. This code will let you know >>>>>>>>>>>>>>>>>> how >>>>>>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>>> PoC >>>>>>>>>>>>>>>>>> works. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> http://cr.openjdk.java.net/~ysuenaga/JDK-4347142/webrev.00/Test.java >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> In NTT, a Japanese telecommunications company. We are >>>>>>>>>>>>>>>>>> providing >>>>>>>>>>>>>>>>>> many >>>>>>>>>>>>>>>>>> enterprise systems to customers. Some of them, we need to >>>>>>>>>>>>>>>>>> implement to >>>>>>>>>>>>>>>>>> handle password-protected zip file. I guess that this >>>>>>>>>>>>>>>>>> proposal >>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>> desired for many developers and users. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm working together with Yasumasa Suenaga, jdk9 committer >>>>>>>>>>>>>>>>>> (ysuenaga). >>>>>>>>>>>>>>>>>> We want to implement it if this proposal accepted. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> [1]: >>>>>>>>>>>>>>>>>> https://pkware.cachefly.net/webdocs/APPNOTE/APPNOTE-6.3.3.TXT >>>>>>>>>>>>>>>>>> (6.0 Traditional PKWARE Encryption) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>> Yuji >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>> From sean.coffey at oracle.com Wed Feb 10 14:41:40 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 10 Feb 2016 14:41:40 +0000 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56BB4932.7060105@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> <56BB4932.7060105@oracle.com> Message-ID: <56BB4C24.7090607@oracle.com> On 10/02/16 14:29, Alan Bateman wrote: > > On 10/02/2016 13:57, Se?n Coffey wrote: >> I'm all for allowing one to introduce a new version of zlib to their >> JDK at runtime. I just don't think it's in the interests of >> enterprises and stability to introduce a dependency to the JDK on the >> underlying OS zlib libraries. Could we at least consider a runtime >> property to allow linking to the (currently bundled) zlib v.1.2.8 >> port in case issues arise ? > Don't the LD_* environment variables serve this need already? Once the > JDK is using the system zlib then this is the simplest way to get it > to use a different libz library at runtime. No - I don't see that as a solution. You've still made the default JDK config become dependent on OS environment for all libzip operations. I don't think we even capture the zlib version that the JDK would be operating with in any diagnostics. An application wanting a tried/tested and stable libzip version has extra work to do now. Letting the default be system dependent has just increased risk for QA teams also. A system property just makes this all go away. In fact - I would say that for JDK9, the default should be the JDK bundled libzip library. For those looking for libzip experimenting and performance benefits, they could take the system property approach. regards, Sean. > > -Alan On 08/02/16 09:55, Alan Bateman wrote: > > On 08/02/2016 10:42, Se?n Coffey wrote: >> Is there an option to fall back to the older v.1.2.8 implementation >> if necessary ? It would certainly alleviate issues for any >> applications that might run into issues with the latest and greatest >> zlib libraries. JDK-8133206 would be one such example. > Just at build time so - we introduce a runtime dependency on the underlying zlib libraries on the OS by default. I would be very concerned with this approach. We've run JDK 6 for 10+ years with zlib v1.1.3. It was consistent and reliable for the most part. When we moved JDK7/8 to zlib v1.2.5, we encountered an inflation issue[1]. When JDK was upgraded to zlib v1.2.8, we received reports of performance/OOM issues [2]. > but if the zlib on the platform is broken then it impacts tools and > probably lots of things. I would assume the OS would patch such issues > quickly. In the case of JDK-8133206 then was the issue addressed in > the libzip wrapper or in the zlib code? I thought it was the former. The code change is proposed in the libzip wrapper but the issue was triggered by the zlib library update. > > On a fallback or some way to configure at launch time then Sandhya > Viswanathan (Intel) has a proposal here last year. I think we mostly > agreed on that thread that switching the build to use the system zlib > by default would be better. I'm all for allowing one to introduce a new version of zlib to their JDK at runtime. I just don't think it's in the interests of enterprises and stability to introduce a dependency to the JDK on the underlying OS zlib libraries. Could we at least consider a runtime property to allow linking to the (currently bundled) zlib v.1.2.8 port in case issues arise ? regards, Sean. [1] https://bugs.openjdk.java.net/browse/JDK-8044725 [2] https://bugs.openjdk.java.net/browse/JDK-8133206 From peter.levart at gmail.com Wed Feb 10 15:50:13 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 10 Feb 2016 16:50:13 +0100 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> References: <56BABEB8.2050505@oracle.com> <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> Message-ID: <56BB5C35.8030006@gmail.com> Hi Chris and others, On 02/10/2016 02:16 PM, Chris Vest wrote: > It?d be a shame to loose the no-zeroing optimisation, so if that requires a constructor, then lets find a constructor. > > To recap, I have possibly multiple buffers and I only want to use parts of each. So with the composed buffer I?d have to first extract view buffers and then compose those, so I end up with 3 levels of buffers. Not ideal. A value type as a tuple of a buffer reference, an offset and a length would be better. We could make an array of those and pass that in. It could also find uses elsewhere in JDK where the buffer+offset+length pattern occurs. The array, a long with an encoding, could be passed up front to a String constructor (or a static method ? I don?t care) and my use case would be equally satisfied. From your description the most suitable API seems to be a method on StringBuilder: public StringBuilder append(ByteBuffer bb, int offset, int length, Charset cs) The only problem with this approach is that even if you guess the correct capacity of StringBuilder, a defensive copy of the byte[] has to be performed in toString(), because in the absence of synchronization, a data race could allow the String's byte[] to be modified after it is constructed. But there is a trick that doesn't need synchronization and prevents data races: a OneShotStringBuilder: http://cr.openjdk.java.net/~plevart/misc/OneShotStringBuilder/OneShotStringBuilder.java We now have optimized strategies for string concatenation (indyfied String concat), but for programmatic building of Strings we are still bound by sub-optimal StringBuilder that does defensive copying at the end although the capacity is exactly estimated an the builder is abandoned afterwards. So perhaps if such OneShotStringBuilder has enough weight on its own to be added to the standard API, it could also be equipped with ByteBuffer appending&decoding operation(s)... As it's use is constrained to single thread and for constructing a single String, there's a big chance that JIT could optimize and eliminate its allocation. In which case such builder is comparable to a String constructor or factory method, although much more flexible in expression. What do you think? Regards, Peter > > If I?ve understood correctly, there?s a plan (or at least a great desire) to turn Optional into a value type once they become available. These ?buffer slice? objects could follow a similar path, and thus be made available independently of value types. Even if they can?t, the overhead they add would be small, I imagine. > > I?m slightly more worried about their array and what it implies, though. This API requires of me that I?m able to present all of the relevant buffers up front. In my case, these buffers are shared between multiple threads, so they are guarded by locks. I must take all the locks for all the buffers and hold them for the duration of the copy. _Maybe_ I have to do that anyway, but if there?s a way for me to only lock one buffer at a time, I?d like to be able to exploit this. The builder approach holds this door open for me. > > Cheers, > Chris > >> On 10 Feb 2016, at 09:54, Paul Sandoz wrote: >> >> Hi, >> >> A more functional approach would be to compose a sequence buffers into one view, perhaps read-only. Then there would be no need to accept arrays of buffers. That should work well for bulk operations. That?s a non-trivial but not very difficult amount of work, and possibly simplified if restricted to read-only views. >> >> Thus i think we should focus Richard?s work with: >> >> String(ByteBuffer src, String charset) >> >> and perhaps a sub-range variant, if perturbing the position/limit of an existing buffer and/or slicing is too problematic. >> >> ? >> >> Zeroing memory and possibly avoiding it can be tricky. Any such optimisations have to be carefully performed otherwise uninitialised regions might leak and be accessed, nefariously or otherwise. I imagine it?s easier to contain/control within a constructor than say a builder. >> >> Paul. >> >>> On 10 Feb 2016, at 05:38, Xueming Shen wrote: >>> >>> Hi Chris, >>> >>> I think basically you are asking a String constructor that takes a ByteBuffer. StringCoding >>> then can take advantage of the current CompactString design to optimize the decoding >>> operation by just a single byte[]/vectorized memory copy from the ByteBuffer to the String's >>> internal byte[], WHEN the charset is 8859-1. >>> >>> String(ByteBuffer src, String charset); >>> >>> Further we will need a "buffer gathering" style constructor >>> >>> String(ByteBuffer[] srcs, String charset); >>> (or more generally, String(ByteBuffer[] srcs, int off, int len, String charset) >>> >>> to create a String object from a sequence of ByteBuffers, if it's really desired. >>> >>> And then I would also assume it will also be desired to extend the current >>> CharsetDecoder/Encoder class as well to add a pair of the "gathering" style coding >>> methods >>> >>> CharBuffer CharsetDecoder.decode(ByteBuffer... ins); >>> ByteBuffer CharsetEncoder.encode(CharBuffer... ins); >>> >>> Though the implementation might have to deal with the tricky "splitting >>> byte/char" issue, in which part of the "byte/char sequence" is in the previous >>> buffer and the continuing byte/chars are in the next following buffer ... >>> >>> -Sherman >>> >>> >>> On 2/9/16 7:20 AM, Chris Vest wrote: >>>> Hi, >>>> >>>> Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string data in Latin1, which is one of the two special encodings for compact strings, we?d ideally like to produce the String object with just the two necessary object allocations and a single, vectorised memory copy. >>>> >>>> Our use case is that we are a database and we do our own file paging, effectively having file data in a large set of DirectByteBuffers. We have string data in our files in a number of different encodings, a popular one being Latin1. Occasionally these String values span multiple buffers. We often need to expose this data as String objects, in which case decoding the bytes and turning them into a String is often very performance sensitive - to the point of being one of our top bottlenecks for the given queries. Part of the story is that in the case of Latin1, I?ll know up front exactly how many bytes my string data takes up, though I might not know how many buffers are going to be involved. >>>> >>>> As far as I can tell, this is currently not possible using public APIs. Using private APIs it may be possible, but will be relying on the JIT for vectorising the memory copying. >>>> >>>> From an API standpoint, CharsetDecoder is close to home, but is not quite there. It?s stateful and not thread-safe, so I either have to allocate new ones every time or cache them in thread-locals. I?m also required to allocate the receiving CharBuffer. Since I may need to decode from multiple buffers, I realise that I might not be able to get away from allocating at least one extra object to keep track of intermediate decoding state. The CharsetDecoder does not have a method where I can specify the offset and length for the desired part of the ByteBuffer I want to decode, which forces be to allocate views instead. >>>> >>>> The CharBuffers are allocated with a length up front, which is nice, but I can?t restrict its encoding so it has to allocate a char array instead of the byte array that I really want. Even if it did allocate a byte array, the CharBuffer is mutable, which would force String do a defensive copy anyway. >>>> >>>> One way I imagine this could be solved would be with a less dynamic kind of decoder, where the target length is given upfront to the decoder. Buffers are then consumed one by one, and a terminal method performs finishing sanity checks (did we get all the bytes we were promised?) and returns the result. >>>> >>>> StringDecoder decoder = Charset.forName(?latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); >>>> String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, len2).done(); >>>> >>>> This will in principle allow the string decoding to be 2 small allocations, an array allocation without zeroing, and a sequence of potentially vectorised memcpys. I don?t see any potentially troubling interactions with fused Strings either, since all the knowledge (except for the string data itself) needed to allocate the String objects are available from the get-go. >>>> >>>> What do you guys think? >>>> >>>> Btw, Richard Warburton has already done some work in this area, and made a patch that adds a constructor to String that takes a buffer, offset, length, and charset. This work now at least needs rebasing: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ >>>> It doesn?t solve the case where multiple buffers are used to build the string, but does remove the need for a separate intermediate state-holding object when a single buffer is enough. It?d be a nice addition if possible, but I (for one) can tolerate a small object allocation otherwise. >>>> >>>> Cheers, >>>> Chris >>>> From martinrb at google.com Wed Feb 10 15:53:13 2016 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Feb 2016 07:53:13 -0800 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: <56BB15F7.1060606@oracle.com> References: <56BB15F7.1060606@oracle.com> Message-ID: Thanks for creating the specdiff, but ... it looks reversed; the green is the old and the red is the new! Sorry for our "endless fiddling"; we do have future changes in mind, but no spec changes. On Wed, Feb 10, 2016 at 2:50 AM, Chris Hegarty wrote: > On 02/02/16 15:23, Martin Buchholz wrote: >> >> On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty >> wrote: >>> >>> >>> On 1 Feb 2016, at 18:45, Martin Buchholz wrote: >>> >>>> After much debate on what to do when CompleteableFuture.whenComplete >>>> encounters an exception in both the source and the action, we chose >>>> what was acceptable to the most people - add the action's exception to >>>> the source exception as a suppressed exception. And added usage >>>> guidelines. And gave handle "top billing" over whenComplete. >>>> >>>> >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ >>> >>> >>> This all looks fine to me. >>> >>> So I assume you only need a small CCC request for CompletionStage, right? >>> Everything else is implementation. >> >> >> If you squint you might argue that CompletionStage's contract hasn't >> actually changed, >> but yeah, go ahead and do a CCC for CompletionStage. Publishing a >> specdiff would be nice - method reordering (for "top billing") has >> made the diffs harder to review. Thanks. > > > Here are the specdiffs that will be used for the CCC, unless there are > any last minute changes. > > http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html > > -Chris. From peter.levart at gmail.com Wed Feb 10 16:01:37 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 10 Feb 2016 17:01:37 +0100 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> References: <56BABEB8.2050505@oracle.com> <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> Message-ID: <56BB5EE1.9000506@gmail.com> On 02/10/2016 02:16 PM, Chris Vest wrote: > I?m slightly more worried about their array and what it implies, though. This API requires of me that I?m able to present all of the relevant buffers up front. In my case, these buffers are shared between multiple threads, so they are guarded by locks. I must take all the locks for all the buffers and hold them for the duration of the copy._Maybe_ I have to do that anyway, but if there?s a way for me to only lock one buffer at a time, I?d like to be able to exploit this. The builder approach holds this door open for me. In that case, you can't estimate the String array length upfront and you are forced to do copying afterwards. So you have to choose what is more important for you. Or do you have a case where the lengths of buffer sub-ranges are known in advance and you only want to gather their content one-by-one? Regards, Peter > > Cheers, > Chris From forax at univ-mlv.fr Wed Feb 10 17:19:07 2016 From: forax at univ-mlv.fr (forax at univ-mlv.fr) Date: Wed, 10 Feb 2016 18:19:07 +0100 (CET) Subject: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory In-Reply-To: <56BB3257.2020601@oracle.com> References: <56BB09BD.4000001@oracle.com> <1845801185.1485912.1455103549086.JavaMail.zimbra@u-pem.fr> <56BB3257.2020601@oracle.com> Message-ID: <1603359659.1661875.1455124747579.JavaMail.zimbra@u-pem.fr> I've miss the fact that you call getName() after defineAnonymousClass. So you're right that each class has a different name but calling the dumper after calling defineAnonymousClass is a mistake in my opinion because you are not able to debug verification errors that will be raised by defineAnonymousClass. cheers, R?mi ----- Mail original ----- > De: "Aleksey Shipilev" > ?: "Remi Forax" > Cc: "Java Core Libs" > Envoy?: Mercredi 10 F?vrier 2016 13:51:35 > Objet: Re: RFR (S) 8149459: StringConcatFactory should be synced up with LambdaMetafactory > > See new webrev that adds a few more tests: > http://cr.openjdk.java.net/~shade/8149459/webrev.02/ > > On 10.02.2016 14:25, Remi Forax wrote: > > Hi Aleksey, if you have a dumper, you should append a value at the > > end of the generated class otherwise if you have more than one string > > concatenation, you will see only one class on the disk. > > No, we are actually okay, because our class names are unique since > Unsafe.defineAnonymousClass handles this: > > $ ls -lR dump/java.lang.String\$Concat/ > dump/java.lang.String$Concat/2008017533.class > dump/java.lang.String$Concat/665576141.class > > Cheers, > -Aleksey > > From mikael.vidstedt at oracle.com Wed Feb 10 19:00:26 2016 From: mikael.vidstedt at oracle.com (Mikael Vidstedt) Date: Wed, 10 Feb 2016 11:00:26 -0800 Subject: RFR JDK-8141491: Unaligned memory access in Bits.c In-Reply-To: References: <56560F32.6040300@oracle.com> <565628F5.2080801@oracle.com> <56A6FCA5.7040808@oracle.com> <56A736B7.4020601@redhat.com> <8D4B5D84-FF89-461E-93B2-7FB2E56F9741@oracle.com> <56A7BF7B.5020006@redhat.com> <17C15340-167F-4B33-A6CF-8510EAC2491C@oracle.com> <56A7C410.4040301@redhat.com> <713CDD14-7C04-4B33-AC48-6A5474351C97@oracle.com> <56A96B55.7050301@oracle.com> <56B102AD.7020800@oracle.com> <79CA457A-E4E3-4D49-B22A-C959C16DAC49@oracle.com> <5E9239AE-D8C9-4A98-9C46-9FE0F130A06C@oracle.com> <56B5207B.8010107@oracle.com> <56B7DDD8.2050701@oracle.com> <56BAB18B.3070707@oracle.com> Message-ID: <56BB88C9.5010502@oracle.com> On 2016-02-10 02:03, Paul Sandoz wrote: >> On 10 Feb 2016, at 04:42, Mikael Vidstedt wrote: >> >> >> Can I please get a quick review of these updated webrevs: >> >> hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/hotspot/webrev/ >> jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05/jdk/webrev/ >> >> incremental webrevs: >> >> hotspot: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/hotspot/webrev/ >> jdk: http://cr.openjdk.java.net/~mikael/webrevs/8141491/webrev.05.incr/jdk/webrev/ >> > +1 > > I agree with David on the JavaDoc, but that could be followed up with any future changes, including potentially the removal of wrapping methods in Bits.java, since buffers any way use Unsafe the wrappers now appear to offer little value. I'm planning on cleaning up Unsafe.java pretty significantly in a separate change, so I will add the relevant javadocs as part of that. I'll also file a separate enhancement to remove the Bits wrappers. Thanks, Mikael > > Paul. > > >> Changes: >> >> * Added asserts in copy.cpp/conjoint_swap >> * Correctness: Moved offset sign checks to only be performed if corresponding base object is null, and added corresponding tests >> >> I'm about to make additional changes in this same area, so unless this last change is horribly broken I'm planning on pushing this and doing any additional cleanup in the upcoming change(s). >> >> Cheers, >> Mikael >> From martinrb at google.com Wed Feb 10 19:43:44 2016 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Feb 2016 11:43:44 -0800 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56BB41E6.3060102@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> Message-ID: There's an endless debate about the pros and cons of "dynamic linking". I think it would be best for the JDK to link to system libraries by default, if possible. For a particular JDK image, one can drop a patched libz into a suitable lib directory to override the system one. It should also be relatively easy to build a JDK with a static libz.a. And all the cool kids are building docker images where they control everything. The zlib/info-zip projects don't exhibit a high level of code health, but OOTO they seem quite stable these days. From Roger.Riggs at Oracle.com Wed Feb 10 20:25:08 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 10 Feb 2016 15:25:08 -0500 Subject: RFR 9: 8142539 : Incorrect definition of ZoneOffset.MIN and other javadoc cleanup Message-ID: <56BB9CA4.6090403@Oracle.com> Please review a trio of javadoc fixes for java.time. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-typos-8142539/ Issues: 8142539: Incorrect definition of ZoneOffset.MIN 8140211: Example in the Documentation is wrong for java.time.ZonedDateTime.minusHours 8139529: java.time.temporal.ChronoUnit.FOREVER typo Thanks, Roger From lance.andersen at oracle.com Wed Feb 10 20:53:07 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 10 Feb 2016 15:53:07 -0500 Subject: RFR 9: 8142539 : Incorrect definition of ZoneOffset.MIN and other javadoc cleanup In-Reply-To: <56BB9CA4.6090403@Oracle.com> References: <56BB9CA4.6090403@Oracle.com> Message-ID: Looks OK Roger On Feb 10, 2016, at 3:25 PM, Roger Riggs wrote: > 8139529 Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From david.holmes at oracle.com Wed Feb 10 21:18:23 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 11 Feb 2016 07:18:23 +1000 Subject: OpenJDK bug question (JDK-6519463) In-Reply-To: References: <56BAB83F.3010109@oracle.com> <56BAC2FA.6040509@oracle.com> Message-ID: <56BBA91F.6020906@oracle.com> On 10/02/2016 11:53 PM, David Hoffer wrote: > No that file is not visible to me either. Should be now. David > -Dave > > On Tue, Feb 9, 2016 at 9:56 PM, David Holmes > wrote: > > On 10/02/2016 2:20 PM, David Hoffer wrote: > > That issue has the same problem, it references files that are > not in the > bug report so it won't help, really need all files to be > attached to the > bug report. > > > Is the attachment a.zip.Z not visible to you on the bug report? > > David > > That being said we think we have tracked our problem down > to a file transfer step after compression so I don't think our issue > pertains to JDK/compression but just in case we are wrong I'd > like to > see the test files attached to the bug report...in case we need > them. > > Thanks, > -Dave > > On Tue, Feb 9, 2016 at 9:10 PM, David Holmes > > >> wrote: > > On 10/02/2016 4:51 AM, David Hoffer wrote: > > We have a production environment where we are getting > errors > similar to > what was reported in JDK-6519463. I would like to test > our code > with the > test file (buggedfile.bin) used in this bug report but > the file > is posted > on an ftp server that is not publicly available. Can > this file > be attached > to the bug report or just email me a copy? > > > Will the test file in: > > https://bugs.openjdk.java.net/browse/JDK-4040920 > > suffice, as it was reported to be the underlying cause. > > David > > Thanks, > -Dave > > > From scolebourne at joda.org Wed Feb 10 22:05:19 2016 From: scolebourne at joda.org (Stephen Colebourne) Date: Wed, 10 Feb 2016 22:05:19 +0000 Subject: RFR 9: 8142539 : Incorrect definition of ZoneOffset.MIN and other javadoc cleanup In-Reply-To: <56BB9CA4.6090403@Oracle.com> References: <56BB9CA4.6090403@Oracle.com> Message-ID: The plusHours() change loses something in the fix. previously, it had an example for summer-to-summer, summer-to-winter, and winter-to-winter. Now the last of these has gone. This would be better:
  • Adding one hour to 01:30+02:00 will result in 02:30+02:00 (both in summer time)
  • Adding one hour to 02:30+02:00 will result in 02:30+01:00 (moving from summer to winter time)
  • Adding one hour to 02:30+01:00 will result in 03:30+01:00 (both in summer time)
  • Adding three hours to 01:30+02:00 will result in 03:30+01:00 (moving from summer to winter time) other changes look fine thanks Stephen On 10 February 2016 at 20:25, Roger Riggs wrote: > Please review a trio of javadoc fixes for java.time. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-typos-8142539/ > > Issues: > 8142539: Incorrect definition of ZoneOffset.MIN > 8140211: Example in the Documentation is wrong for > java.time.ZonedDateTime.minusHours > 8139529: java.time.temporal.ChronoUnit.FOREVER typo > > Thanks, Roger > From chris.hegarty at oracle.com Wed Feb 10 22:51:49 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Wed, 10 Feb 2016 22:51:49 +0000 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: <56BB15F7.1060606@oracle.com> Message-ID: <4C0F0336-BC90-4652-9D84-A58CDC1C8590@oracle.com> > On 10 Feb 2016, at 15:53, Martin Buchholz wrote: > > Thanks for creating the specdiff, but ... it looks reversed; the green > is the old and the red is the new! D?oh, of course. Updated in-place. http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html > Sorry for our "endless fiddling"; we do have future changes in mind, > but no spec changes. No problem. -Chris. > On Wed, Feb 10, 2016 at 2:50 AM, Chris Hegarty wrote: >> On 02/02/16 15:23, Martin Buchholz wrote: >>> >>> On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty >>> wrote: >>>> >>>> >>>> On 1 Feb 2016, at 18:45, Martin Buchholz wrote: >>>> >>>>> After much debate on what to do when CompleteableFuture.whenComplete >>>>> encounters an exception in both the source and the action, we chose >>>>> what was acceptable to the most people - add the action's exception to >>>>> the source exception as a suppressed exception. And added usage >>>>> guidelines. And gave handle "top billing" over whenComplete. >>>>> >>>>> >>>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ >>>> >>>> >>>> This all looks fine to me. >>>> >>>> So I assume you only need a small CCC request for CompletionStage, right? >>>> Everything else is implementation. >>> >>> >>> If you squint you might argue that CompletionStage's contract hasn't >>> actually changed, >>> but yeah, go ahead and do a CCC for CompletionStage. Publishing a >>> specdiff would be nice - method reordering (for "top billing") has >>> made the diffs harder to review. Thanks. >> >> >> Here are the specdiffs that will be used for the CCC, unless there are >> any last minute changes. >> >> http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html >> >> -Chris. From martinrb at google.com Wed Feb 10 23:32:48 2016 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Feb 2016 15:32:48 -0800 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: <4C0F0336-BC90-4652-9D84-A58CDC1C8590@oracle.com> References: <56BB15F7.1060606@oracle.com> <4C0F0336-BC90-4652-9D84-A58CDC1C8590@oracle.com> Message-ID: The specdiff is very helpful. It was much easier to see that the cross-method links from whenComplete to handle could be improved, so we'll change (modulo reflow) --- src/main/java/util/concurrent/CompletionStage.java 24 Jan 2016 21:22:16 -0000 1.37 +++ src/main/java/util/concurrent/CompletionStage.java 10 Feb 2016 23:29:49 -0000 @@ -736,7 +736,7 @@ * {@code null} if none) of this stage as arguments. The returned * stage is completed when the action returns. * - *

    Unlike method {@link #handle}, this method is not designed + *

    Unlike method {@link #handle handle}, this method is not designed * to translate completion outcomes, so the supplied action should * not throw an exception. However, if it does, the following * rules apply: if this stage completed normally but the supplied @@ -762,7 +762,7 @@ * if none) of this stage as arguments. The returned stage is completed * when the action returns. * - *

    Unlike method {@link #handle}, this method is not designed + *

    Unlike method {@link #handleAsync(BiFunction) handleAsync}, this method is not designed * to translate completion outcomes, so the supplied action should * not throw an exception. However, if it does, the following * rules apply: If this stage completed normally but the supplied @@ -788,7 +788,7 @@ * if none) of this stage as arguments. The returned stage is completed * when the action returns. * - *

    Unlike method {@link #handle}, this method is not designed + *

    Unlike method {@link #handleAsync(BiFunction,Executor) handleAsync}, this method is not designed * to translate completion outcomes, so the supplied action should * not throw an exception. However, if it does, the following * rules apply: If this stage completed normally but the supplied On Wed, Feb 10, 2016 at 2:51 PM, Chris Hegarty wrote: > > On 10 Feb 2016, at 15:53, Martin Buchholz wrote: > > Thanks for creating the specdiff, but ... it looks reversed; the green > is the old and the red is the new! > > > D?oh, of course. Updated in-place. > > http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html > > Sorry for our "endless fiddling"; we do have future changes in mind, > but no spec changes. > > > No problem. > > -Chris. > > On Wed, Feb 10, 2016 at 2:50 AM, Chris Hegarty > wrote: > > On 02/02/16 15:23, Martin Buchholz wrote: > > > On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty > wrote: > > > > On 1 Feb 2016, at 18:45, Martin Buchholz wrote: > > After much debate on what to do when CompleteableFuture.whenComplete > encounters an exception in both the source and the action, we chose > what was acceptable to the most people - add the action's exception to > the source exception as a suppressed exception. And added usage > guidelines. And gave handle "top billing" over whenComplete. > > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ > > > > This all looks fine to me. > > So I assume you only need a small CCC request for CompletionStage, right? > Everything else is implementation. > > > > If you squint you might argue that CompletionStage's contract hasn't > actually changed, > but yeah, go ahead and do a CCC for CompletionStage. Publishing a > specdiff would be nice - method reordering (for "top billing") has > made the diffs harder to review. Thanks. > > > > Here are the specdiffs that will be used for the CCC, unless there are > any last minute changes. > > http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html > > -Chris. > > From xueming.shen at oracle.com Thu Feb 11 00:25:51 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 10 Feb 2016 16:25:51 -0800 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56BB4C24.7090607@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> <56BB4932.7060105@oracle.com> <56BB4C24.7090607@oracle.com> Message-ID: <56BBD50F.8070803@oracle.com> One of the benefits of moving to the system libz is actually for better/easy maintenance. Just replacing the offending version of libz with an earlier/later version that works, instead of waiting for a customized jdk/jre image with a working/bundled libz, or the next update release. Especially given the fact that we have decided not to touch the libz at source level. Having dependency on the system libz is really not that bad. The experience suggests those binaries are quite stable. And it should always be easier to replace the system libz than a java runtime, in case of problem. Sherman On 02/10/2016 06:41 AM, Se?n Coffey wrote: > > On 10/02/16 14:29, Alan Bateman wrote: >> >> On 10/02/2016 13:57, Se?n Coffey wrote: >>> I'm all for allowing one to introduce a new version of zlib to their JDK at runtime. I just don't think it's in the interests of enterprises and stability to introduce a dependency to the JDK on the underlying OS zlib libraries. Could we at least consider a runtime property to allow linking to the (currently bundled) zlib v.1.2.8 port in case issues arise ? >> Don't the LD_* environment variables serve this need already? Once the JDK is using the system zlib then this is the simplest way to get it to use a different libz library at runtime. > No - I don't see that as a solution. You've still made the default JDK config become dependent on OS environment for all libzip operations. I don't think we even capture the zlib version that the JDK would be operating with in any diagnostics. An application wanting a tried/tested and stable libzip version has extra work to do now. Letting the default be system dependent has just increased risk for QA teams also. A system property just makes this all go away. In fact - I would say that for JDK9, the default should be the JDK bundled libzip library. For those looking for libzip experimenting and performance benefits, they could take the system property approach. > > regards, > Sean. >> >> -Alan > > > On 08/02/16 09:55, Alan Bateman wrote: >> >> On 08/02/2016 10:42, Se?n Coffey wrote: >>> Is there an option to fall back to the older v.1.2.8 implementation if necessary ? It would certainly alleviate issues for any applications that might run into issues with the latest and greatest zlib libraries. JDK-8133206 would be one such example. >> Just at build time > so - we introduce a runtime dependency on the underlying zlib libraries on the OS by default. I would be very concerned with this approach. We've run JDK 6 for 10+ years with zlib v1.1.3. It was consistent and reliable for the most part. When we moved JDK7/8 to zlib v1.2.5, we encountered an inflation issue[1]. When JDK was upgraded to zlib v1.2.8, we received reports of performance/OOM issues [2]. >> but if the zlib on the platform is broken then it impacts tools and probably lots of things. I would assume the OS would patch such issues quickly. In the case of JDK-8133206 then was the issue addressed in the libzip wrapper or in the zlib code? I thought it was the former. > The code change is proposed in the libzip wrapper but the issue was triggered by the zlib library update. >> >> On a fallback or some way to configure at launch time then Sandhya Viswanathan (Intel) has a proposal here last year. I think we mostly agreed on that thread that switching the build to use the system zlib by default would be better. > I'm all for allowing one to introduce a new version of zlib to their JDK at runtime. I just don't think it's in the interests of enterprises and stability to introduce a dependency to the JDK on the underlying OS zlib libraries. Could we at least consider a runtime property to allow linking to the (currently bundled) zlib v.1.2.8 port in case issues arise ? > > regards, > Sean. > > [1] https://bugs.openjdk.java.net/browse/JDK-8044725 > [2] https://bugs.openjdk.java.net/browse/JDK-8133206 > From martinrb at google.com Thu Feb 11 01:37:18 2016 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Feb 2016 17:37:18 -0800 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: <56BB15F7.1060606@oracle.com> <4C0F0336-BC90-4652-9D84-A58CDC1C8590@oracle.com> Message-ID: and ... webrev regenerated On Wed, Feb 10, 2016 at 3:32 PM, Martin Buchholz wrote: > The specdiff is very helpful. It was much easier to see that the > cross-method links from whenComplete to handle could be improved, so > we'll change (modulo reflow) > > --- src/main/java/util/concurrent/CompletionStage.java 24 Jan 2016 > 21:22:16 -0000 1.37 > +++ src/main/java/util/concurrent/CompletionStage.java 10 Feb 2016 > 23:29:49 -0000 > @@ -736,7 +736,7 @@ > * {@code null} if none) of this stage as arguments. The returned > * stage is completed when the action returns. > * > - *

    Unlike method {@link #handle}, this method is not designed > + *

    Unlike method {@link #handle handle}, this method is not designed > * to translate completion outcomes, so the supplied action should > * not throw an exception. However, if it does, the following > * rules apply: if this stage completed normally but the supplied > @@ -762,7 +762,7 @@ > * if none) of this stage as arguments. The returned stage is completed > * when the action returns. > * > - *

    Unlike method {@link #handle}, this method is not designed > + *

    Unlike method {@link #handleAsync(BiFunction) handleAsync}, > this method is not designed > * to translate completion outcomes, so the supplied action should > * not throw an exception. However, if it does, the following > * rules apply: If this stage completed normally but the supplied > @@ -788,7 +788,7 @@ > * if none) of this stage as arguments. The returned stage is completed > * when the action returns. > * > - *

    Unlike method {@link #handle}, this method is not designed > + *

    Unlike method {@link #handleAsync(BiFunction,Executor) > handleAsync}, this method is not designed > * to translate completion outcomes, so the supplied action should > * not throw an exception. However, if it does, the following > * rules apply: If this stage completed normally but the supplied > > On Wed, Feb 10, 2016 at 2:51 PM, Chris Hegarty wrote: >> >> On 10 Feb 2016, at 15:53, Martin Buchholz wrote: >> >> Thanks for creating the specdiff, but ... it looks reversed; the green >> is the old and the red is the new! >> >> >> D?oh, of course. Updated in-place. >> >> http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html >> >> Sorry for our "endless fiddling"; we do have future changes in mind, >> but no spec changes. >> >> >> No problem. >> >> -Chris. >> >> On Wed, Feb 10, 2016 at 2:50 AM, Chris Hegarty >> wrote: >> >> On 02/02/16 15:23, Martin Buchholz wrote: >> >> >> On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty >> wrote: >> >> >> >> On 1 Feb 2016, at 18:45, Martin Buchholz wrote: >> >> After much debate on what to do when CompleteableFuture.whenComplete >> encounters an exception in both the source and the action, we chose >> what was acceptable to the most people - add the action's exception to >> the source exception as a suppressed exception. And added usage >> guidelines. And gave handle "top billing" over whenComplete. >> >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ >> >> >> >> This all looks fine to me. >> >> So I assume you only need a small CCC request for CompletionStage, right? >> Everything else is implementation. >> >> >> >> If you squint you might argue that CompletionStage's contract hasn't >> actually changed, >> but yeah, go ahead and do a CCC for CompletionStage. Publishing a >> specdiff would be nice - method reordering (for "top billing") has >> made the diffs harder to review. Thanks. >> >> >> >> Here are the specdiffs that will be used for the CCC, unless there are >> any last minute changes. >> >> http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html >> >> -Chris. >> >> From martinrb at google.com Thu Feb 11 03:19:35 2016 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Feb 2016 19:19:35 -0800 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56BBD50F.8070803@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> <56BB4932.7060105@oracle.com> <56BB4C24.7090607@oracle.com> <56BBD50F.8070803@oracle.com> Message-ID: I'm actually very happy that we've dropped private patches against libz. And using the system libz seems like the right thing to do on Unix systems, where libz should be ubiquitous. On Wed, Feb 10, 2016 at 4:25 PM, Xueming Shen wrote: > > One of the benefits of moving to the system libz is actually for better/easy > maintenance. Just replacing the offending version of libz with an > earlier/later > version that works, instead of waiting for a customized jdk/jre image with a > working/bundled libz, or the next update release. Especially given the fact > that we have decided not to touch the libz at source level. Having > dependency > on the system libz is really not that bad. The experience suggests those > binaries are quite stable. And it should always be easier to replace the > system libz than a java runtime, in case of problem. > > Sherman > > > On 02/10/2016 06:41 AM, Se?n Coffey wrote: >> >> >> On 10/02/16 14:29, Alan Bateman wrote: >>> >>> >>> On 10/02/2016 13:57, Se?n Coffey wrote: >>>> >>>> I'm all for allowing one to introduce a new version of zlib to their JDK >>>> at runtime. I just don't think it's in the interests of enterprises and >>>> stability to introduce a dependency to the JDK on the underlying OS zlib >>>> libraries. Could we at least consider a runtime property to allow linking to >>>> the (currently bundled) zlib v.1.2.8 port in case issues arise ? >>> >>> Don't the LD_* environment variables serve this need already? Once the >>> JDK is using the system zlib then this is the simplest way to get it to use >>> a different libz library at runtime. >> >> No - I don't see that as a solution. You've still made the default JDK >> config become dependent on OS environment for all libzip operations. I don't >> think we even capture the zlib version that the JDK would be operating with >> in any diagnostics. An application wanting a tried/tested and stable libzip >> version has extra work to do now. Letting the default be system dependent >> has just increased risk for QA teams also. A system property just makes this >> all go away. In fact - I would say that for JDK9, the default should be the >> JDK bundled libzip library. For those looking for libzip experimenting and >> performance benefits, they could take the system property approach. >> >> regards, >> Sean. >>> >>> >>> -Alan >> >> >> >> On 08/02/16 09:55, Alan Bateman wrote: >>> >>> >>> On 08/02/2016 10:42, Se?n Coffey wrote: >>>> >>>> Is there an option to fall back to the older v.1.2.8 implementation if >>>> necessary ? It would certainly alleviate issues for any applications that >>>> might run into issues with the latest and greatest zlib libraries. >>>> JDK-8133206 would be one such example. >>> >>> Just at build time >> >> so - we introduce a runtime dependency on the underlying zlib libraries on >> the OS by default. I would be very concerned with this approach. We've run >> JDK 6 for 10+ years with zlib v1.1.3. It was consistent and reliable for the >> most part. When we moved JDK7/8 to zlib v1.2.5, we encountered an inflation >> issue[1]. When JDK was upgraded to zlib v1.2.8, we received reports of >> performance/OOM issues [2]. >>> >>> but if the zlib on the platform is broken then it impacts tools and >>> probably lots of things. I would assume the OS would patch such issues >>> quickly. In the case of JDK-8133206 then was the issue addressed in the >>> libzip wrapper or in the zlib code? I thought it was the former. >> >> The code change is proposed in the libzip wrapper but the issue was >> triggered by the zlib library update. >>> >>> >>> On a fallback or some way to configure at launch time then Sandhya >>> Viswanathan (Intel) has a proposal here last year. I think we mostly agreed >>> on that thread that switching the build to use the system zlib by default >>> would be better. >> >> I'm all for allowing one to introduce a new version of zlib to their JDK >> at runtime. I just don't think it's in the interests of enterprises and >> stability to introduce a dependency to the JDK on the underlying OS zlib >> libraries. Could we at least consider a runtime property to allow linking to >> the (currently bundled) zlib v.1.2.8 port in case issues arise ? >> >> regards, >> Sean. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8044725 >> [2] https://bugs.openjdk.java.net/browse/JDK-8133206 >> > From peter.firmstone at zeus.net.au Thu Feb 11 08:47:26 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Thu, 11 Feb 2016 18:47:26 +1000 Subject: DeserializationPermission Proposal Message-ID: <56BC4A9E.4000807@zeus.net.au> The use case here is: Do we trust the code to deserialize unverified data from an untrusted source? For example, an attacker might look for the following: 1. Deserialization into privileged context 2. Deserialization with a class that catches and ignores IOException. 3. A hash collission that will cause equals to be invoked on a proxy and invocation handler. History has shown us that no permission was required to perform these actions. In this case, I'm not concerned who or which object might obtain a reference to some object that's serializable, because anyone can do so using a crafted stream reference. An attacker can obtain a reference to any object in the stream using a stream reference. Objects that shouldn't escape a permission check in a constructor or being returned from some method call containing a permission check, shouldn't be serializable. What I'm concerned about is, do we trust all classes belonging to this object's heirarchy to read unverified data from the stream and verify it before creating an object? Ideally the stream should be deserialized into a context that has no permission, but currently we have no way of restricting the attack surface to include only those classes necessary to perform some function. We could use the traditional stack context of the calling thread, as you suggest. It is true, an object that contains a field that didn't have deserialization permission, would itself become unserializable. Regards, Peter. > On 02/08/2016 10:19 PM, Peter Firmstone wrote: > >/ Why not, just prior to instantiating an object just prior to deserializing, add each class' ProtectionDomain in the objects hierarchy to an AccessControlContext and pass this to the SecurityManager's two argument checkPermission call? > />/ > />/ This permission could never be granted to a principal, it is only ever a code trust concern. This would allow an administrator to minimise the attack surface of Serializable classes. > / > I think rather than using the ProtectionDomain of the objects in the > serialization stream, would it not make more sense to capture and use > (exclusively) the access control context of the entity which is > constructing the stream? The reason I say this is that it's very > possible for a less-privileged object to have references to > more-privileged objects and vice-versa; also, in some cases you're > predicating the success of deserialization upon the order in which > objects are deserialized. > > For example, if I have a four-object graph of A, B, C, and D, in a > diamond like this: > > A->B > A->C > B->D > C->D > > If B's class does not have privileges to construct D, deserialization > will fail, even if C does. On the other hand, if B has permission to > construct D, but C doesn't, C can escape its restriction by relying on B > to have already deserialized the object. > > But by using one permission set - the captured context of the creator of > the stream, or perhaps the captured context of the "root" readObject() > call, you cannot change the authorization outcome of the deserialization > just by fiddling with the bits. A graph in this case is either valid or > not. > > -- > - DML From peter.firmstone at zeus.net.au Thu Feb 11 09:52:51 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Thu, 11 Feb 2016 19:52:51 +1000 Subject: DeserializationPermission Proposal In-Reply-To: <56BC4A9E.4000807@zeus.net.au> References: <56BC4A9E.4000807@zeus.net.au> Message-ID: <56BC59F3.4050109@zeus.net.au> An example might be more useful. Traditionally, when the first non serializable superclass zero argument constructor is called, the domains of the subclasses aren't present on the call stack. Any security checks performed in the constructor of the superclass will not include the subclasses domains. In the proposed case, prior to construction, all domains in the class heirarchy of the to be deserialized object via the local ObjectStreamClass, are added to an AccessControlContext, which is then passed to the SecurityManager two argument permission check. Now an attacker will not be able to construct this object unless all domains have DeserializationPermission. If we use the stack context, it won't contain the yet to be deserialized classes. Regards, Peter. On 11/02/2016 6:47 PM, Peter Firmstone wrote: > The use case here is: > > Do we trust the code to deserialize unverified data from an untrusted > source? > > For example, an attacker might look for the following: > > 1. Deserialization into privileged context > 2. Deserialization with a class that catches and ignores IOException. > 3. A hash collission that will cause equals to be invoked on a proxy > and invocation handler. > > History has shown us that no permission was required to perform these > actions. > > In this case, I'm not concerned who or which object might obtain a > reference to some object that's serializable, because anyone can do so > using a crafted stream reference. > > An attacker can obtain a reference to any object in the stream using a > stream reference. > > Objects that shouldn't escape a permission check in a constructor or > being returned from some method call containing a permission check, > shouldn't be serializable. > > What I'm concerned about is, do we trust all classes belonging to this > object's heirarchy to read unverified data from the stream and verify > it before creating an object? > > Ideally the stream should be deserialized into a context that has no > permission, but currently we have no way of restricting the attack > surface to include only those classes necessary to perform some function. > > We could use the traditional stack context of the calling thread, as > you suggest. > > It is true, an object that contains a field that didn't have > deserialization permission, would itself become unserializable. > > Regards, > > Peter. > > >> On 02/08/2016 10:19 PM, Peter Firmstone wrote: >> >/ Why not, just prior to instantiating an object just prior to >> deserializing, add each class' ProtectionDomain in the objects >> hierarchy to an AccessControlContext and pass this to the >> SecurityManager's two argument checkPermission call? >> />/ >> />/ This permission could never be granted to a principal, it is >> only ever a code trust concern. This would allow an administrator to >> minimise the attack surface of Serializable classes. >> / >> I think rather than using the ProtectionDomain of the objects in the >> serialization stream, would it not make more sense to capture and use >> (exclusively) the access control context of the entity which is >> constructing the stream? The reason I say this is that it's very >> possible for a less-privileged object to have references to >> more-privileged objects and vice-versa; also, in some cases you're >> predicating the success of deserialization upon the order in which >> objects are deserialized. >> >> For example, if I have a four-object graph of A, B, C, and D, in a >> diamond like this: >> >> A->B >> A->C >> B->D >> C->D >> >> If B's class does not have privileges to construct D, deserialization >> will fail, even if C does. On the other hand, if B has permission to >> construct D, but C doesn't, C can escape its restriction by relying on B >> to have already deserialized the object. >> >> But by using one permission set - the captured context of the creator of >> the stream, or perhaps the captured context of the "root" readObject() >> call, you cannot change the authorization outcome of the deserialization >> just by fiddling with the bits. A graph in this case is either valid or >> not. >> >> -- >> - DML From sean.coffey at oracle.com Thu Feb 11 10:34:20 2016 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 11 Feb 2016 10:34:20 +0000 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56BBD50F.8070803@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> <56BB4932.7060105@oracle.com> <56BB4C24.7090607@oracle.com> <56BBD50F.8070803@oracle.com> Message-ID: <56BC63AC.9050702@oracle.com> On 11/02/2016 00:25, Xueming Shen wrote: > > One of the benefits of moving to the system libz is actually for > better/easy > maintenance. Just replacing the offending version of libz with an > earlier/later > version that works, instead of waiting for a customized jdk/jre image > with a > working/bundled libz, or the next update release. Especially given the > fact > that we have decided not to touch the libz at source level. Having > dependency > on the system libz is really not that bad. The experience suggests those > binaries are quite stable. And it should always be easier to replace the > system libz than a java runtime, in case of problem. I think people overestimate people's ability to 'just replace' a zlib library. Adding a new system property is a struggle for some enterprises. We'll enjoy supporting a many to one matrix of zlib:JDK scenarios now rather than old style 1:1. It's great to have system library support - I'm just highlighting a possible risk. A fall back option solves that but there's no appetite for such a solution. Let's see how it goes. regards, Sean. > > Sherman > > On 02/10/2016 06:41 AM, Se?n Coffey wrote: >> >> On 10/02/16 14:29, Alan Bateman wrote: >>> >>> On 10/02/2016 13:57, Se?n Coffey wrote: >>>> I'm all for allowing one to introduce a new version of zlib to >>>> their JDK at runtime. I just don't think it's in the interests of >>>> enterprises and stability to introduce a dependency to the JDK on >>>> the underlying OS zlib libraries. Could we at least consider a >>>> runtime property to allow linking to the (currently bundled) zlib >>>> v.1.2.8 port in case issues arise ? >>> Don't the LD_* environment variables serve this need already? Once >>> the JDK is using the system zlib then this is the simplest way to >>> get it to use a different libz library at runtime. >> No - I don't see that as a solution. You've still made the default >> JDK config become dependent on OS environment for all libzip >> operations. I don't think we even capture the zlib version that the >> JDK would be operating with in any diagnostics. An application >> wanting a tried/tested and stable libzip version has extra work to do >> now. Letting the default be system dependent has just increased risk >> for QA teams also. A system property just makes this all go away. In >> fact - I would say that for JDK9, the default should be the JDK >> bundled libzip library. For those looking for libzip experimenting >> and performance benefits, they could take the system property approach. >> >> regards, >> Sean. >>> >>> -Alan >> >> >> On 08/02/16 09:55, Alan Bateman wrote: >>> >>> On 08/02/2016 10:42, Se?n Coffey wrote: >>>> Is there an option to fall back to the older v.1.2.8 implementation >>>> if necessary ? It would certainly alleviate issues for any >>>> applications that might run into issues with the latest and >>>> greatest zlib libraries. JDK-8133206 would be one such example. >>> Just at build time >> so - we introduce a runtime dependency on the underlying zlib >> libraries on the OS by default. I would be very concerned with this >> approach. We've run JDK 6 for 10+ years with zlib v1.1.3. It was >> consistent and reliable for the most part. When we moved JDK7/8 to >> zlib v1.2.5, we encountered an inflation issue[1]. When JDK was >> upgraded to zlib v1.2.8, we received reports of performance/OOM >> issues [2]. >>> but if the zlib on the platform is broken then it impacts tools and >>> probably lots of things. I would assume the OS would patch such >>> issues quickly. In the case of JDK-8133206 then was the issue >>> addressed in the libzip wrapper or in the zlib code? I thought it >>> was the former. >> The code change is proposed in the libzip wrapper but the issue was >> triggered by the zlib library update. >>> >>> On a fallback or some way to configure at launch time then Sandhya >>> Viswanathan (Intel) has a proposal here last year. I think we mostly >>> agreed on that thread that switching the build to use the system >>> zlib by default would be better. >> I'm all for allowing one to introduce a new version of zlib to their >> JDK at runtime. I just don't think it's in the interests of >> enterprises and stability to introduce a dependency to the JDK on the >> underlying OS zlib libraries. Could we at least consider a runtime >> property to allow linking to the (currently bundled) zlib v.1.2.8 >> port in case issues arise ? >> >> regards, >> Sean. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8044725 >> [2] https://bugs.openjdk.java.net/browse/JDK-8133206 >> > From chris.hegarty at oracle.com Thu Feb 11 10:56:16 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Feb 2016 10:56:16 +0000 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: References: <56BB15F7.1060606@oracle.com> <4C0F0336-BC90-4652-9D84-A58CDC1C8590@oracle.com> Message-ID: <28FC2496-E44B-46A4-941F-1772C52466B4@oracle.com> On 11 Feb 2016, at 01:37, Martin Buchholz wrote: > and ... webrev regenerated Thank you Martin, this is better. For completeness, the specdiff has been updated too: http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html -Chris. > On Wed, Feb 10, 2016 at 3:32 PM, Martin Buchholz wrote: >> The specdiff is very helpful. It was much easier to see that the >> cross-method links from whenComplete to handle could be improved, so >> we'll change (modulo reflow) >> >> --- src/main/java/util/concurrent/CompletionStage.java 24 Jan 2016 >> 21:22:16 -0000 1.37 >> +++ src/main/java/util/concurrent/CompletionStage.java 10 Feb 2016 >> 23:29:49 -0000 >> @@ -736,7 +736,7 @@ >> * {@code null} if none) of this stage as arguments. The returned >> * stage is completed when the action returns. >> * >> - *

    Unlike method {@link #handle}, this method is not designed >> + *

    Unlike method {@link #handle handle}, this method is not designed >> * to translate completion outcomes, so the supplied action should >> * not throw an exception. However, if it does, the following >> * rules apply: if this stage completed normally but the supplied >> @@ -762,7 +762,7 @@ >> * if none) of this stage as arguments. The returned stage is completed >> * when the action returns. >> * >> - *

    Unlike method {@link #handle}, this method is not designed >> + *

    Unlike method {@link #handleAsync(BiFunction) handleAsync}, >> this method is not designed >> * to translate completion outcomes, so the supplied action should >> * not throw an exception. However, if it does, the following >> * rules apply: If this stage completed normally but the supplied >> @@ -788,7 +788,7 @@ >> * if none) of this stage as arguments. The returned stage is completed >> * when the action returns. >> * >> - *

    Unlike method {@link #handle}, this method is not designed >> + *

    Unlike method {@link #handleAsync(BiFunction,Executor) >> handleAsync}, this method is not designed >> * to translate completion outcomes, so the supplied action should >> * not throw an exception. However, if it does, the following >> * rules apply: If this stage completed normally but the supplied >> >> On Wed, Feb 10, 2016 at 2:51 PM, Chris Hegarty wrote: >>> >>> On 10 Feb 2016, at 15:53, Martin Buchholz wrote: >>> >>> Thanks for creating the specdiff, but ... it looks reversed; the green >>> is the old and the red is the new! >>> >>> >>> D?oh, of course. Updated in-place. >>> >>> http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html >>> >>> Sorry for our "endless fiddling"; we do have future changes in mind, >>> but no spec changes. >>> >>> >>> No problem. >>> >>> -Chris. >>> >>> On Wed, Feb 10, 2016 at 2:50 AM, Chris Hegarty >>> wrote: >>> >>> On 02/02/16 15:23, Martin Buchholz wrote: >>> >>> >>> On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty >>> wrote: >>> >>> >>> >>> On 1 Feb 2016, at 18:45, Martin Buchholz wrote: >>> >>> After much debate on what to do when CompleteableFuture.whenComplete >>> encounters an exception in both the source and the action, we chose >>> what was acceptable to the most people - add the action's exception to >>> the source exception as a suppressed exception. And added usage >>> guidelines. And gave handle "top billing" over whenComplete. >>> >>> >>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ >>> >>> >>> >>> This all looks fine to me. >>> >>> So I assume you only need a small CCC request for CompletionStage, right? >>> Everything else is implementation. >>> >>> >>> >>> If you squint you might argue that CompletionStage's contract hasn't >>> actually changed, >>> but yeah, go ahead and do a CCC for CompletionStage. Publishing a >>> specdiff would be nice - method reordering (for "top billing") has >>> made the diffs harder to review. Thanks. >>> >>> >>> >>> Here are the specdiffs that will be used for the CCC, unless there are >>> any last minute changes. >>> >>> http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html >>> >>> -Chris. >>> >>> From mr.chrisvest at gmail.com Thu Feb 11 11:00:06 2016 From: mr.chrisvest at gmail.com (Chris Vest) Date: Thu, 11 Feb 2016 12:00:06 +0100 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <56BB5EE1.9000506@gmail.com> References: <56BABEB8.2050505@oracle.com> <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> <56BB5EE1.9000506@gmail.com> Message-ID: The OneShortStringBuilder looks nice. I would have to specify the Charset in its constructor, or somehow specify the capacity in bytes rather than characters, for it to optimally allocate the underlying byte array. It?s otherwise very similar, though probably more generally useful, to the builder I suggested in the beginning. Cheers, Chris > On 10 Feb 2016, at 17:01, Peter Levart wrote: > > > On 02/10/2016 02:16 PM, Chris Vest wrote: >> I?m slightly more worried about their array and what it implies, though. This API requires of me that I?m able to present all of the relevant buffers up front. In my case, these buffers are shared between multiple threads, so they are guarded by locks. I must take all the locks for all the buffers and hold them for the duration of the copy._Maybe_ I have to do that anyway, but if there?s a way for me to only lock one buffer at a time, I?d like to be able to exploit this. The builder approach holds this door open for me. > > In that case, you can't estimate the String array length upfront and you are forced to do copying afterwards. So you have to choose what is more important for you. Or do you have a case where the lengths of buffer sub-ranges are known in advance and you only want to gather their content one-by-one? I?m not entirely sure what you are telling me. I know from the start what length of String I want in the end, but I?d prefer to feed it buffers one-by-one. The slice I need out of each buffer will definitely differ from buffer to buffer, and I probably can?t tell up-front how many buffers I?ll use without also taking all their locks up-front - this I?d like to avoid, and instead only take my locks as I need them, and hold them for as short a duration as possible. > > Regards, Peter > >> >> Cheers, >> Chris > From paul.sandoz at oracle.com Thu Feb 11 11:13:50 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 11 Feb 2016 12:13:50 +0100 Subject: Compact Strings and APIs for fast decoding of string data In-Reply-To: <56BB5C35.8030006@gmail.com> References: <56BABEB8.2050505@oracle.com> <13D8351E-06E4-4C6E-84B1-EDE47880EC3C@oracle.com> <639EC624-37AF-4A6C-B6DF-DF8E4C95B952@gmail.com> <56BB5C35.8030006@gmail.com> Message-ID: <7F7226AA-4435-4730-95F3-F8B722461EB0@oracle.com> Hi Peter, That?s a clever little wrapper, but it seems very particular. What if we could have (excuse the name): StringBuilder.toStringWithReset(); That would reset it?s own array to that as if by the default constructor. A copy is still required but? we could make this intrinsic (as is the case already for toString). If the intrinsic can determine the builder has not escaped between construction and toStringWithReset() it may be able to apply optimisations like transfer of array ref and perhaps avoiding the zeroing of that array. I dunno if it?s possible to be smarter and for this to work correctly with the toString intrinsic e.g. if the runtime compiler can detect no further usages of the builder after toString. Paul. > On 10 Feb 2016, at 16:50, Peter Levart wrote: > > Hi Chris and others, > > On 02/10/2016 02:16 PM, Chris Vest wrote: >> It?d be a shame to loose the no-zeroing optimisation, so if that requires a constructor, then lets find a constructor. >> >> To recap, I have possibly multiple buffers and I only want to use parts of each. So with the composed buffer I?d have to first extract view buffers and then compose those, so I end up with 3 levels of buffers. Not ideal. A value type as a tuple of a buffer reference, an offset and a length would be better. We could make an array of those and pass that in. It could also find uses elsewhere in JDK where the buffer+offset+length pattern occurs. The array, a long with an encoding, could be passed up front to a String constructor (or a static method ? I don?t care) and my use case would be equally satisfied. > > From your description the most suitable API seems to be a method on StringBuilder: > > public StringBuilder append(ByteBuffer bb, int offset, int length, Charset cs) > > The only problem with this approach is that even if you guess the correct capacity of StringBuilder, a defensive copy of the byte[] has to be performed in toString(), because in the absence of synchronization, a data race could allow the String's byte[] to be modified after it is constructed. > > But there is a trick that doesn't need synchronization and prevents data races: a OneShotStringBuilder: > > http://cr.openjdk.java.net/~plevart/misc/OneShotStringBuilder/OneShotStringBuilder.java > > We now have optimized strategies for string concatenation (indyfied String concat), but for programmatic building of Strings we are still bound by sub-optimal StringBuilder that does defensive copying at the end although the capacity is exactly estimated an the builder is abandoned afterwards. > > So perhaps if such OneShotStringBuilder has enough weight on its own to be added to the standard API, it could also be equipped with ByteBuffer appending&decoding operation(s)... > > As it's use is constrained to single thread and for constructing a single String, there's a big chance that JIT could optimize and eliminate its allocation. In which case such builder is comparable to a String constructor or factory method, although much more flexible in expression. > > > What do you think? > > Regards, Peter > >> >> If I?ve understood correctly, there?s a plan (or at least a great desire) to turn Optional into a value type once they become available. These ?buffer slice? objects could follow a similar path, and thus be made available independently of value types. Even if they can?t, the overhead they add would be small, I imagine. >> >> I?m slightly more worried about their array and what it implies, though. This API requires of me that I?m able to present all of the relevant buffers up front. In my case, these buffers are shared between multiple threads, so they are guarded by locks. I must take all the locks for all the buffers and hold them for the duration of the copy. _Maybe_ I have to do that anyway, but if there?s a way for me to only lock one buffer at a time, I?d like to be able to exploit this. The builder approach holds this door open for me. >> >> Cheers, >> Chris >> >>> On 10 Feb 2016, at 09:54, Paul Sandoz wrote: >>> >>> Hi, >>> >>> A more functional approach would be to compose a sequence buffers into one view, perhaps read-only. Then there would be no need to accept arrays of buffers. That should work well for bulk operations. That?s a non-trivial but not very difficult amount of work, and possibly simplified if restricted to read-only views. >>> >>> Thus i think we should focus Richard?s work with: >>> >>> String(ByteBuffer src, String charset) >>> >>> and perhaps a sub-range variant, if perturbing the position/limit of an existing buffer and/or slicing is too problematic. >>> >>> ? >>> >>> Zeroing memory and possibly avoiding it can be tricky. Any such optimisations have to be carefully performed otherwise uninitialised regions might leak and be accessed, nefariously or otherwise. I imagine it?s easier to contain/control within a constructor than say a builder. >>> >>> Paul. >>> >>>> On 10 Feb 2016, at 05:38, Xueming Shen wrote: >>>> >>>> Hi Chris, >>>> >>>> I think basically you are asking a String constructor that takes a ByteBuffer. StringCoding >>>> then can take advantage of the current CompactString design to optimize the decoding >>>> operation by just a single byte[]/vectorized memory copy from the ByteBuffer to the String's >>>> internal byte[], WHEN the charset is 8859-1. >>>> >>>> String(ByteBuffer src, String charset); >>>> >>>> Further we will need a "buffer gathering" style constructor >>>> >>>> String(ByteBuffer[] srcs, String charset); >>>> (or more generally, String(ByteBuffer[] srcs, int off, int len, String charset) >>>> >>>> to create a String object from a sequence of ByteBuffers, if it's really desired. >>>> >>>> And then I would also assume it will also be desired to extend the current >>>> CharsetDecoder/Encoder class as well to add a pair of the "gathering" style coding >>>> methods >>>> >>>> CharBuffer CharsetDecoder.decode(ByteBuffer... ins); >>>> ByteBuffer CharsetEncoder.encode(CharBuffer... ins); >>>> >>>> Though the implementation might have to deal with the tricky "splitting >>>> byte/char" issue, in which part of the "byte/char sequence" is in the previous >>>> buffer and the continuing byte/chars are in the next following buffer ... >>>> >>>> -Sherman >>>> >>>> >>>> On 2/9/16 7:20 AM, Chris Vest wrote: >>>>> Hi, >>>>> >>>>> Aleksey Shipilev did a talk on his journey to implement compact strings and indified string concat at the JVM Tech Summit yesterday, and this reminded me that we (Neo4j) have a need for turning segments of DirectByteBuffers into Strings as fast as possible. If we already store the string data in Latin1, which is one of the two special encodings for compact strings, we?d ideally like to produce the String object with just the two necessary object allocations and a single, vectorised memory copy. >>>>> >>>>> Our use case is that we are a database and we do our own file paging, effectively having file data in a large set of DirectByteBuffers. We have string data in our files in a number of different encodings, a popular one being Latin1. Occasionally these String values span multiple buffers. We often need to expose this data as String objects, in which case decoding the bytes and turning them into a String is often very performance sensitive - to the point of being one of our top bottlenecks for the given queries. Part of the story is that in the case of Latin1, I?ll know up front exactly how many bytes my string data takes up, though I might not know how many buffers are going to be involved. >>>>> >>>>> As far as I can tell, this is currently not possible using public APIs. Using private APIs it may be possible, but will be relying on the JIT for vectorising the memory copying. >>>>> >>>>> From an API standpoint, CharsetDecoder is close to home, but is not quite there. It?s stateful and not thread-safe, so I either have to allocate new ones every time or cache them in thread-locals. I?m also required to allocate the receiving CharBuffer. Since I may need to decode from multiple buffers, I realise that I might not be able to get away from allocating at least one extra object to keep track of intermediate decoding state. The CharsetDecoder does not have a method where I can specify the offset and length for the desired part of the ByteBuffer I want to decode, which forces be to allocate views instead. >>>>> >>>>> The CharBuffers are allocated with a length up front, which is nice, but I can?t restrict its encoding so it has to allocate a char array instead of the byte array that I really want. Even if it did allocate a byte array, the CharBuffer is mutable, which would force String do a defensive copy anyway. >>>>> >>>>> One way I imagine this could be solved would be with a less dynamic kind of decoder, where the target length is given upfront to the decoder. Buffers are then consumed one by one, and a terminal method performs finishing sanity checks (did we get all the bytes we were promised?) and returns the result. >>>>> >>>>> StringDecoder decoder = Charset.forName(?latin1").newStringDecoder(lengthInCharactersOrBytesImNotSureWhichIsBest); >>>>> String result = decoder.decode(buf1, off1, len1).decode(buf2, off2, len2).done(); >>>>> >>>>> This will in principle allow the string decoding to be 2 small allocations, an array allocation without zeroing, and a sequence of potentially vectorised memcpys. I don?t see any potentially troubling interactions with fused Strings either, since all the knowledge (except for the string data itself) needed to allocate the String objects are available from the get-go. >>>>> >>>>> What do you guys think? >>>>> >>>>> Btw, Richard Warburton has already done some work in this area, and made a patch that adds a constructor to String that takes a buffer, offset, length, and charset. This work now at least needs rebasing: http://cr.openjdk.java.net/~rwarburton/string-patch-webrev/ >>>>> It doesn?t solve the case where multiple buffers are used to build the string, but does remove the need for a separate intermediate state-holding object when a single buffer is enough. It?d be a nice addition if possible, but I (for one) can tolerate a small object allocation otherwise. >>>>> >>>>> Cheers, >>>>> Chris >>>>> > From david.lloyd at redhat.com Thu Feb 11 12:16:20 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Thu, 11 Feb 2016 06:16:20 -0600 Subject: DeserializationPermission Proposal In-Reply-To: <56BC59F3.4050109@zeus.net.au> References: <56BC4A9E.4000807@zeus.net.au> <56BC59F3.4050109@zeus.net.au> Message-ID: <56BC7B94.8060502@redhat.com> On 02/11/2016 03:52 AM, Peter Firmstone wrote: > An example might be more useful. > > Traditionally, when the first non serializable superclass zero argument > constructor is called, the domains of the subclasses aren't present on > the call stack. Any security checks performed in the constructor of the > superclass will not include the subclasses domains. > > In the proposed case, prior to construction, all domains in the class > heirarchy of the to be deserialized object via the local > ObjectStreamClass, are added to an AccessControlContext, which is then > passed to the SecurityManager two argument permission check. Sure, that makes sense; in fact this could be a very good standalone/incremental enhancement. > Now an attacker will not be able to construct this object unless all > domains have DeserializationPermission. > > If we use the stack context, it won't contain the yet to be deserialized > classes. True; perhaps the ideal solution would use the initial context plus a per-deserializing-class context, so that both the original caller and the class being deserialized have permissions. This would have the advantage of consistent behavior, and would also allow the PD of each class to restrict whether it can be deserialized (which would apply globally no matter who was doing the deserialization in the VM). -- - DML From chris.hegarty at oracle.com Thu Feb 11 14:04:32 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Feb 2016 14:04:32 +0000 Subject: RFR [9] 8149653: Move sun.misc.InnocuousThread to jdk.internal.misc Message-ID: In preparation for JEP 260, InnocuousThread should be moved out of the sun.misc package and placed into a more appropriate package where it can be encapsulated. http://cr.openjdk.java.net/~chegar/8149653/ -Chris. From Roger.Riggs at Oracle.com Thu Feb 11 15:23:19 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Feb 2016 10:23:19 -0500 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <56BB4861.8000206@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> <56BB41CD.2050801@oracle.com> <56BB4437.4000205@oracle.com> <56BB4861.8000206@oracle.com> Message-ID: <56BCA767.3090806@Oracle.com> Hi Chris, The change to preallocate the StringBuffer is fine. But I have misgivings about not consuming the data from the stream and returning the empty string for out of range lengths. It will silently leave the input stream in an unusable state. I think it should throw an exception (StreamCorruptedException would be appropriate) if the size is negative or too large. BTW, a UTF string can be up to three times the length of a Java string due to the encoding. So utflen should be allowed to be Integer.MAX_VALUE * 3 (in the absence of some other implementation limitation). Roger On 2/10/2016 9:25 AM, Aleksey Shipilev wrote: > On 10.02.2016 17:07, Chris Hegarty wrote: >> Thanks Aleksey, your proposal is better. So the complete change is: >> >> diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java >> b/src/java.base/share/classes/java/io/ObjectInputStream.java >> --- a/src/java.base/share/classes/java/io/ObjectInputStream.java >> +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java >> @@ -3144,7 +3144,12 @@ >> * utflen bytes. >> */ >> private String readUTFBody(long utflen) throws IOException { >> - StringBuilder sbuf = new StringBuilder(); >> + if (utflen < 0 || utflen > Integer.MAX_VALUE) >> + return ""; >> + >> + // a reasonable initial capacity based on the UTF length >> + int initialCapacity = Math.min((int)utflen, 16384); >> + StringBuilder sbuf = new StringBuilder(initialCapacity); >> if (!blkmode) { >> end = pos = 0; >> } > > +1 > > -Aleksey > > From Roger.Riggs at Oracle.com Thu Feb 11 15:32:08 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Feb 2016 10:32:08 -0500 Subject: RFR [9] 8149653: Move sun.misc.InnocuousThread to jdk.internal.misc In-Reply-To: References: Message-ID: <56BCA978.3000709@Oracle.com> Hi Chris, Looks fine. I did wonder why src/java.base/share/classes/sun/nio/ch/ThreadPool.java didn't use the import while all the other cases did. But it would be just for consistency of style. Roger On 2/11/2016 9:04 AM, Chris Hegarty wrote: > In preparation for JEP 260, InnocuousThread should be moved out of the > sun.misc package and placed into a more appropriate package where it > can be encapsulated. > > http://cr.openjdk.java.net/~chegar/8149653/ > > -Chris. From chris.hegarty at oracle.com Thu Feb 11 15:37:52 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Feb 2016 15:37:52 +0000 Subject: RFR [9] 8149653: Move sun.misc.InnocuousThread to jdk.internal.misc In-Reply-To: <56BCA978.3000709@Oracle.com> References: <56BCA978.3000709@Oracle.com> Message-ID: <81C9BA79-91A4-4084-B5FF-F6463EDED18A@oracle.com> On 11 Feb 2016, at 15:32, Roger Riggs wrote: > Hi Chris, > > Looks fine. Thanks Roger. > I did wonder why src/java.base/share/classes/sun/nio/ch/ThreadPool.java > didn't use the import while all the other cases did. But it would be just for consistency of style. I can go either way. I?ll change it before pushing. -Chris. > Roger > > > On 2/11/2016 9:04 AM, Chris Hegarty wrote: >> In preparation for JEP 260, InnocuousThread should be moved out of the >> sun.misc package and placed into a more appropriate package where it >> can be encapsulated. >> >> http://cr.openjdk.java.net/~chegar/8149653/ >> >> -Chris. > From chris.hegarty at oracle.com Thu Feb 11 15:45:04 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Feb 2016 15:45:04 +0000 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <56BCA767.3090806@Oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> <56BB41CD.2050801@oracle.com> <56BB4437.4000205@oracle.com> <56BB4861.8000206@oracle.com> <56BCA767.3090806@Oracle.com> Message-ID: <575B13AF-EFB2-46FD-B52A-1898EEE154C8@oracle.com> Ha, such a seemingly innocuous little perf change has generated much noise ;-) You are of course right. On 11 Feb 2016, at 15:23, Roger Riggs wrote: > Hi Chris, > > The change to preallocate the StringBuffer is fine. > > But I have misgivings about not consuming the data from the stream and returning the empty string > for out of range lengths. It will silently leave the input stream in an unusable state. > > I think it should throw an exception (StreamCorruptedException would be appropriate) > if the size is negative or too large. > > BTW, a UTF string can be up to three times the length of a Java string due to the encoding. > So utflen should be allowed to be Integer.MAX_VALUE * 3 (in the absence of some other implementation limitation). How about this, preserving existing behaviour for non-common cases: diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java @@ -3144,7 +3144,14 @@ * utflen bytes. */ private String readUTFBody(long utflen) throws IOException { - StringBuilder sbuf = new StringBuilder(); + StringBuilder sbuf ; + if (utflen > 0 && utflen < Integer.MAX_VALUE) { + // a reasonable initial capacity based on the UTF length + int initialCapacity = Math.min((int)utflen, 16384); + sbuf = new StringBuilder(initialCapacity); + } else { + sbuf = new StringBuilder(); + } if (!blkmode) { end = pos = 0; } -Chris. From Roger.Riggs at Oracle.com Thu Feb 11 15:59:33 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Feb 2016 10:59:33 -0500 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <575B13AF-EFB2-46FD-B52A-1898EEE154C8@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> <56BB41CD.2050801@oracle.com> <56BB4437.4000205@oracle.com> <56BB4861.8000206@oracle.com> <56BCA767.3090806@Oracle.com> <575B13AF-EFB2-46FD-B52A-1898EEE154C8@oracle.com> Message-ID: <56BCAFE5.90403@Oracle.com> Hi Chris, ok, but can I suggest using 65535 instead of 16384, it will match the length of the short encoded strings in writeObjectStream.writeString. Thanks, Roger On 2/11/2016 10:45 AM, Chris Hegarty wrote: > Ha, such a seemingly innocuous little perf change has generated much noise ;-) > > You are of course right. > > On 11 Feb 2016, at 15:23, Roger Riggs wrote: > >> Hi Chris, >> >> The change to preallocate the StringBuffer is fine. >> >> But I have misgivings about not consuming the data from the stream and returning the empty string >> for out of range lengths. It will silently leave the input stream in an unusable state. >> >> I think it should throw an exception (StreamCorruptedException would be appropriate) >> if the size is negative or too large. >> >> BTW, a UTF string can be up to three times the length of a Java string due to the encoding. >> So utflen should be allowed to be Integer.MAX_VALUE * 3 (in the absence of some other implementation limitation). > How about this, preserving existing behaviour for non-common cases: > > diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java > --- a/src/java.base/share/classes/java/io/ObjectInputStream.java > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java > @@ -3144,7 +3144,14 @@ > * utflen bytes. > */ > private String readUTFBody(long utflen) throws IOException { > - StringBuilder sbuf = new StringBuilder(); > + StringBuilder sbuf ; > + if (utflen > 0 && utflen < Integer.MAX_VALUE) { > + // a reasonable initial capacity based on the UTF length > + int initialCapacity = Math.min((int)utflen, 16384); > + sbuf = new StringBuilder(initialCapacity); > + } else { > + sbuf = new StringBuilder(); > + } > if (!blkmode) { > end = pos = 0; > } > > > -Chris. From chris.hegarty at oracle.com Thu Feb 11 16:28:55 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Feb 2016 16:28:55 +0000 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <56BCAFE5.90403@Oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> <56BB41CD.2050801@oracle.com> <56BB4437.4000205@oracle.com> <56BB4861.8000206@oracle.com> <56BCA767.3090806@Oracle.com> <575B13AF-EFB2-46FD-B52A-1898EEE154C8@oracle.com> <56BCAFE5.90403@Oracle.com> Message-ID: <7A8256CC-CBCB-411A-BD57-BD0B093A4B6C@oracle.com> On 11 Feb 2016, at 15:59, Roger Riggs wrote: > Hi Chris, > > ok, but can I suggest using 65535 instead of 16384, it will match the length > of the short encoded strings in writeObjectStream.writeString. That sounds reasonable. So the final change is: diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java --- a/src/java.base/share/classes/java/io/ObjectInputStream.java +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java @@ -3144,7 +3144,14 @@ * utflen bytes. */ private String readUTFBody(long utflen) throws IOException { - StringBuilder sbuf = new StringBuilder(); + StringBuilder sbuf ; + if (utflen > 0 && utflen < Integer.MAX_VALUE) { + // a reasonable initial capacity based on the UTF length + int initialCapacity = Math.min((int)utflen, 65536); + sbuf = new StringBuilder(initialCapacity); + } else { + sbuf = new StringBuilder(); + } if (!blkmode) { end = pos = 0; } -Chris. From Roger.Riggs at Oracle.com Thu Feb 11 16:42:25 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Feb 2016 11:42:25 -0500 Subject: RFR [9]8134424: BlockDataInputStream.readUTFBody: examine sizing local StringBuffer with the given length In-Reply-To: <7A8256CC-CBCB-411A-BD57-BD0B093A4B6C@oracle.com> References: <467B2823-AF81-44B5-B337-C661E9A3C96D@oracle.com> <65CACF15-C7B0-4FAE-9032-618C1FCFA96D@oracle.com> <594C356E-6150-4716-A9F5-D00FD0D6A6FD@oracle.com> <56B90EA6.7020406@oracle.com> <56BB4045.40405@oracle.com> <56BB41CD.2050801@oracle.com> <56BB4437.4000205@oracle.com> <56BB4861.8000206@oracle.com> <56BCA767.3090806@Oracle.com> <575B13AF-EFB2-46FD-B52A-1898EEE154C8@oracle.com> <56BCAFE5.90403@Oracle.com> <7A8256CC-CBCB-411A-BD57-BD0B093A4B6C@oracle.com> Message-ID: <56BCB9F1.8010206@Oracle.com> Thanks, looks good. On 2/11/2016 11:28 AM, Chris Hegarty wrote: > On 11 Feb 2016, at 15:59, Roger Riggs wrote: > >> Hi Chris, >> >> ok, but can I suggest using 65535 instead of 16384, it will match the length >> of the short encoded strings in writeObjectStream.writeString. > That sounds reasonable. So the final change is: > > diff --git a/src/java.base/share/classes/java/io/ObjectInputStream.java b/src/java.base/share/classes/java/io/ObjectInputStream.java > --- a/src/java.base/share/classes/java/io/ObjectInputStream.java > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java > @@ -3144,7 +3144,14 @@ > * utflen bytes. > */ > private String readUTFBody(long utflen) throws IOException { > - StringBuilder sbuf = new StringBuilder(); > + StringBuilder sbuf ; > + if (utflen > 0 && utflen < Integer.MAX_VALUE) { > + // a reasonable initial capacity based on the UTF length > + int initialCapacity = Math.min((int)utflen, 65536); > + sbuf = new StringBuilder(initialCapacity); > + } else { > + sbuf = new StringBuilder(); > + } > if (!blkmode) { > end = pos = 0; > } > -Chris. From iris.clark at oracle.com Thu Feb 11 17:13:14 2016 From: iris.clark at oracle.com (Iris Clark) Date: Thu, 11 Feb 2016 09:13:14 -0800 (PST) Subject: RFR: 8149601 Update references from "1.9" to "9" Message-ID: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> Hi. Please review the following changes to the jaxp and jdk repositories to fix a few additional references to "1.9". Bug 8149601 Update references from "1.9" to "9" https://bugs.openjdk.java.net/browse/JDK-8149601 Webrev http://cr.openjdk.java.net/~iris/verona/8149601/webrev/ This is a follow-up to JDK-8136494 which changed "@since 1.9" to "@since 9" matching java.specification.version. Thanks, iris From anubis1 at linux-ecke.de Wed Feb 10 19:46:09 2016 From: anubis1 at linux-ecke.de (Thomas Kamps) Date: Wed, 10 Feb 2016 20:46:09 +0100 Subject: New Collection framework? In-Reply-To: <56BAB6A0.7070402@oracle.com> References: <2485208.rZGvsojIaB@linux-vgv1> <56BAB6A0.7070402@oracle.com> Message-ID: <2599851.in5580rWKj@linux-vgv1> Am Mittwoch, 10. Februar 2016, 14:03:44 schrieben Sie: > On 9/02/2016 5:29 AM, Thomas Kamps wrote: > > Hi everyone, > > > > Did you ever think about a new Collection framework for Java? > > A new framework would need to be significantly better than the existing > one to ever consider any kind of replacement. It would need to be > compatible in such a way that users can transition over time, and that > time would be quite lengthy - at least 2 major releases, more likely 3 IMHO. > > The Java 8 Streams provide a new way for working with collections that > can avoid problems/limitations in the base API's. > > > My main complains about the java.util Collections are: > > > > * No co/contra variance for assigning. Which forbids assigments like > > List foo = new List(); > > Currently it can be bypassed when using wildcards, but then you cannot add > > items to foo. Even more problematic are interfaces with some kind of > > List> as parameter > > That's a language issue not a j.u.Collection issue per-se. But of course > if generics had been around at the beginning there would likely have > been numerous API differences in the Collection framework. > > > * The main Collection interface allowes too much. If you get a Collection > > and you should not modify it, your programm will either behave stragly > > with a hard to detect bug, or crash with some kind of > > UnsupportedOperationException, when you do it. > > I have my problems with UnsupprtedOperationException, because when i get a > > Collection, i have to read the documentation (if existing) to know what i > > am allowed to do, or run unexpectedly into an exception. Would it be > > great to hava an "readonly" interface, which simply does not have and > > modifing methods? > I don't understand your problem. If you write a method that takes a > collection then your method has a specification as to what it does that > involves the collection that is passed in. If the passed in collection > doesn't in fact support that then the UnsupportedOperationException > propagates back to the caller that passed you an inappropriate > collection - and that's exactly as it should be. If you take a > collection and then try to do something to it that you didn't specify > you were going to do, and get UnsupportedOperationException then that is > a bug in your code - if you let it propagate then your caller gets it > and the user of your code submits a bug report. > > In case you have not seen it there is a document that discusses the > design of the Collections framework and why the existing approach was taken: > > https://docs.oracle.com/javase/8/docs/technotes/guides/collections/designfaq > .html The point of my argument was: If you dont habe modifiing methods, then you can't create such bugs. In many cases (at least at my work) there are Objects that holds some kind of Colletions, that are modified during its lifetime and have a getter for other to read them. If you dont want to have the inner collection to be modified by someone else, you can do either: * Make a copy * Wrap a Unmodifiable around it The first solution can be expensive. The second one sound better, but as said, wrong usage will be only seen at runtime/tests. Yes it is true, that a distinction in ReadOnly, AddOnly, RemoveOnly and Mutable can cause in many classes. I have count the public Interfaces in my Collection-Framework: 24 in total (4 Collection, 4 Sets, 4 Maps, 4 List, 4 ListIterators, 2 Iterable and 2 Iterator). To keep an overwiev i have put them into differen packages. I hav read the link. The main reason seems to be "Avoid Interface hell", which is understandable, when needing 24 Interfaces for only 3 different Collecction-types. but in my point of view it just feels right using the typesystem to avoid errors (Even it adds some kind of bureaucracy). That's what type systems are for: Find errors at compile time. I also had a very brief look at Doug Lees Collection package from 1995. It has two Interfaces per Collectiontype (i you leave Immutablecollections out). If those Colelctions does not indirect removal (i.e remove at Iterator) of items, this would be enough. From chris.hegarty at oracle.com Thu Feb 11 17:35:54 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 11 Feb 2016 17:35:54 +0000 Subject: RFR: 8149601 Update references from "1.9" to "9" In-Reply-To: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> References: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> Message-ID: On 11 Feb 2016, at 17:13, Iris Clark wrote: > Hi. > > Please review the following changes to the jaxp and jdk > repositories to fix a few additional references to "1.9". > > Bug > > 8149601 Update references from "1.9" to "9" > https://bugs.openjdk.java.net/browse/JDK-8149601 > > Webrev > > http://cr.openjdk.java.net/~iris/verona/8149601/webrev/ Looks fine. -Chris. From martinrb at google.com Thu Feb 11 18:49:21 2016 From: martinrb at google.com (Martin Buchholz) Date: Thu, 11 Feb 2016 10:49:21 -0800 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: <56BC63AC.9050702@oracle.com> References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> <56BB4932.7060105@oracle.com> <56BB4C24.7090607@oracle.com> <56BBD50F.8070803@oracle.com> <56BC63AC.9050702@oracle.com> Message-ID: Currently the pendulum is swinging away from multiple applications sharing common libraries towards every application being self-contained, perhaps because disk space is dirt cheap and because of the rise of "containers". It may be that much of the packaging of jdks will be picked up by third parties who package up the JDK and any of its dependencies - e.g. I already see https://hub.docker.com/_/java/ On Thu, Feb 11, 2016 at 2:34 AM, Se?n Coffey wrote: > > On 11/02/2016 00:25, Xueming Shen wrote: >> >> >> One of the benefits of moving to the system libz is actually for >> better/easy >> maintenance. Just replacing the offending version of libz with an >> earlier/later >> version that works, instead of waiting for a customized jdk/jre image with >> a >> working/bundled libz, or the next update release. Especially given the >> fact >> that we have decided not to touch the libz at source level. Having >> dependency >> on the system libz is really not that bad. The experience suggests those >> binaries are quite stable. And it should always be easier to replace the >> system libz than a java runtime, in case of problem. > > > I think people overestimate people's ability to 'just replace' a zlib > library. Adding a new system property is a struggle for some enterprises. > We'll enjoy supporting a many to one matrix of zlib:JDK scenarios now rather > than old style 1:1. It's great to have system library support - I'm just > highlighting a possible risk. A fall back option solves that but there's no > appetite for such a solution. Let's see how it goes. > > regards, > Sean. > >> >> Sherman >> >> On 02/10/2016 06:41 AM, Se?n Coffey wrote: >>> >>> >>> On 10/02/16 14:29, Alan Bateman wrote: >>>> >>>> >>>> On 10/02/2016 13:57, Se?n Coffey wrote: >>>>> >>>>> I'm all for allowing one to introduce a new version of zlib to their >>>>> JDK at runtime. I just don't think it's in the interests of enterprises and >>>>> stability to introduce a dependency to the JDK on the underlying OS zlib >>>>> libraries. Could we at least consider a runtime property to allow linking to >>>>> the (currently bundled) zlib v.1.2.8 port in case issues arise ? >>>> >>>> Don't the LD_* environment variables serve this need already? Once the >>>> JDK is using the system zlib then this is the simplest way to get it to use >>>> a different libz library at runtime. >>> >>> No - I don't see that as a solution. You've still made the default JDK >>> config become dependent on OS environment for all libzip operations. I don't >>> think we even capture the zlib version that the JDK would be operating with >>> in any diagnostics. An application wanting a tried/tested and stable libzip >>> version has extra work to do now. Letting the default be system dependent >>> has just increased risk for QA teams also. A system property just makes this >>> all go away. In fact - I would say that for JDK9, the default should be the >>> JDK bundled libzip library. For those looking for libzip experimenting and >>> performance benefits, they could take the system property approach. >>> >>> regards, >>> Sean. >>>> >>>> >>>> -Alan >>> >>> >>> >>> On 08/02/16 09:55, Alan Bateman wrote: >>>> >>>> >>>> On 08/02/2016 10:42, Se?n Coffey wrote: >>>>> >>>>> Is there an option to fall back to the older v.1.2.8 implementation if >>>>> necessary ? It would certainly alleviate issues for any applications that >>>>> might run into issues with the latest and greatest zlib libraries. >>>>> JDK-8133206 would be one such example. >>>> >>>> Just at build time >>> >>> so - we introduce a runtime dependency on the underlying zlib libraries >>> on the OS by default. I would be very concerned with this approach. We've >>> run JDK 6 for 10+ years with zlib v1.1.3. It was consistent and reliable for >>> the most part. When we moved JDK7/8 to zlib v1.2.5, we encountered an >>> inflation issue[1]. When JDK was upgraded to zlib v1.2.8, we received >>> reports of performance/OOM issues [2]. >>>> >>>> but if the zlib on the platform is broken then it impacts tools and >>>> probably lots of things. I would assume the OS would patch such issues >>>> quickly. In the case of JDK-8133206 then was the issue addressed in the >>>> libzip wrapper or in the zlib code? I thought it was the former. >>> >>> The code change is proposed in the libzip wrapper but the issue was >>> triggered by the zlib library update. >>>> >>>> >>>> On a fallback or some way to configure at launch time then Sandhya >>>> Viswanathan (Intel) has a proposal here last year. I think we mostly agreed >>>> on that thread that switching the build to use the system zlib by default >>>> would be better. >>> >>> I'm all for allowing one to introduce a new version of zlib to their JDK >>> at runtime. I just don't think it's in the interests of enterprises and >>> stability to introduce a dependency to the JDK on the underlying OS zlib >>> libraries. Could we at least consider a runtime property to allow linking to >>> the (currently bundled) zlib v.1.2.8 port in case issues arise ? >>> >>> regards, >>> Sean. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8044725 >>> [2] https://bugs.openjdk.java.net/browse/JDK-8133206 >>> >> > From iris.clark at oracle.com Thu Feb 11 19:06:40 2016 From: iris.clark at oracle.com (Iris Clark) Date: Thu, 11 Feb 2016 11:06:40 -0800 (PST) Subject: RFR: 8149601 Update references from "1.9" to "9" In-Reply-To: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> References: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> Message-ID: <1191cdc7-6751-425b-b430-4a6d86c809e1@default> Hi. A private review comment as noted that I missed a few files in the jaxp repository. The webrev has been updated in place. Thanks, iris -----Original Message----- From: Iris Clark Sent: Thursday, February 11, 2016 9:13 AM To: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Cc: Iris Clark; Kumar Srinivasan Subject: RFR: 8149601 Update references from "1.9" to "9" Hi. Please review the following changes to the jaxp and jdk repositories to fix a few additional references to "1.9". Bug 8149601 Update references from "1.9" to "9" https://bugs.openjdk.java.net/browse/JDK-8149601 Webrev http://cr.openjdk.java.net/~iris/verona/8149601/webrev/ This is a follow-up to JDK-8136494 which changed "@since 1.9" to "@since 9" matching java.specification.version. Thanks, iris From iris.clark at oracle.com Thu Feb 11 19:07:43 2016 From: iris.clark at oracle.com (Iris Clark) Date: Thu, 11 Feb 2016 11:07:43 -0800 (PST) Subject: RFR: 8149601 Update references from "1.9" to "9" In-Reply-To: References: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> Message-ID: Hi, Chris. >> http://cr.openjdk.java.net/~iris/verona/8149601/webrev/ > > Looks fine. Thanks for the Review. Regards, Iris From huizhe.wang at oracle.com Thu Feb 11 19:17:24 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 11 Feb 2016 11:17:24 -0800 Subject: RFR: 8149601 Update references from "1.9" to "9" In-Reply-To: <1191cdc7-6751-425b-b430-4a6d86c809e1@default> References: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> <1191cdc7-6751-425b-b430-4a6d86c809e1@default> Message-ID: <56BCDE44.9040700@oracle.com> Hi Iris, The change looks good. Two of the previously included classes appeared twice in the webrev index page :-) Thanks, Joe On 2/11/2016 11:06 AM, Iris Clark wrote: > Hi. > > A private review comment as noted that I missed a few files in the jaxp repository. > > The webrev has been updated in place. > > Thanks, > iris > > -----Original Message----- > From: Iris Clark > Sent: Thursday, February 11, 2016 9:13 AM > To: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net > Cc: Iris Clark; Kumar Srinivasan > Subject: RFR: 8149601 Update references from "1.9" to "9" > > Hi. > > Please review the following changes to the jaxp and jdk repositories to fix a few additional references to "1.9". > > Bug > > 8149601 Update references from "1.9" to "9" > https://bugs.openjdk.java.net/browse/JDK-8149601 > > Webrev > > http://cr.openjdk.java.net/~iris/verona/8149601/webrev/ > > This is a follow-up to JDK-8136494 which changed "@since 1.9" > to "@since 9" matching java.specification.version. > > Thanks, > iris From nadeesh.tv at oracle.com Thu Feb 11 20:22:33 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Fri, 12 Feb 2016 01:52:33 +0530 Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with short time zone offset ("+01") Message-ID: <56BCED89.7040007@oracle.com> Hi all, Please review a fix for Bug Id https://bugs.openjdk.java.net/browse/JDK-8032051 webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/ -- Thanks and Regards, Nadeesh TV From igor.ignatyev at oracle.com Thu Feb 11 21:03:09 2016 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Feb 2016 00:03:09 +0300 Subject: RFR(XXS) : 8149700 : Remove jdk_svc test group from tier * groups Message-ID: http://cr.openjdk.java.net/~iignatyev/8149700/webrev.00/ > > 2 lines changed: 0 ins; 1 del; 1 mod; Hi all, could you please review the tiny patch which removes jdk_svc test group from tier2 test group? Many of svc tests fail intermittently due to different reasons, that prevents us from having all-pass runs in tier 2+. JBS: https://bugs.openjdk.java.net/browse/JDK-8149700 Thanks, ? Igor From joe.darcy at oracle.com Fri Feb 12 00:52:25 2016 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 11 Feb 2016 16:52:25 -0800 Subject: RFR(XXS) : 8149700 : Remove jdk_svc test group from tier * groups In-Reply-To: References: Message-ID: <56BD2CC9.8040606@oracle.com> Hi Igor, The change to the test list in and of itself looks fine. Once some stable subset of the svc tests can be committed to, it would be good to have them back in tier 2. Thanks, -Joe On 2/11/2016 1:03 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev/8149700/webrev.00/ >> 2 lines changed: 0 ins; 1 del; 1 mod; > Hi all, > > could you please review the tiny patch which removes jdk_svc test group from tier2 test group? Many of svc tests fail intermittently due to different reasons, that prevents us from having all-pass runs in tier 2+. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8149700 > > Thanks, > ? Igor From mandy.chung at oracle.com Fri Feb 12 00:53:17 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Thu, 11 Feb 2016 16:53:17 -0800 Subject: RFR [9] 8149653: Move sun.misc.InnocuousThread to jdk.internal.misc In-Reply-To: References: Message-ID: <5EEC600A-C85D-4C0F-AA62-85254C47D257@oracle.com> > On Feb 11, 2016, at 6:04 AM, Chris Hegarty wrote: > > In preparation for JEP 260, InnocuousThread should be moved out of the > sun.misc package and placed into a more appropriate package where it > can be encapsulated. > > http://cr.openjdk.java.net/~chegar/8149653/ Looks good. Mandy From peter.firmstone at zeus.net.au Fri Feb 12 06:30:35 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Fri, 12 Feb 2016 16:30:35 +1000 Subject: DeserializationPermission Proposal Message-ID: <56BD7C0B.60107@zeus.net.au> Thanks David, I'd originally considered something like that, but I later realised I'd need to grant DeserializationPermission to other domains I might not want allow objects to be deserialized with, simply because their domain is included in the call stack. It looked like it would widen the scope, which was contrary to the intended goal of minimising the attack surface. In the end I decided to limit the context to only the domains of classes involved in deserialization. Cheers, Peter. > On 02/11/2016 03:52 AM, Peter Firmstone wrote: > >/ An example might be more useful. > />/ > />/ Traditionally, when the first non serializable superclass zero argument > />/ constructor is called, the domains of the subclasses aren't present on > />/ the call stack. Any security checks performed in the constructor of the > />/ superclass will not include the subclasses domains. > />/ > />/ In the proposed case, prior to construction, all domains in the class > />/ heirarchy of the to be deserialized object via the local > />/ ObjectStreamClass, are added to an AccessControlContext, which is then > />/ passed to the SecurityManager two argument permission check. > / > Sure, that makes sense; in fact this could be a very good > standalone/incremental enhancement. > > >/ Now an attacker will not be able to construct this object unless all > />/ domains have DeserializationPermission. > />/ > />/ If we use the stack context, it won't contain the yet to be deserialized > />/ classes. > / > True; perhaps the ideal solution would use the initial context plus a > per-deserializing-class context, so that both the original caller and > the class being deserialized have permissions. This would have the > advantage of consistent behavior, and would also allow the PD of each > class to restrict whether it can be deserialized (which would apply > globally no matter who was doing the deserialization in the VM). > > -- > - DML From david.lloyd at redhat.com Fri Feb 12 14:55:55 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Fri, 12 Feb 2016 08:55:55 -0600 Subject: DeserializationPermission Proposal In-Reply-To: <56BD7C0B.60107@zeus.net.au> References: <56BD7C0B.60107@zeus.net.au> Message-ID: <56BDF27B.9080405@redhat.com> Things are getting confused (for me anyway) so I'm going to try and spell out each idea separately. Given a class hierarchy H of: C -> B -> A -> NonSerializableClass -> Object where C, B, and A are serializable classes, and protection domain M which is initiating deserialization, I think the following requirements definitely apply: * When M deserializes an instance of A, both M and the protection domain of A must be sufficient to allow the deserialization of A * When M deserializes an instance of B, both M and the protection domain of B must be sufficient to allow the deserialization of B * When M deserializes an instance of C, both M and the protection domain of C must be sufficient to allow the deserialization of C * Extrapolate generally The following requirements have definite pros and cons, which is what I think we're touching on in this thread: * When M deserializes an instance of B (or any subtype thereof), the protection domain of A must also be sufficient to allow the deserialization of B * When M deserializes an instance of C (or any subtype thereof), the protection domains of both A and B must also be sufficient to allow the deserialization of C * Extrapolate generally Based on the above, the following requirements might or might not make sense: * When M deserializes an instance of A, the protection domains of NonSerializableClass and also Object must also be sufficient to allow the deserialization of A * Likewise for B & C, extrapolate generally Regardless of which is the best set of requirements, one of the major complicating questions is, what is the ideal representation of the permission(s) needed to allow the above? I can see several use cases: 1. Grant the ability for a serializable class S to deserialize itself 2. Grant the ability for M to deserialize a specific class (by name?) 3. Grant the ability to any (serializable?) class X to allow deserialization of a specific subtype (by name?) 4. Grant the ability to any (serializable?) class X to allow deserialization of any subtype (think java.lang.Object, or general (but harmless) base classes like java.util.EventObject) The big problem with permissions by name is that names are completely relative. It is totally valid (and happens in the wild today) that a serialization graph contains classes from a variety of class loaders which are isolated from one another - in fact, it's even possible to have more than one class with the same class name and the same superclass in the same graph! Given this fact, a flat class namespace *cannot* be assumed. I think that the *only* possible remedy for this is that it should be up to each class' class loader (and/or module) to determine what permission is required in order to deserialize that class. In this way, the JDK (9) can use a permission of the form class+module name for installed module classes, Java EE systems could use permissions in the form of application+module+jar+class name for deployment, OSGi systems could use permissions including bundle identification information, etc. To be secure, the default class loader implementation of this method would have to produce a result which causes all checks to fail, preventing deserialization in a security manager environment of classes whose class loaders do not explicitly allow for it. On 02/12/2016 12:30 AM, Peter Firmstone wrote: > Thanks David, > > I'd originally considered something like that, but I later realised I'd > need to grant DeserializationPermission to other domains I might not > want allow objects to be deserialized with, simply because their domain > is included in the call stack. It looked like it would widen the scope, > which was contrary to the intended goal of minimising the attack > surface. In the end I decided to limit the context to only the domains > of classes involved in deserialization. > > Cheers, > > Peter. > >> On 02/11/2016 03:52 AM, Peter Firmstone wrote: >> >/ An example might be more useful. >> />/ >> />/ Traditionally, when the first non serializable superclass zero >> argument >> />/ constructor is called, the domains of the subclasses aren't >> present on >> />/ the call stack. Any security checks performed in the constructor >> of the >> />/ superclass will not include the subclasses domains. >> />/ >> />/ In the proposed case, prior to construction, all domains in the >> class >> />/ heirarchy of the to be deserialized object via the local >> />/ ObjectStreamClass, are added to an AccessControlContext, which is >> then >> />/ passed to the SecurityManager two argument permission check. >> / >> Sure, that makes sense; in fact this could be a very good >> standalone/incremental enhancement. >> >> >/ Now an attacker will not be able to construct this object unless all >> />/ domains have DeserializationPermission. >> />/ >> />/ If we use the stack context, it won't contain the yet to be >> deserialized >> />/ classes. >> / >> True; perhaps the ideal solution would use the initial context plus a >> per-deserializing-class context, so that both the original caller and >> the class being deserialized have permissions. This would have the >> advantage of consistent behavior, and would also allow the PD of each >> class to restrict whether it can be deserialized (which would apply >> globally no matter who was doing the deserialization in the VM). >> >> -- >> - DML -- - DML From chris.hegarty at oracle.com Fri Feb 12 15:45:00 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 12 Feb 2016 15:45:00 +0000 Subject: RFR [9 ] 8149656: Examine usages of sun.misc.LRUCache Message-ID: <56BDFDFC.1040004@oracle.com> sun.misc.LRUCache is a utility class for implementing Least Recently Used Caches. It is only used by j.u.Scanner in the JDK. It should be relocated to a more suitable location, possibly an inner class of Scanner. http://cr.openjdk.java.net/~chegar/8149656/ -Chris. From claes.redestad at oracle.com Fri Feb 12 15:59:45 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 12 Feb 2016 16:59:45 +0100 Subject: RFR [9 ] 8149656: Examine usages of sun.misc.LRUCache In-Reply-To: <56BDFDFC.1040004@oracle.com> References: <56BDFDFC.1040004@oracle.com> Message-ID: <56BE0171.5050005@oracle.com> +1 nits: I guess the class should be made static if nested here, or even made concrete to get rid of the anonymous class - that depends on if we think we'll ever reuse this utility for other things. /Claes On 2016-02-12 16:45, Chris Hegarty wrote: > sun.misc.LRUCache is a utility class for implementing Least Recently > Used Caches. It is only used by j.u.Scanner in the JDK. It should be > relocated to a more suitable location, possibly an inner class of > Scanner. > > http://cr.openjdk.java.net/~chegar/8149656/ > > -Chris. From ecki at zusammenkunft.net Fri Feb 12 16:05:03 2016 From: ecki at zusammenkunft.net (ecki at zusammenkunft.net) Date: Fri, 12 Feb 2016 17:05:03 +0100 Subject: RFR [9 ] 8149656: Examine usages of sun.misc.LRUCache In-Reply-To: <56BDFDFC.1040004@oracle.com> References: <56BDFDFC.1040004@oracle.com> Message-ID: Since it is only implemented with one anonymous class you could also think about making it a concrete PatternLRUCache -- http://bernd.eckenfels.net -----Original Message----- From: Chris Hegarty To: Core-Libs-Dev Sent: Fr., 12 Feb. 2016 16:55 Subject: RFR [9 ] 8149656: Examine usages of sun.misc.LRUCache sun.misc.LRUCache is a utility class for implementing Least Recently Used Caches. It is only used by j.u.Scanner in the JDK. It should be relocated to a more suitable location, possibly an inner class of Scanner. http://cr.openjdk.java.net/~chegar/8149656/ -Chris. From paul.sandoz at oracle.com Fri Feb 12 16:17:08 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 12 Feb 2016 17:17:08 +0100 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> References: <661AD1C3-AA58-452A-9F16-8254D876A367@oracle.com> <5640572E.8070207@oracle.com> <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC! .7080102@oracle.com> <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> Message-ID: <9F05153C-C6C3-481E-B321-DC1A425031DA@oracle.com> Hi Steve, This patch has been cooking and re-heated enough times that i think it?s as good as done in terms of being reviewed. We can follow up with further issues for other aspects, such as performance if need be. Paul. > On 10 Feb 2016, at 02:04, Steve Drach wrote: > > Hi, > > Yet another webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.06/ , with a change to JarEntryIterator to fix a problem discovered by performance tests ? calling hasNext() twice as often as needed. I also removed the @since 9 tags from the methods entries() and stream(), and added an additional sentence to the spec for each of those methods that clarifies what a base entry is (actually is not). > >> I think having stream and entries do this is right although I would like to see some performance data if possible. > > See http://cr.openjdk.java.net/~sdrach/8132734/JarFile%20Performance.pdf > > I used JMH to run the benchmark. See http://cr.openjdk.java.net/~sdrach/8132734/MyBenchmark.java . I used two jar files, the rt.jar file from JDK 8 that has 20653 entries and the multi-release.jar found in the test directory with 14 entries. Obviously rt.jar is not a multi-release jar file. > > The first two tables (1 and 2) are comparable and the second two tables are somewhat comparable (3 and 4). > > Tables 1 and 2 have 4 sections that show the results of tests on the two jar files in 4 configurations of JarFile. The tests were done with a JarFile object constructed without the Release object argument, essentially the legacy constructor. The section labeled "JDK 8 JarFile? was done with JDK 8u66. The section labeled ?JDK 9 JarFile? was done with the latest build of openjdk/jdk9/dev without any changes in my 8132734 changeset. I chose this section as the reference, so the last column shows the values normalized to 1 micro/milli second per operation (rt.jar times are in milliseconds and multi-release.jar times are in microseconds). It should be obvious that JDK 9 is much faster than JDK 8, somewhere on the order of 5-6 times faster. I think that is because ZipFile doesn?t use JNI in JDK 9. > > Of the two remaining sections in Tables 1 and 2, the section labeled ?MultiRelease JarFile? differs from the section labeled ?MultiRelease JarFile, new iterator? only in the JarEntryIterator class. The first one uses the original iterator in JarFile.java that can be seen starting with line 551 of webrev.04 , and the new iterator starts with line 553 of webrev.06 . The results are strange. The new, more complex, iterator appears to be faster than the old, simpler, iterator. I double, and triple, checked it, but it was always faster. I used jitwatch to look at the hotspot logs generated during compilation and neither method was compiled. I suppose I could dig into it further but decided not to. Consider it good news. The results do show that the multi-release enhancement slows JarFile entries/stream down by 2-18% depending on the size of the jar file. But they are still much better than the JDK 8 values. > >> Also I would expect that if a JAR file is not mult-release but the library opens it with Release.RUNTIME to perform the same as opening the JAR file with the Release-less constructors. > > The results in Table 3 attempts to answer this question since rt.jar is not a multi-release jar file. This tells me that if one opens the JarFile with Release.RUNTIME, that there is a performance penalty of 2-6% on this very large jar file. > > Finally, the results in Table 4 tell me that processing a true multi-release jar file takes about 80% more time per entries() or stream() operation. I?ve looked at this in a profiler and there is no particular area that stands out to me, it?s just more complicated to process a multi-release jar file, as would be expected. > >> I think the javadoc will need to also need to make it clear whether entries with names starting with META-INF/versions/ are returned. > > Fixed. > >> >> I see you've added @since 9 to the existing methods, I assume you didn't mean to do this. > > Fixed. > >> >> At some point then we need to discuss how RUNTIME_VERSION is computed. Iris (via Mandy) has pushed jdk.Version to jdk9/dev but having it exported by java.base conflicts with the design principles in JEP 200. Moving it to another module means that code in java.base cannot use it and thus the JAR file can't use it. > > Left as is. > From chris.hegarty at oracle.com Fri Feb 12 16:26:30 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 12 Feb 2016 16:26:30 +0000 Subject: RFR [9 ] 8149656: Examine usages of sun.misc.LRUCache In-Reply-To: References: <56BDFDFC.1040004@oracle.com> Message-ID: <56BE07B6.8010605@oracle.com> Thanks for the feedback. On 12/02/16 16:05, ecki at zusammenkunft.net wrote: > Since it is only implemented with one anonymous class you could also think about making it a concrete PatternLRUCache On 12/02/16 15:59, Claes Redestad wrote: > +1 > > nits: I guess the class should be made static if nested here, or even > made concrete to get rid of the anonymous class - that depends on if we > think we'll ever reuse this utility for other things. With all these changes, my opening position is to keep things the same. Accepting the move, then there is of course much refactoring/cleanup that can be done. Here is the next iteration, with your suggestions. http://cr.openjdk.java.net/~chegar/8149656.01/ -Chris. From claes.redestad at oracle.com Fri Feb 12 16:30:31 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 12 Feb 2016 17:30:31 +0100 Subject: RFR [9 ] 8149656: Examine usages of sun.misc.LRUCache In-Reply-To: <56BE07B6.8010605@oracle.com> References: <56BDFDFC.1040004@oracle.com> <56BE07B6.8010605@oracle.com> Message-ID: <56BE08A7.4020907@oracle.com> On 2016-02-12 17:26, Chris Hegarty wrote: > > http://cr.openjdk.java.net/~chegar/8149656.01/ This looks great, thanks! /Claes From Roger.Riggs at Oracle.com Fri Feb 12 16:45:17 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Feb 2016 11:45:17 -0500 Subject: RFR [9 ] 8149656: Examine usages of sun.misc.LRUCache In-Reply-To: <56BE08A7.4020907@oracle.com> References: <56BDFDFC.1040004@oracle.com> <56BE07B6.8010605@oracle.com> <56BE08A7.4020907@oracle.com> Message-ID: <56BE0C1D.2020803@Oracle.com> +1 On 2/12/2016 11:30 AM, Claes Redestad wrote: > On 2016-02-12 17:26, Chris Hegarty wrote: >> >> http://cr.openjdk.java.net/~chegar/8149656.01/ > > This looks great, thanks! > > /Claes From Roger.Riggs at Oracle.com Fri Feb 12 17:02:07 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Feb 2016 12:02:07 -0500 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module Message-ID: <56BE100F.9060604@Oracle.com> Please review moving the functionality of sun.misc.Signal to jdk.internal.misc and creating a wrapper sun.misc.Signal for existing external uses. A new replacement API will be considered separately. The update includes a test that passes with or without the changes. (Except for an NPE instead of SEGV if null is passed). Webrev: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ Issue: https://bugs.openjdk.java.net/browse/JDK-8149750 Thanks, Roger JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 From vladimir.x.ivanov at oracle.com Fri Feb 12 17:15:38 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 12 Feb 2016 20:15:38 +0300 Subject: [9] RFR (S): 8148994: Replacing MH::invokeBasic with a direct call breaks LF customization In-Reply-To: <56B4CE00.3040809@oracle.com> References: <56B37387.2040204@oracle.com> <6CCE2F2C-1836-47CC-B13E-4371C2CE61FF@oracle.com> <56B4CE00.3040809@oracle.com> Message-ID: <56BE133A.70508@oracle.com> Any reviews, please? Best regards, Vladimir Ivanov On 2/5/16 7:29 PM, Vladimir Ivanov wrote: > Thanks, John. > > On 2/4/16 9:52 PM, John Rose wrote: >> Looks good. Except I don't recall how is_inline interacts, if at all, >> with customization. It's not clear what is being gated, and which >> optimizations get let through. > Inlining through MH::invokeBasic doesn't happen when there's @DontInline > on the target. It is the case for GWT which has CountingWrapper on both > paths. It starts with a reinvoker.dontInline LF and then switches to a > reinvoker LF (after 30 invocations). If a compilation happens when > reinvoker.dontInline is used, a direct call is generated and even if the > branch becomes hot, all the calls goes through reinvoker.dontInline. > Update to reinvoker LF isn't visible unless the dispatch goes through > MH::invokeBasic linker. > > I had an impression that I introduced LF customization logic to the > non-inlined case in GWT, but I can't find it in the code. So, it seems > the regression is caused by repeated checks performed by > reinvoker.dontInline LF and not by LF customization missed in generated > code. > > Also, I stumbled upon a quirk in inlining policy: inlining through > linkers can't be delayed. It happens when JIT hits some limit on IR > size. So, right now inlining can stop in the middle of MH chain and a > linker call is generated. > > But after JDK-8072008 there's no problem with delaying inlining. C2 can > decide whether to keep the direct call or inline through it. So, I > enabled late inlining for all linkers. (Surprisingly, no significant > performance difference on nashorn.) > > So, I decided to keep invokeBasic linker call replacement for now, but > piggypack LF customization on the stuck counting LF. > > Updated webrev: > http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/hotspot > http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/jdk > > * enabled late inlining for linkers; > * add LF customization check in CountingWrapper::countDown, so when > counting is stuck, the target will be customized. > > Testing: octane, JPRT. > > I'm working on JDK-8071793 [1] and will reconsider how LF customization > on non-inlined branch in GWTs works given there is a more JVM-friendly > mechanism to control inlining. With the proposed fix > reinvoker.dontInline is never customized (only the target it points to) > and serves as a trampoline between GWT & target LF. It would be better > to jump right to the customized LF, if non-inlined branch becomes hot. > > Best regards, > Vladimir Ivanov > > [1] https://bugs.openjdk.java.net/browse/JDK-8071793 > "Prune rarely called invokedynamic call sites during inlining" >> >> Maybe a comment about that would be useful >> Reviewed. >> >> ? John >> >>> On Feb 4, 2016, at 7:51 AM, Vladimir Ivanov >>> wrote: >>> >>> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.00 >>> https://bugs.openjdk.java.net/browse/JDK-8148994 >>> >>> JDK-8072008 [1] introduced an optimization to bypass linker calls in >>> generated code when target method can't be inlined. >>> >>> Unfortunately, it doesn't work well with MH::invokeBasic(), because >>> it clashes with LambdaForm customization: if a call is not inlined, >>> it is important for performance to update (customize) corresponding >>> LambdaForm on-the-fly. MH::invokeBasic() linker does MH -> LF -> >>> Method* -> _from_compiled_entry traversal on every invocation, while >>> direct call points right to the _from_compiled_entry. So, any >>> LambdaForm updates aren't visible until recompilation. >>> >>> The fix is to keep MH::invokeBasic linker, so up-to-date LambdaForm >>> instance is used on every invocation. >>> >>> Also, cleaned up relevant tests a bit. >>> >>> Testing: octane, JPRT. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8072008 >>> "Emit direct call instead of linkTo* for recursive >>> indy/MH.invoke* calls" From Roger.Riggs at Oracle.com Fri Feb 12 17:47:48 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Feb 2016 12:47:48 -0500 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56BE100F.9060604@Oracle.com> References: <56BE100F.9060604@Oracle.com> Message-ID: <56BE1AC4.2040405@Oracle.com> Please review moving the functionality of sun.misc.Signal to jdk.internal.misc and creating a wrapper sun.misc.Signal for existing external uses. +++ This time including the hotspot changes to update the target of the upcalls. A new replacement API will be considered separately. The update includes a test that passes with or without the changes. (Except for an NPE instead of SEGV if null is passed). Webrev: jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ Issue: https://bugs.openjdk.java.net/browse/JDK-8149750 Thanks, Roger JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 From iris.clark at oracle.com Fri Feb 12 17:57:57 2016 From: iris.clark at oracle.com (Iris Clark) Date: Fri, 12 Feb 2016 09:57:57 -0800 (PST) Subject: RFR: 8149601 Update references from "1.9" to "9" In-Reply-To: <56BCDE44.9040700@oracle.com> References: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> <1191cdc7-6751-425b-b430-4a6d86c809e1@default> <56BCDE44.9040700@oracle.com> Message-ID: <9f2996c9-d7a3-41e2-8dea-fd6a875b0006@default> Hi, Joe. Thanks for the review. Regards, iris -----Original Message----- From: huizhe wang Sent: Thursday, February 11, 2016 11:17 AM To: Iris Clark Cc: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Subject: Re: RFR: 8149601 Update references from "1.9" to "9" Hi Iris, The change looks good. Two of the previously included classes appeared twice in the webrev index page :-) Thanks, Joe On 2/11/2016 11:06 AM, Iris Clark wrote: > Hi. > > A private review comment as noted that I missed a few files in the jaxp repository. > > The webrev has been updated in place. > > Thanks, > iris > > -----Original Message----- > From: Iris Clark > Sent: Thursday, February 11, 2016 9:13 AM > To: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net > Cc: Iris Clark; Kumar Srinivasan > Subject: RFR: 8149601 Update references from "1.9" to "9" > > Hi. > > Please review the following changes to the jaxp and jdk repositories to fix a few additional references to "1.9". > > Bug > > 8149601 Update references from "1.9" to "9" > https://bugs.openjdk.java.net/browse/JDK-8149601 > > Webrev > > http://cr.openjdk.java.net/~iris/verona/8149601/webrev/ > > This is a follow-up to JDK-8136494 which changed "@since 1.9" > to "@since 9" matching java.specification.version. > > Thanks, > iris From steve.drach at oracle.com Fri Feb 12 21:11:21 2016 From: steve.drach at oracle.com (Steve Drach) Date: Fri, 12 Feb 2016 13:11:21 -0800 Subject: RFR JDK-8149769: Null pointer exception in ZipFileSystemProvider Message-ID: Hi, Please review this simple fix to ZipFileSystemProvider. The issue is JDK-8149769 . I didn?t do a webrev but instead provide the following patch. Thanks Steve diff -r 2d6c2c75f338 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Tue Feb 09 14:07:28 2016 -0800 +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Fri Feb 12 12:59:46 2016 -0800 @@ -100,7 +100,7 @@ } ZipFileSystem zipfs = null; try { - if (env.containsKey("multi-release")) { + if (env != null && env.containsKey("multi-release")) { zipfs = new JarFileSystem(this, path, env); } else { zipfs = new ZipFileSystem(this, path, env); From roger.riggs at oracle.com Fri Feb 12 21:22:38 2016 From: roger.riggs at oracle.com (Roger Riggs) Date: Fri, 12 Feb 2016 16:22:38 -0500 Subject: RFR JDK-8149769: Null pointer exception in ZipFileSystemProvider In-Reply-To: References: Message-ID: <56BE4D1E.1070706@oracle.com> Look fine. Roger On 2/12/16 4:11 PM, Steve Drach wrote: > Hi, > > Please review this simple fix to ZipFileSystemProvider. The issue is JDK-8149769 . I didn?t do a webrev but instead provide the following patch. > > Thanks > Steve > > diff -r 2d6c2c75f338 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java > --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Tue Feb 09 14:07:28 2016 -0800 > +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Fri Feb 12 12:59:46 2016 -0800 > @@ -100,7 +100,7 @@ > } > ZipFileSystem zipfs = null; > try { > - if (env.containsKey("multi-release")) { > + if (env != null && env.containsKey("multi-release")) { > zipfs = new JarFileSystem(this, path, env); > } else { > zipfs = new ZipFileSystem(this, path, env); From Alan.Bateman at oracle.com Fri Feb 12 21:33:11 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Feb 2016 21:33:11 +0000 Subject: RFR JDK-8149769: Null pointer exception in ZipFileSystemProvider In-Reply-To: References: Message-ID: <56BE4F97.2070203@oracle.com> On 12/02/2016 21:11, Steve Drach wrote: > Hi, > > Please review this simple fix to ZipFileSystemProvider. The issue is JDK-8149769 . I didn?t do a webrev but instead provide the following patch. > > This looks okay. Can one of the existing tests be updated to cover this case? -Alan From iris.clark at oracle.com Fri Feb 12 21:47:36 2016 From: iris.clark at oracle.com (Iris Clark) Date: Fri, 12 Feb 2016 13:47:36 -0800 (PST) Subject: RFR: 8149601 Update references from "1.9" to "9" In-Reply-To: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> References: <808e959b-aa71-4eec-a056-cb6e08f6d251@default> Message-ID: <81dd015c-6738-496e-9bee-79811d3f8fbc@default> Hi, Kumar. Thank you so much for pushing changesets to fix this bug. Regards, iris -----Original Message----- From: Iris Clark Sent: Thursday, February 11, 2016 9:13 AM To: core-libs-dev at openjdk.java.net; verona-dev at openjdk.java.net Cc: Iris Clark; Kumar Srinivasan Subject: RFR: 8149601 Update references from "1.9" to "9" Hi. Please review the following changes to the jaxp and jdk repositories to fix a few additional references to "1.9". Bug 8149601 Update references from "1.9" to "9" https://bugs.openjdk.java.net/browse/JDK-8149601 Webrev http://cr.openjdk.java.net/~iris/verona/8149601/webrev/ This is a follow-up to JDK-8136494 which changed "@since 1.9" to "@since 9" matching java.specification.version. Thanks, iris From steve.drach at oracle.com Fri Feb 12 22:04:43 2016 From: steve.drach at oracle.com (Steve Drach) Date: Fri, 12 Feb 2016 14:04:43 -0800 Subject: RFR JDK-8149769: Null pointer exception in ZipFileSystemProvider In-Reply-To: <56BE4F97.2070203@oracle.com> References: <56BE4F97.2070203@oracle.com> Message-ID: >> Please review this simple fix to ZipFileSystemProvider. The issue is JDK-8149769 . I didn?t do a webrev but instead provide the following patch. >> >> > This looks okay. Can one of the existing tests be updated to cover this case? Yes, almost any of them can be updated. There is one, ZipFSTester.java that seems to be associated with a lot of bugs, so maybe that one. Or I can put it in MultiReleaseJarTest, which makes sense because the code it tests was the code that caused the bug. Any preferences? From Roger.Riggs at Oracle.com Fri Feb 12 22:42:18 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Feb 2016 17:42:18 -0500 Subject: RFR 9:8148775 : Spec for j.l.ProcessBuilder.Redirect.DISCARD need to be improved Message-ID: <56BE5FCA.5050609@Oracle.com> Please review a minor javadoc cleanup to remove a reference to a method (append) of the implementation. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-discard-8148775/ Issue: https://bugs.openjdk.java.net/browse/JDK-8148775 Thanks, Roger From martinrb at google.com Fri Feb 12 22:48:37 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 12 Feb 2016 14:48:37 -0800 Subject: RFR 9:8148775 : Spec for j.l.ProcessBuilder.Redirect.DISCARD need to be improved In-Reply-To: <56BE5FCA.5050609@Oracle.com> References: <56BE5FCA.5050609@Oracle.com> Message-ID: looks good. The previous line Redirect.DISCARD.file() the filename appropriate for the operating system 557 * and may be null && could use the word "is" On Fri, Feb 12, 2016 at 2:42 PM, Roger Riggs wrote: > Please review a minor javadoc cleanup to remove a reference to a method > (append) > of the implementation. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-discard-8148775/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8148775 > > Thanks, Roger > From xueming.shen at oracle.com Fri Feb 12 22:58:36 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 12 Feb 2016 14:58:36 -0800 Subject: RFR JDK-8149769: Null pointer exception in ZipFileSystemProvider In-Reply-To: References: Message-ID: <56BE639C.2030803@oracle.com> Steve, What's the issue? The bug description only includes the fix. If the env is null, shouldn't it trigger a NPE? The java.nio.file.spi package does have the note that "NPE, unless otherwise noted ..." The api for FilesystemProvider.newFileSystem(..., env) says "env" can be empty, means NPE for "null". Maybe I miss something here? -Sherman On 2/12/16 1:11 PM, Steve Drach wrote: > Hi, > > Please review this simple fix to ZipFileSystemProvider. The issue is JDK-8149769 . I didn?t do a webrev but instead provide the following patch. > > Thanks > Steve > > diff -r 2d6c2c75f338 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java > --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Tue Feb 09 14:07:28 2016 -0800 > +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Fri Feb 12 12:59:46 2016 -0800 > @@ -100,7 +100,7 @@ > } > ZipFileSystem zipfs = null; > try { > - if (env.containsKey("multi-release")) { > + if (env != null && env.containsKey("multi-release")) { > zipfs = new JarFileSystem(this, path, env); > } else { > zipfs = new ZipFileSystem(this, path, env); From Roger.Riggs at Oracle.com Fri Feb 12 23:08:19 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Feb 2016 18:08:19 -0500 Subject: RFR 9:8148775 : Spec for j.l.ProcessBuilder.Redirect.DISCARD need to be improved In-Reply-To: References: <56BE5FCA.5050609@Oracle.com> Message-ID: <56BE65E3.2010706@Oracle.com> Thanks, Martin Updated in place. On 2/12/2016 5:48 PM, Martin Buchholz wrote: > looks good. > The previous line > > Redirect.DISCARD.file() the filename appropriate for the operating system > 557 * and may be null && > > could use the word "is" > > On Fri, Feb 12, 2016 at 2:42 PM, Roger Riggs wrote: >> Please review a minor javadoc cleanup to remove a reference to a method >> (append) >> of the implementation. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-discard-8148775/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8148775 >> >> Thanks, Roger >> From martinrb at google.com Fri Feb 12 23:10:18 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 12 Feb 2016 15:10:18 -0800 Subject: RFR 9:8148775 : Spec for j.l.ProcessBuilder.Redirect.DISCARD need to be improved In-Reply-To: <56BE65E3.2010706@Oracle.com> References: <56BE5FCA.5050609@Oracle.com> <56BE65E3.2010706@Oracle.com> Message-ID: Still looks good! On Fri, Feb 12, 2016 at 3:08 PM, Roger Riggs wrote: > Thanks, Martin > > Updated in place. > > > > > On 2/12/2016 5:48 PM, Martin Buchholz wrote: > > looks good. > The previous line > > Redirect.DISCARD.file() the filename appropriate for the operating system > 557 * and may be null && > > could use the word "is" > > On Fri, Feb 12, 2016 at 2:42 PM, Roger Riggs wrote: > > Please review a minor javadoc cleanup to remove a reference to a method > (append) > of the implementation. > > Webrev: > http://cr.openjdk.java.net/~rriggs/webrev-discard-8148775/ > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8148775 > > Thanks, Roger > > From claes.redestad at oracle.com Fri Feb 12 23:12:11 2016 From: claes.redestad at oracle.com (Claes Redestad) Date: Sat, 13 Feb 2016 00:12:11 +0100 Subject: [9] RFR (S): 8148994: Replacing MH::invokeBasic with a direct call breaks LF customization In-Reply-To: <56BE133A.70508@oracle.com> References: <56B37387.2040204@oracle.com> <6CCE2F2C-1836-47CC-B13E-4371C2CE61FF@oracle.com> <56B4CE00.3040809@oracle.com> <56BE133A.70508@oracle.com> Message-ID: <56BE66CB.4080503@oracle.com> Hi, this seems OK to me, and I've verified that this gets octane performance back to 9-b102 levels across a number of platforms. Thanks! /Claes On 2016-02-12 18:15, Vladimir Ivanov wrote: > Any reviews, please? > > Best regards, > Vladimir Ivanov > > On 2/5/16 7:29 PM, Vladimir Ivanov wrote: >> Thanks, John. >> >> On 2/4/16 9:52 PM, John Rose wrote: >>> Looks good. Except I don't recall how is_inline interacts, if at all, >>> with customization. It's not clear what is being gated, and which >>> optimizations get let through. >> Inlining through MH::invokeBasic doesn't happen when there's @DontInline >> on the target. It is the case for GWT which has CountingWrapper on both >> paths. It starts with a reinvoker.dontInline LF and then switches to a >> reinvoker LF (after 30 invocations). If a compilation happens when >> reinvoker.dontInline is used, a direct call is generated and even if the >> branch becomes hot, all the calls goes through reinvoker.dontInline. >> Update to reinvoker LF isn't visible unless the dispatch goes through >> MH::invokeBasic linker. >> >> I had an impression that I introduced LF customization logic to the >> non-inlined case in GWT, but I can't find it in the code. So, it seems >> the regression is caused by repeated checks performed by >> reinvoker.dontInline LF and not by LF customization missed in generated >> code. >> >> Also, I stumbled upon a quirk in inlining policy: inlining through >> linkers can't be delayed. It happens when JIT hits some limit on IR >> size. So, right now inlining can stop in the middle of MH chain and a >> linker call is generated. >> >> But after JDK-8072008 there's no problem with delaying inlining. C2 can >> decide whether to keep the direct call or inline through it. So, I >> enabled late inlining for all linkers. (Surprisingly, no significant >> performance difference on nashorn.) >> >> So, I decided to keep invokeBasic linker call replacement for now, but >> piggypack LF customization on the stuck counting LF. >> >> Updated webrev: >> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/hotspot >> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/jdk >> >> * enabled late inlining for linkers; >> * add LF customization check in CountingWrapper::countDown, so when >> counting is stuck, the target will be customized. >> >> Testing: octane, JPRT. >> >> I'm working on JDK-8071793 [1] and will reconsider how LF customization >> on non-inlined branch in GWTs works given there is a more JVM-friendly >> mechanism to control inlining. With the proposed fix >> reinvoker.dontInline is never customized (only the target it points to) >> and serves as a trampoline between GWT & target LF. It would be better >> to jump right to the customized LF, if non-inlined branch becomes hot. >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8071793 >> "Prune rarely called invokedynamic call sites during inlining" >>> >>> Maybe a comment about that would be useful >>> Reviewed. >>> >>> ? John >>> >>>> On Feb 4, 2016, at 7:51 AM, Vladimir Ivanov >>>> wrote: >>>> >>>> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.00 >>>> https://bugs.openjdk.java.net/browse/JDK-8148994 >>>> >>>> JDK-8072008 [1] introduced an optimization to bypass linker calls in >>>> generated code when target method can't be inlined. >>>> >>>> Unfortunately, it doesn't work well with MH::invokeBasic(), because >>>> it clashes with LambdaForm customization: if a call is not inlined, >>>> it is important for performance to update (customize) corresponding >>>> LambdaForm on-the-fly. MH::invokeBasic() linker does MH -> LF -> >>>> Method* -> _from_compiled_entry traversal on every invocation, while >>>> direct call points right to the _from_compiled_entry. So, any >>>> LambdaForm updates aren't visible until recompilation. >>>> >>>> The fix is to keep MH::invokeBasic linker, so up-to-date LambdaForm >>>> instance is used on every invocation. >>>> >>>> Also, cleaned up relevant tests a bit. >>>> >>>> Testing: octane, JPRT. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8072008 >>>> "Emit direct call instead of linkTo* for recursive >>>> indy/MH.invoke* calls" From steve.drach at oracle.com Fri Feb 12 23:55:26 2016 From: steve.drach at oracle.com (Steve Drach) Date: Fri, 12 Feb 2016 15:55:26 -0800 Subject: RFR JDK-8149769: Null pointer exception in ZipFileSystemProvider In-Reply-To: <56BE639C.2030803@oracle.com> References: <56BE639C.2030803@oracle.com> Message-ID: <95F5813F-84BD-4F31-8732-8AF4CCBCD582@oracle.com> > What's the issue? The bug description only includes the fix. If the env is null, shouldn't > it trigger a NPE? > > The java.nio.file.spi package does have the note that "NPE, unless otherwise noted ..." > The api for FilesystemProvider.newFileSystem(..., env) says "env" can be empty, means > NPE for "null". > > Maybe I miss something here? No, you are not missing anything, I was. I see ZipFileSystem will throw the NPE instead of ZipFileSystemProvider if my ?fix? goes in, so it?ll still happen. I rescind my request and will close the bug report with appropriate comment. > > -Sherman > > On 2/12/16 1:11 PM, Steve Drach wrote: >> Hi, >> >> Please review this simple fix to ZipFileSystemProvider. The issue is JDK-8149769 . I didn?t do a webrev but instead provide the following patch. >> >> Thanks >> Steve >> >> diff -r 2d6c2c75f338 src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java >> --- a/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Tue Feb 09 14:07:28 2016 -0800 >> +++ b/src/jdk.zipfs/share/classes/jdk/nio/zipfs/ZipFileSystemProvider.java Fri Feb 12 12:59:46 2016 -0800 >> @@ -100,7 +100,7 @@ >> } >> ZipFileSystem zipfs = null; >> try { >> - if (env.containsKey("multi-release")) { >> + if (env != null && env.containsKey("multi-release")) { >> zipfs = new JarFileSystem(this, path, env); >> } else { >> zipfs = new ZipFileSystem(this, path, env); > From xueming.shen at oracle.com Sat Feb 13 01:19:19 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 12 Feb 2016 17:19:19 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> References: <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC.7080102@oracle.com> <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> Message-ID: <56BE8497.8020207@oracle.com> Steve, I would assume the difference of the webrev.04 "old" iterator and the webrev.06 "new" iterator that might trigger a performance is how you create the JarFileEntry. The one parameter constructor invokes isMultiRelease(), which might be relatively expensive, when the "mr" is enabled. There are couple "if" checks involved. For the "new" iterator is slower than the current jdk9 one. It might be desired to have two JarEntryIterator classes defined, one for "notVersioned", one for the "versioned". Of course keep the two parameter constructor for the "notVersioned". This might bring performance back for the normal "notVersioned" usage, which I would assume is the majority. There is also a "bug" in the new iterator. The iterator/Enumeration returned from ZipFile.entries() checks "ensureOpen() in both hasNext() and next(). So close() the ZipFile after itr.hasNext() will fails the next next() invocation in the old implementation. The latest itr will returns the cached "ze". Not a big issue for sure, but a kind of "regression". Defining two iterator classes as suggested above will also workaround this issue, as the "notVersioned" one will work just as expected, no regression. -Sherman On 2/9/16 5:04 PM, Steve Drach wrote: > Hi, > > Yet another webrev, > http://cr.openjdk.java.net/~sdrach/8132734/webrev.06/ > , with a > change to JarEntryIterator to fix a problem discovered by performance > tests ? calling hasNext() twice as often as needed. I also removed > the @since 9 tags from the methods entries() and stream(), and added > an additional sentence to the spec for each of those methods that > clarifies what a base entry is (actually is not). > >> I think having stream and entries do this is right although I would >> like to see some performance data if possible. > > See > http://cr.openjdk.java.net/~sdrach/8132734/JarFile%20Performance.pdf > > > I used JMH to run the benchmark. See > http://cr.openjdk.java.net/~sdrach/8132734/MyBenchmark.java > . I > used two jar files, the rt.jar file from JDK 8 that has 20653 entries > and the multi-release.jar found in the test directory with 14 entries. > Obviously rt.jar is not a multi-release jar file. > > The first two tables (1 and 2) are comparable and the second two > tables are somewhat comparable (3 and 4). > > Tables 1 and 2 have 4 sections that show the results of tests on the > two jar files in 4 configurations of JarFile. The tests were done > with a JarFile object constructed without the Release object argument, > essentially the legacy constructor. The section labeled "JDK 8 > JarFile? was done with JDK 8u66. The section labeled ?JDK 9 JarFile? > was done with the latest build of openjdk/jdk9/dev without any changes > in my 8132734 changeset. I chose this section as the reference, so > the last column shows the values normalized to 1 micro/milli second > per operation (rt.jar times are in milliseconds and multi-release.jar > times are in microseconds). It should be obvious that JDK 9 is much > faster than JDK 8, somewhere on the order of 5-6 times faster. I > think that is because ZipFile doesn?t use JNI in JDK 9. > > Of the two remaining sections in Tables 1 and 2, the section labeled > ?MultiRelease JarFile? differs from the section labeled ?MultiRelease > JarFile, new iterator? only in the JarEntryIterator class. The first > one uses the original iterator in JarFile.java that can be seen > starting with line 551 of webrev.04 > , and the new > iterator starts with line 553 of webrev.06 > . The > results are strange. The new, more complex, iterator appears to be > faster than the old, simpler, iterator. I double, and triple, checked > it, but it was always faster. I used jitwatch to look at the hotspot > logs generated during compilation and neither method was compiled. I > suppose I could dig into it further but decided not to. Consider it > good news. The results do show that the multi-release enhancement > slows JarFile entries/stream down by 2-18% depending on the size of > the jar file. But they are still much better than the JDK 8 values. > >> Also I would expect that if a JAR file is not mult-release but the >> library opens it with Release.RUNTIME to perform the same as opening >> the JAR file with the Release-less constructors. > > The results in Table 3 attempts to answer this question since rt.jar > is not a multi-release jar file. This tells me that if one opens the > JarFile with Release.RUNTIME, that there is a performance penalty of > 2-6% on this very large jar file. > > Finally, the results in Table 4 tell me that processing a true > multi-release jar file takes about 80% more time per entries() or > stream() operation. I?ve looked at this in a profiler and there is no > particular area that stands out to me, it?s just more complicated to > process a multi-release jar file, as would be expected. > >> I think the javadoc will need to also need to make it clear whether >> entries with names starting with META-INF/versions/ are returned. > > Fixed. > >> >> I see you've added @since 9 to the existing methods, I assume you >> didn't mean to do this. > > Fixed. > >> >> At some point then we need to discuss how RUNTIME_VERSION is >> computed. Iris (via Mandy) has pushed jdk.Version to jdk9/dev but >> having it exported by java.base conflicts with the design principles >> in JEP 200. Moving it to another module means that code in java.base >> cannot use it and thus the JAR file can't use it. > > Left as is. > From john.r.rose at oracle.com Sat Feb 13 04:02:53 2016 From: john.r.rose at oracle.com (John Rose) Date: Fri, 12 Feb 2016 20:02:53 -0800 Subject: [9] RFR (S): 8148994: Replacing MH::invokeBasic with a direct call breaks LF customization In-Reply-To: <56BE133A.70508@oracle.com> References: <56B37387.2040204@oracle.com> <6CCE2F2C-1836-47CC-B13E-4371C2CE61FF@oracle.com> <56B4CE00.3040809@oracle.com> <56BE133A.70508@oracle.com> Message-ID: Better! You moved some inlining policy out of the VM and into the JDK runtime. Reviewed. Some loose brainstorming here: We are creating MH nodes that provide hints to the JIT inlining policy. The hint must vary over time as the profile evolves. The CMH does this indirectly via generating a @DontInline LF, with the LF patchable. Another way to do it would be to have a @Stable MH pointer initially null, but which would go non-null when an inlineable MH is determined. Another way would be to use a mutable CS. The handshake might need to include a "push" notification to the JIT that something has changed and recompilation should be attempted. The MCS can do this. Or perhaps the push should happen after some sort of further countdown. ? John > On Feb 12, 2016, at 9:15 AM, Vladimir Ivanov wrote: > > Any reviews, please? > > Best regards, > Vladimir Ivanov > > On 2/5/16 7:29 PM, Vladimir Ivanov wrote: >> Thanks, John. >> >> On 2/4/16 9:52 PM, John Rose wrote: >>> Looks good. Except I don't recall how is_inline interacts, if at all, >>> with customization. It's not clear what is being gated, and which >>> optimizations get let through. >> Inlining through MH::invokeBasic doesn't happen when there's @DontInline >> on the target. It is the case for GWT which has CountingWrapper on both >> paths. It starts with a reinvoker.dontInline LF and then switches to a >> reinvoker LF (after 30 invocations). If a compilation happens when >> reinvoker.dontInline is used, a direct call is generated and even if the >> branch becomes hot, all the calls goes through reinvoker.dontInline. >> Update to reinvoker LF isn't visible unless the dispatch goes through >> MH::invokeBasic linker. >> >> I had an impression that I introduced LF customization logic to the >> non-inlined case in GWT, but I can't find it in the code. So, it seems >> the regression is caused by repeated checks performed by >> reinvoker.dontInline LF and not by LF customization missed in generated >> code. >> >> Also, I stumbled upon a quirk in inlining policy: inlining through >> linkers can't be delayed. It happens when JIT hits some limit on IR >> size. So, right now inlining can stop in the middle of MH chain and a >> linker call is generated. >> >> But after JDK-8072008 there's no problem with delaying inlining. C2 can >> decide whether to keep the direct call or inline through it. So, I >> enabled late inlining for all linkers. (Surprisingly, no significant >> performance difference on nashorn.) >> >> So, I decided to keep invokeBasic linker call replacement for now, but >> piggypack LF customization on the stuck counting LF. >> >> Updated webrev: >> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/hotspot >> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.01/jdk >> >> * enabled late inlining for linkers; >> * add LF customization check in CountingWrapper::countDown, so when >> counting is stuck, the target will be customized. >> >> Testing: octane, JPRT. >> >> I'm working on JDK-8071793 [1] and will reconsider how LF customization >> on non-inlined branch in GWTs works given there is a more JVM-friendly >> mechanism to control inlining. With the proposed fix >> reinvoker.dontInline is never customized (only the target it points to) >> and serves as a trampoline between GWT & target LF. It would be better >> to jump right to the customized LF, if non-inlined branch becomes hot. >> >> Best regards, >> Vladimir Ivanov >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8071793 >> "Prune rarely called invokedynamic call sites during inlining" >>> >>> Maybe a comment about that would be useful >>> Reviewed. >>> >>> ? John >>> >>>> On Feb 4, 2016, at 7:51 AM, Vladimir Ivanov >>>> wrote: >>>> >>>> http://cr.openjdk.java.net/~vlivanov/8148994/webrev.00 >>>> https://bugs.openjdk.java.net/browse/JDK-8148994 >>>> >>>> JDK-8072008 [1] introduced an optimization to bypass linker calls in >>>> generated code when target method can't be inlined. >>>> >>>> Unfortunately, it doesn't work well with MH::invokeBasic(), because >>>> it clashes with LambdaForm customization: if a call is not inlined, >>>> it is important for performance to update (customize) corresponding >>>> LambdaForm on-the-fly. MH::invokeBasic() linker does MH -> LF -> >>>> Method* -> _from_compiled_entry traversal on every invocation, while >>>> direct call points right to the _from_compiled_entry. So, any >>>> LambdaForm updates aren't visible until recompilation. >>>> >>>> The fix is to keep MH::invokeBasic linker, so up-to-date LambdaForm >>>> instance is used on every invocation. >>>> >>>> Also, cleaned up relevant tests a bit. >>>> >>>> Testing: octane, JPRT. >>>> >>>> Best regards, >>>> Vladimir Ivanov >>>> >>>> [1] https://bugs.openjdk.java.net/browse/JDK-8072008 >>>> "Emit direct call instead of linkTo* for recursive >>>> indy/MH.invoke* calls" From xueming.shen at oracle.com Sat Feb 13 04:58:37 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 12 Feb 2016 20:58:37 -0800 Subject: RFR JDK-8148624: Test failure of ConstructInflaterOutput.java Message-ID: <56BEB7FD.305@oracle.com> Hi, Please help review the change for JDK-8148624 Issue: https://bugs.openjdk.java.net/browse/JDK-8148624 Webrev: http://cr.openjdk.java.net/~sherman/8148624/webrev/ The tests are intended to test/verify that the Deflater/Inflater.end() method will not be invoked when they are passed in as the parameter to the constructor of the DefaultInputStream and InflateOutputStream class, when the corresponding close&/finish() method are invoked. The stacktrace suggests that the end() method is not actually being called (failure) by the close/finish(), but by the finalize(). It appears the deflater/inflater is somehow being finalized by gc between A and B showed below public static void main(String[] args) throws Throwable { try {realMain(args);} catch (Throwable t) {unexpected(t);} [A]--> System.out.println("\nPassed = " + passed + " failed = " + failed); [B]--> if (failed > 0) throw new AssertionError("Some tests failed");} the problem can be easily reproduced by inserting a System.gc() in between. The easy fix is just move the inflater/deflater out of realMain() to be the static variable to prevent them from being gc-ed. Thanks, Sherman From peter.firmstone at zeus.net.au Sun Feb 14 00:32:33 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Sun, 14 Feb 2016 10:32:33 +1000 Subject: Fwd: Re: DeserializationPermission Proposal Message-ID: <56BFCB21.5070800@zeus.net.au> If ObjectInputStream's creation was guarded by a Permission, this would prevent serialization for untrusted users. Regards, Peter. -------- Original Message -------- Subject: Re: DeserializationPermission Proposal Date: Fri, 12 Feb 2016 16:30:35 +1000 From: Peter Firmstone To: Core-Libs-Dev Thanks David, I'd originally considered something like that, but I later realised I'd need to grant DeserializationPermission to other domains I might not want allow objects to be deserialized with, simply because their domain is included in the call stack. It looked like it would widen the scope, which was contrary to the intended goal of minimising the attack surface. In the end I decided to limit the context to only the domains of classes involved in deserialization. Cheers, Peter. > On 02/11/2016 03:52 AM, Peter Firmstone wrote: > >/ An example might be more useful. > />/ > />/ Traditionally, when the first non serializable superclass zero argument > />/ constructor is called, the domains of the subclasses aren't present on > />/ the call stack. Any security checks performed in the constructor of the > />/ superclass will not include the subclasses domains. > />/ > />/ In the proposed case, prior to construction, all domains in the class > />/ heirarchy of the to be deserialized object via the local > />/ ObjectStreamClass, are added to an AccessControlContext, which is then > />/ passed to the SecurityManager two argument permission check. > / > Sure, that makes sense; in fact this could be a very good > standalone/incremental enhancement. > > >/ Now an attacker will not be able to construct this object unless all > />/ domains have DeserializationPermission. > />/ > />/ If we use the stack context, it won't contain the yet to be deserialized > />/ classes. > / > True; perhaps the ideal solution would use the initial context plus a > per-deserializing-class context, so that both the original caller and > the class being deserialized have permissions. This would have the > advantage of consistent behavior, and would also allow the PD of each > class to restrict whether it can be deserialized (which would apply > globally no matter who was doing the deserialization in the VM). > > -- > - DML From amaembo at gmail.com Sun Feb 14 14:53:43 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Sun, 14 Feb 2016 20:53:43 +0600 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite Message-ID: <6728448.20160214205343@gmail.com> Hello! I wanted to work on foldLeft, but Brian asked me to take this issue instead. So here's webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ I don't like iterator-based Stream source implementations, so I made them AbstractSpliterator-based. I also implemented manually forEachRemaining as, I believe, this improves the performance in non-short-circuiting cases. I also decided to keep two flags (started and finished) to track the state. Currently existing implementation of infinite iterate() does not use started flag, but instead reads one element ahead for primitive streams. This seems wrong to me and may even lead to unexpected exceptions (*). I could get rid of "started" flag for Stream.iterate() using Streams.NONE, but this would make object implementation different from primitive implementations. It would also be possible to keep single three-state variable (byte or int, NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve the performance or footprint. Having two flags looks more readable to me. Currently existing two-arg iterate methods can now be expressed as a partial case of the new method: public static Stream iterate(final T seed, final UnaryOperator f) { return iterate(seed, x -> true, f); } (same for primitive streams). I may do this if you think it's reasonable. I created new test class and added new iterate sources to existing data providers. Please review and sponsor! With best regards, Tagir Valeev. (*) Consider the following code: int[] data = {1,2,3,4,-1}; IntStream.iterate(0, x -> data[x]) .takeWhile(x -> x >= 0) .forEach(System.out::println); Currently this unexpectedly throws an AIOOBE, because IntStream.iterate unnecessarily tries to read one element ahead. From spliterator at gmail.com Sun Feb 14 18:15:59 2016 From: spliterator at gmail.com (Stefan Zobel) Date: Sun, 14 Feb 2016 19:15:59 +0100 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <6728448.20160214205343@gmail.com> References: <6728448.20160214205343@gmail.com> Message-ID: Hi Tagir, this looks good. I wonder, however, if the signature should accept a wider range of types, i.e., something like static Stream iterate(S seed, Predicate predicate, UnaryOperator f) I know that the corresponding static Stream iterate(T seed, UnaryOperator f) is less general than that, but I don't know whether this is the outcome of a thoughtful decision or just an oversight. What do you think? Regards, Stefan 2016-02-14 15:53 GMT+01:00 Tagir F. Valeev : > Hello! > > I wanted to work on foldLeft, but Brian asked me to take this issue > instead. So here's webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ > > I don't like iterator-based Stream source implementations, so I made > them AbstractSpliterator-based. I also implemented manually > forEachRemaining as, I believe, this improves the performance in > non-short-circuiting cases. > > I also decided to keep two flags (started and finished) to track the > state. Currently existing implementation of infinite iterate() does > not use started flag, but instead reads one element ahead for > primitive streams. This seems wrong to me and may even lead to > unexpected exceptions (*). I could get rid of "started" flag for > Stream.iterate() using Streams.NONE, but this would make object > implementation different from primitive implementations. It would also > be possible to keep single three-state variable (byte or int, > NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve > the performance or footprint. Having two flags looks more readable to > me. > > Currently existing two-arg iterate methods can now be expressed as a > partial case of the new method: > > public static Stream iterate(final T seed, final UnaryOperator f) { > return iterate(seed, x -> true, f); > } > (same for primitive streams). I may do this if you think it's > reasonable. > > I created new test class and added new iterate sources to existing > data providers. > > Please review and sponsor! > > With best regards, > Tagir Valeev. > > (*) Consider the following code: > > int[] data = {1,2,3,4,-1}; > IntStream.iterate(0, x -> data[x]) > .takeWhile(x -> x >= 0) > .forEach(System.out::println); > > Currently this unexpectedly throws an AIOOBE, because > IntStream.iterate unnecessarily tries to read one element ahead. > From xueming.shen at oracle.com Sun Feb 14 20:24:57 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Sun, 14 Feb 2016 12:24:57 -0800 Subject: RFR: JDK-8149787 test/java/util/regex/GraphemeTest.java source file has non-ascii character u+00f7 Message-ID: <56C0E299.4020809@oracle.com> Hi, Please help review the change for bug JDK-8149787, which "accidentally" embeds couple non-ascii character u+00f7 in several comment lines. This will trigger javac failure if using encoding=ascii. The fix is to replace the DIV with a simple "+" character. issue: https://bugs.openjdk.java.net/browse/JDK-8149787 webrev: http://cr.openjdk.java.net/~sherman/8149787/webrev/ Thanks, Sherman From ivan.gerasimov at oracle.com Sun Feb 14 21:27:21 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 15 Feb 2016 00:27:21 +0300 Subject: RFR: JDK-8149787 test/java/util/regex/GraphemeTest.java source file has non-ascii character u+00f7 In-Reply-To: <56C0E299.4020809@oracle.com> References: <56C0E299.4020809@oracle.com> Message-ID: <56C0F139.8080902@oracle.com> Hi! This looks good! Sincerely yours, Ivan On 14.02.2016 23:24, Xueming Shen wrote: > Hi, > > Please help review the change for bug JDK-8149787, which > "accidentally" embeds > couple non-ascii character u+00f7 in several comment lines. This will > trigger > javac failure if using encoding=ascii. The fix is to replace the DIV > with a simple > "+" character. > > issue: https://bugs.openjdk.java.net/browse/JDK-8149787 > webrev: http://cr.openjdk.java.net/~sherman/8149787/webrev/ > > Thanks, > Sherman > From steve.drach at oracle.com Mon Feb 15 00:42:51 2016 From: steve.drach at oracle.com (Steve Drach) Date: Sun, 14 Feb 2016 16:42:51 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56BE8497.8020207@oracle.com> References: <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC.7080102@oracle.com> <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> <56BE8497! .8020207@oracle.com> Message-ID: <567A3A2F-9C0A-475E-A14A-9D3A06C2EB49@oracle.com> Sherman, I like your suggestions and will open an issue to work on the performance after integration. I?ll fix the minor ?bug? you pointed out at the same time. Steve > On Feb 12, 2016, at 5:19 PM, Xueming Shen wrote: > > > Steve, > > I would assume the difference of the webrev.04 "old" iterator and the webrev.06 "new" iterator > that might trigger a performance is how you create the JarFileEntry. The one parameter constructor > invokes isMultiRelease(), which might be relatively expensive, when the "mr" is enabled. There > are couple "if" checks involved. > > For the "new" iterator is slower than the current jdk9 one. It might be desired to have two JarEntryIterator > classes defined, one for "notVersioned", one for the "versioned". Of course keep the two parameter > constructor for the "notVersioned". This might bring performance back for the normal "notVersioned" > usage, which I would assume is the majority. > > There is also a "bug" in the new iterator. The iterator/Enumeration returned from ZipFile.entries() > checks "ensureOpen() in both hasNext() and next(). So close() the ZipFile after itr.hasNext() will > fails the next next() invocation in the old implementation. The latest itr will returns the cached > "ze". Not a big issue for sure, but a kind of "regression". Defining two iterator classes as suggested > above will also workaround this issue, as the "notVersioned" one will work just as expected, no > regression. > > -Sherman > > On 2/9/16 5:04 PM, Steve Drach wrote: >> Hi, >> >> Yet another webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.06/ , with a change to JarEntryIterator to fix a problem discovered by performance tests ? calling hasNext() twice as often as needed. I also removed the @since 9 tags from the methods entries() and stream(), and added an additional sentence to the spec for each of those methods that clarifies what a base entry is (actually is not). >> >>> I think having stream and entries do this is right although I would like to see some performance data if possible. >> >> See http://cr.openjdk.java.net/~sdrach/8132734/JarFile%20Performance.pdf >> >> I used JMH to run the benchmark. See http://cr.openjdk.java.net/~sdrach/8132734/MyBenchmark.java . I used two jar files, the rt.jar file from JDK 8 that has 20653 entries and the multi-release.jar found in the test directory with 14 entries. Obviously rt.jar is not a multi-release jar file. >> >> The first two tables (1 and 2) are comparable and the second two tables are somewhat comparable (3 and 4). >> >> Tables 1 and 2 have 4 sections that show the results of tests on the two jar files in 4 configurations of JarFile. The tests were done with a JarFile object constructed without the Release object argument, essentially the legacy constructor. The section labeled "JDK 8 JarFile? was done with JDK 8u66. The section labeled ?JDK 9 JarFile? was done with the latest build of openjdk/jdk9/dev without any changes in my 8132734 changeset. I chose this section as the reference, so the last column shows the values normalized to 1 micro/milli second per operation (rt.jar times are in milliseconds and multi-release.jar times are in microseconds). It should be obvious that JDK 9 is much faster than JDK 8, somewhere on the order of 5-6 times faster. I think that is because ZipFile doesn?t use JNI in JDK 9. >> >> Of the two remaining sections in Tables 1 and 2, the section labeled ?MultiRelease JarFile? differs from the section labeled ?MultiRelease JarFile, new iterator? only in the JarEntryIterator class. The first one uses the original iterator in JarFile.java that can be seen starting with line 551 of webrev.04 , and the new iterator starts with line 553 of webrev.06 . The results are strange. The new, more complex, iterator appears to be faster than the old, simpler, iterator. I double, and triple, checked it, but it was always faster. I used jitwatch to look at the hotspot logs generated during compilation and neither method was compiled. I suppose I could dig into it further but decided not to. Consider it good news. The results do show that the multi-release enhancement slows JarFile entries/stream down by 2-18% depending on the size of the jar file. But they are still much better than the JDK 8 values. >> >>> Also I would expect that if a JAR file is not mult-release but the library opens it with Release.RUNTIME to perform the same as opening the JAR file with the Release-less constructors. >> >> The results in Table 3 attempts to answer this question since rt.jar is not a multi-release jar file. This tells me that if one opens the JarFile with Release.RUNTIME, that there is a performance penalty of 2-6% on this very large jar file. >> >> Finally, the results in Table 4 tell me that processing a true multi-release jar file takes about 80% more time per entries() or stream() operation. I?ve looked at this in a profiler and there is no particular area that stands out to me, it?s just more complicated to process a multi-release jar file, as would be expected. >> >>> I think the javadoc will need to also need to make it clear whether entries with names starting with META-INF/versions/ are returned. >> >> Fixed. >> >>> >>> I see you've added @since 9 to the existing methods, I assume you didn't mean to do this. >> >> Fixed. >> >>> >>> At some point then we need to discuss how RUNTIME_VERSION is computed. Iris (via Mandy) has pushed jdk.Version to jdk9/dev but having it exported by java.base conflicts with the design principles in JEP 200. Moving it to another module means that code in java.base cannot use it and thus the JAR file can't use it. >> >> Left as is. >> > From stuart.marks at oracle.com Mon Feb 15 01:03:15 2016 From: stuart.marks at oracle.com (Stuart Marks) Date: Sun, 14 Feb 2016 17:03:15 -0800 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <6728448.20160214205343@gmail.com> References: <6728448.20160214205343@gmail.com> Message-ID: <56C123D3.8000608@oracle.com> Hi Tagir, Thanks for picking this up. I'll be at a conference this week and I won't have much time to discuss this in detail until afterward. But here are some quick thoughts about the proposal. I'd suggest focusing on the API first before worrying about how to track the stream state with booleans, etc. Is the API convenient to use, and how well does it support the use cases we envision for it? In particular, I can imagine a number of cases where it would be very helpful to be able to support an empty stream, or where the computation to produce the first element is the same as the computation to produce subsequent elements. Requiring a value for the first stream element is at odds with that. Here are some ideas for use cases to try out: - a series of dice rolls representing a round of craps [1] - elements drawn from a queue until the queue is empty or until a sentinel is reached - a sequence of numbers that (probably) terminates but whose length isn't necessarily known in advance (e.g. Collatz sequence [2]) [1] https://en.wikipedia.org/wiki/Craps [2] https://en.wikipedia.org/wiki/Collatz_conjecture Note that in some cases the sentinel value that terminates the stream should be part of the stream, and in other cases it's not. I'm sure you can find more uses cases by perusing Stack Overflow. :-) I'm a bit skeptical of the use of "iterate" for producing a finite stream. There are the usual issues with overloading, but there's also potential confusion as some forms of iterate() are infinite and others finite. I'll suggest the name "produce" instead, but there are surely better terms. One thing to think about is where the state of the producer is stored. Is it expected to be in an argument that's passed to each invocation of the functional argument, or is it expected to be captured? I don't think there's an answer in isolation; examining use cases would probably shed some light here. Here are a few API ideas (wildcards elided): -- Stream iterate(T seed, Predicate predicate, UnaryOperator f) The API from your proposal, for comparison purposes. -- Stream produce(Supplier>) Produces elements until empty Optional is returned. This box/unboxes every element, maybe(?) alleviated by Valhalla. -- Stream produce(BooleanSupplier, Supplier) Calls the BooleanSupplier; if true the next stream element is what's returned by calling the Supplier. If BooleanSupplier returns false, end of stream. If you have an iterator already, this enables produce(iterator::hasNext, iterator::next) But if you don't have an iterator already, coming up with the functions to satisfy the iterator-style protocol is sometimes painful. -- Stream produce(Predicate> advancer) This has an odd signature, but the function is like Spliterator.tryAdvance(). It must either call the consumer once and return true, or return false without calling the consumer. -- Stream produce(Consumer> advancer) A variation of the above, without a boolean return. The advancer calls the consumer one or more times to add elements to the stream. End of stream occurs when the advancer doesn't call the consumer. -- Stream produce(Supplier>) A variation of Supplier> where the supplier returns a stream containing zero or more elements. The stream terminates if the supplier returns an empty stream. There "boxing" overhead here, but we don't seem to be bothered by this with flatMap(). -- s'marks On 2/14/16 6:53 AM, Tagir F. Valeev wrote: > Hello! > > I wanted to work on foldLeft, but Brian asked me to take this issue > instead. So here's webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ > > I don't like iterator-based Stream source implementations, so I made > them AbstractSpliterator-based. I also implemented manually > forEachRemaining as, I believe, this improves the performance in > non-short-circuiting cases. > > I also decided to keep two flags (started and finished) to track the > state. Currently existing implementation of infinite iterate() does > not use started flag, but instead reads one element ahead for > primitive streams. This seems wrong to me and may even lead to > unexpected exceptions (*). I could get rid of "started" flag for > Stream.iterate() using Streams.NONE, but this would make object > implementation different from primitive implementations. It would also > be possible to keep single three-state variable (byte or int, > NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve > the performance or footprint. Having two flags looks more readable to > me. > > Currently existing two-arg iterate methods can now be expressed as a > partial case of the new method: > > public static Stream iterate(final T seed, final UnaryOperator f) { > return iterate(seed, x -> true, f); > } > (same for primitive streams). I may do this if you think it's > reasonable. > > I created new test class and added new iterate sources to existing > data providers. > > Please review and sponsor! > > With best regards, > Tagir Valeev. > > (*) Consider the following code: > > int[] data = {1,2,3,4,-1}; > IntStream.iterate(0, x -> data[x]) > .takeWhile(x -> x >= 0) > .forEach(System.out::println); > > Currently this unexpectedly throws an AIOOBE, because > IntStream.iterate unnecessarily tries to read one element ahead. > From david.holmes at oracle.com Mon Feb 15 04:55:46 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 15 Feb 2016 14:55:46 +1000 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56BE1AC4.2040405@Oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> Message-ID: <56C15A52.1050403@oracle.com> Hi Roger, A few mostly minor comments ... On 13/02/2016 3:47 AM, Roger Riggs wrote: > Please review moving the functionality of sun.misc.Signal to > jdk.internal.misc and > creating a wrapper sun.misc.Signal for existing external uses. > > +++ This time including the hotspot changes to update the target of the > upcalls. > > A new replacement API will be considered separately. > > The update includes a test that passes with or without the changes. > (Except for an NPE instead of SEGV if null is passed). > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ I take it we don't care about reorder files any more? --- src/java.base/share/classes/sun/misc/Signal.java The @since should not be changed to 9. You only changed the implementation not the API. Conversely the renamed class should arguably be @since 9 and not @since 1.2. But it is wrong to say sun.misc.Signal is @since 9. The *Handler.of method name reads strangely to me - "for" would be better but I presume that is not allowed. --- src/java.base/share/classes/jdk/internal/misc/Signal.java Not sure I understand the role of NativeSignalHandler any more - given it can't actually be used ?? --- test/sun/misc/SunMiscSignalTest.java Based on our email discussion this test can not be testing that the handler actually gets invoked - as we know it does not in some cases. I have doubts that sun.misc.Signal ever intended to support all the signals you are testing. > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ Hotspot changes are fine. Thanks, David ----- > Issue: > https://bugs.openjdk.java.net/browse/JDK-8149750 > > Thanks, Roger > > JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 > > From peter.levart at gmail.com Mon Feb 15 06:48:42 2016 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 15 Feb 2016 07:48:42 +0100 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: References: <6728448.20160214205343@gmail.com> Message-ID: <56C174CA.9050800@gmail.com> On 02/14/2016 07:15 PM, Stefan Zobel wrote: > Hi Tagir, > > this looks good. I wonder, however, if the signature should > accept a wider range of types, i.e., something like > > > static Stream iterate(S seed, Predicate > predicate, UnaryOperator f) > > > I know that the corresponding > > static Stream iterate(T seed, UnaryOperator f) > > is less general than that, but I don't know whether this is > the outcome of a thoughtful decision or just an oversight. > > What do you think? What about even more permissive: static Stream iterate3(S seed, Predicate predicate, Function f) ... public class SignatureTest { static Stream iterate1(T seed, Predicate predicate, UnaryOperator f) { ... } static Stream iterate2(S seed, Predicate predicate, UnaryOperator f) { ... } static Stream iterate3(S seed, Predicate predicate, Function f) { ... } public static void main(String[] args) { Stream si1 = iterate1(0, i -> i < 10, i -> i + 1); // OK Stream si2 = iterate2(0, i -> i < 10, i -> i + 1); // OK Stream si3 = iterate3(0, i -> i < 10, i -> i + 1); // OK Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); //SignatureTest.java:32: error: bad operand types for binary operator '<' // Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); // ^ //SignatureTest.java:32: error: bad operand types for binary operator '+' // Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); // ^ Stream sn2 = iterate2(0, i -> i < 10, i -> i + 1); // OK Stream sn3 = iterate3(0, i -> i < 10, i -> i + 1); // OK Predicate csNotEmpty = cs -> cs.length() > 0; Stream css1 = iterate1("abcde", csNotEmpty, (String s) -> s.substring(1)); //SignatureTest.java:44: error: method iterate1 in class SignatureTest cannot be applied to given types; // Stream css1 = iterate1("abcde", csNotEmpty, (String s) -> s.substring(1)); // ^ // required: T,Predicate,UnaryOperator // found: String,Predicate,(String s)[...]ng(1) // reason: inference variable T has incompatible equality constraints String,CharSequence // where T is a type-variable: // T extends Object declared in method iterate1(T,Predicate,UnaryOperator) Stream css2 = iterate2("abcde", csNotEmpty, (String s) -> s.substring(1)); //SignatureTest.java:54: error: method iterate2 in class SignatureTest cannot be applied to given types; // Stream css2 = iterate2("abcde", csNotEmpty, (String s) -> s.substring(1)); // ^ // required: S,Predicate,UnaryOperator // found: String,Predicate,(String s)[...]ng(1) // reason: inference variable S has incompatible equality constraints String,CharSequence // where S,T are type-variables: // S extends T declared in method iterate2(S,Predicate,UnaryOperator) // T extends Object declared in method iterate2(S,Predicate,UnaryOperator) Stream css3 = iterate3("abcde", csNotEmpty, (String s) -> s.substring(1)); // OK } } Regards, Peter > > Regards, > Stefan > > > 2016-02-14 15:53 GMT+01:00 Tagir F. Valeev : >> Hello! >> >> I wanted to work on foldLeft, but Brian asked me to take this issue >> instead. So here's webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ >> >> I don't like iterator-based Stream source implementations, so I made >> them AbstractSpliterator-based. I also implemented manually >> forEachRemaining as, I believe, this improves the performance in >> non-short-circuiting cases. >> >> I also decided to keep two flags (started and finished) to track the >> state. Currently existing implementation of infinite iterate() does >> not use started flag, but instead reads one element ahead for >> primitive streams. This seems wrong to me and may even lead to >> unexpected exceptions (*). I could get rid of "started" flag for >> Stream.iterate() using Streams.NONE, but this would make object >> implementation different from primitive implementations. It would also >> be possible to keep single three-state variable (byte or int, >> NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve >> the performance or footprint. Having two flags looks more readable to >> me. >> >> Currently existing two-arg iterate methods can now be expressed as a >> partial case of the new method: >> >> public static Stream iterate(final T seed, final UnaryOperator f) { >> return iterate(seed, x -> true, f); >> } >> (same for primitive streams). I may do this if you think it's >> reasonable. >> >> I created new test class and added new iterate sources to existing >> data providers. >> >> Please review and sponsor! >> >> With best regards, >> Tagir Valeev. >> >> (*) Consider the following code: >> >> int[] data = {1,2,3,4,-1}; >> IntStream.iterate(0, x -> data[x]) >> .takeWhile(x -> x >= 0) >> .forEach(System.out::println); >> >> Currently this unexpectedly throws an AIOOBE, because >> IntStream.iterate unnecessarily tries to read one element ahead. >> From chris.hegarty at oracle.com Mon Feb 15 09:47:38 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 15 Feb 2016 09:47:38 +0000 Subject: RFR 9:8148775 : Spec for j.l.ProcessBuilder.Redirect.DISCARD need to be improved In-Reply-To: References: <56BE5FCA.5050609@Oracle.com> <56BE65E3.2010706@Oracle.com> Message-ID: <56C19EBA.3090000@oracle.com> On 12/02/16 23:10, Martin Buchholz wrote: > Still looks good! +1. -Chris. > On Fri, Feb 12, 2016 at 3:08 PM, Roger Riggs wrote: >> Thanks, Martin >> >> Updated in place. >> >> >> >> >> On 2/12/2016 5:48 PM, Martin Buchholz wrote: >> >> looks good. >> The previous line >> >> Redirect.DISCARD.file() the filename appropriate for the operating system >> 557 * and may be null && >> >> could use the word "is" >> >> On Fri, Feb 12, 2016 at 2:42 PM, Roger Riggs wrote: >> >> Please review a minor javadoc cleanup to remove a reference to a method >> (append) >> of the implementation. >> >> Webrev: >> http://cr.openjdk.java.net/~rriggs/webrev-discard-8148775/ >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8148775 >> >> Thanks, Roger >> >> From chris.hegarty at oracle.com Mon Feb 15 11:22:05 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 15 Feb 2016 11:22:05 +0000 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56BE1AC4.2040405@Oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> Message-ID: <56C1B4DD.2060903@oracle.com> On 12/02/16 17:47, Roger Riggs wrote: > Please review moving the functionality of sun.misc.Signal to > jdk.internal.misc and > creating a wrapper sun.misc.Signal for existing external uses. > > +++ This time including the hotspot changes to update the target of the > upcalls. > > A new replacement API will be considered separately. > > The update includes a test that passes with or without the changes. > (Except for an NPE instead of SEGV if null is passed). > > Webrev: > jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ Overall looks ok, and satisfies the requirement of JEP 260. It took me a while to satisfy myself that it is ok to "ignore" the passed Signal in the wrapper's 'handle' methods. The assumption is that this wrapper's signal field and the passes signal are, MUST, represent the same underlying signal. Maybe an assert to make this explicit? Looking at j.i.m.Signal., I can see that you explicitly check for the 'SIG' prefix and prepend it if not there, but toString() also prepends it. Will getName also be impacted by this ( it may now have the name prepended with 'SIG', where it previously was not ). > hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ Looks fine. -Chris. From Alan.Bateman at oracle.com Mon Feb 15 12:30:43 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 15 Feb 2016 12:30:43 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> References: <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC.7080102@oracle.com> <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> Message-ID: <56C1C4F3.9000505@oracle.com> On 10/02/2016 01:04, Steve Drach wrote: > Hi, > > Yet another webrev, > http://cr.openjdk.java.net/~sdrach/8132734/webrev.06/ > , with a > change to JarEntryIterator to fix a problem discovered by performance > tests ? calling hasNext() twice as often as needed. I also removed > the @since 9 tags from the methods entries() and stream(), and added > an additional sentence to the spec for each of those methods that > clarifies what a base entry is (actually is not). > I went through the latest webrev and it looks quite good. A few comments on the javadoc: "... partitioned by the major version of Java platform releases" - this might be better as "... partitioned by the major version of the Java platform". In JarFile.Release then it uses the phrase "top-most (base) directory". I thought we had purged "top-*" from the javadoc in previous iterations because it hints of classes or resources in the top most directory (which isn't the case with classes in a named package). "... will not be accessible by this JarFile" hints of access control or even security manager. Would it clearer to re-word this to something like "will not be located by methods such as getEntry" ? "returned depends whether" -> "returned depends on whether". In the javadoc for entries() and stream() then it mentions "the constructor" many times. I would be tempted to replace many of these - for example "all entries are returned, regardless of the constructor" might be better as "all entries of returned, regards of how the JarFile is created". A couple of nits on the implementation: vze = JarFile.super.getEntry(META_INF_VERSIONS + i-- + sname); - it would be more readable if you move the decrement to its own line. Also I assume that JarFile is not needed here. L942-943 looks messy too, I assume that can be cleaned up. JarFileFactory - "earl" will confuse readers, needs a comment or a better name. I think this is all that I have for now. -Alan. From chris.hegarty at oracle.com Mon Feb 15 14:57:41 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Mon, 15 Feb 2016 14:57:41 +0000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B9EB17.7020303@oracle.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> Message-ID: <56C1E765.7080603@oracle.com> Peter, > http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/ Given the changes in 6857566, I think your proposal to remove jdk.internal.ref.Cleaner should be good. ( Note: third party libraries digging into internals of direct buffers will need to change again ). -Chris. On 09/02/16 13:35, Chris Hegarty wrote: > Peter, > > On 08/02/16 10:58, Peter Levart wrote: > >> On 02/08/2016 10:33 AM, Chris Hegarty wrote: >>> On 8 Feb 2016, at 06:27, Alan Bateman wrote: >>> >>>> On 07/02/2016 22:20, Peter Levart wrote: >>>>> : >>>>> >>>>> If the decision to remove sun.misc.Cleaner was partly influenced by >>>>> the desire to maintain just 2 instead of 3 Cleaner(s), then my >>>>> proposal to migrate JDK code to the public API might enable Oracle >>>>> to reconsider keeping sun.misc.Cleaner. >>>> The main issue driving this is the design principles that we have >>>> listed in JEP 200. We don't want a standard module (java.base in >>>> this case) exporting a non-standard API. This means surgery to >>>> jettison the sun.misc package from java.base and move it to its own >>>> module (jdk.unsupported is the proposal in JEP 260). This is painful >>>> of course but we are at least in good shape for the most critical >>>> internal API, sun.misc.Unsafe. >>>> >>>> For sun.misc.Cleaner then the original proposal was for it to be a >>>> critical internal API too but it become clear that changing the type >>>> of internal/private fields in the NIO buffer and channel classes >>>> would break libraries that have been hacking into those fields. If >>>> they are changing away then there seems little motive to keep >>>> sun.misc.Cleaner so Chris moved into into jdk.internal.ref for now. >>>> As to whether we even need to keep jdk.internal.ref.Cleaner then I >>>> think the only remaining question was whether the special handling >>>> of Cleaner in the Reference implementation was worth it or not. It >>>> may not be, in which case your current proposal to just remove seems >>>> right. >>> Alan?s summary of the situation is spot on. >>> >>> Before moving sun.misc.Cleaner to jdk.internal.ref, I did enquire if >>> it would be >>> possible to just remove it and use the new public Cleaner, but I got >>> feedback >>> that there were some issues with failing NIO tests ( it appeared that >>> Cleaners >>> were not running as quickly as possible ). I didn?t look further into >>> that at the >>> time, since the VM still had special knowledge of these cleaners, but >>> as you >>> say this is now removed. It is probably a good time to revisit this. >>> >>> -Chris. >> >> Hi Chris, >> >> Are you referring to the following test: >> >> test/java/nio/Buffer/DirectBufferAllocTest.java ? >> >> This test was written specifically for the following issue: >> >> https://bugs.openjdk.java.net/browse/JDK-6857566 >> >> ...which exercises multi-threaded allocation of direct buffers. Even >> sun.misc.Cleaner was not quick enough to promptly deallocate them, so >> allocation path was patched to help ReferenceHandler thread while >> re-attempting to allocate new direct buffers. Transitioning to >> java.lang.ref.Cleaner which uses additional thread to process >> Cleanable(s), direct-buffer allocation path must help the Cleaner thread >> too. See: > > Ah, I was not aware of this. So additional threads attempting to > allocate already help out with cleaning. So moving away from the > ReferenceHandler Thread is not such a big deal as I was thinking. > >> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/src/java.base/share/classes/java/nio/Bits.java.sdiff.html >> >> >> ...for the place where this is performed... > > I did look through your changes and I think they are good. I'd like > to spend a little more time on the details. > > -Chris. > >> Regards, Peter >> >> >> From steve.drach at oracle.com Mon Feb 15 16:30:06 2016 From: steve.drach at oracle.com (Steve Drach) Date: Mon, 15 Feb 2016 08:30:06 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56C1C4F3.9000505@oracle.com> References: <466DC6AF-8FDE-459F-80BD-3DE945EC224F@oracle.com> <592844C9-AFE8-47C8-92BC-A01DF78AF20D@oracle.com> <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC.7080102@oracle.com> <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> <56C1C4F3! .9000505@oracle.com> Message-ID: Thank you Alan. I?ll address the issues you bring up before integration. > On Feb 15, 2016, at 4:30 AM, Alan Bateman wrote: > > > On 10/02/2016 01:04, Steve Drach wrote: >> Hi, >> >> Yet another webrev, http://cr.openjdk.java.net/~sdrach/8132734/webrev.06/ , with a change to JarEntryIterator to fix a problem discovered by performance tests ? calling hasNext() twice as often as needed. I also removed the @since 9 tags from the methods entries() and stream(), and added an additional sentence to the spec for each of those methods that clarifies what a base entry is (actually is not). >> > I went through the latest webrev and it looks quite good. > > A few comments on the javadoc: > > "... partitioned by the major version of Java platform releases" - this might be better as "... partitioned by the major version of the Java platform". > > In JarFile.Release then it uses the phrase "top-most (base) directory". I thought we had purged "top-*" from the javadoc in previous iterations because it hints of classes or resources in the top most directory (which isn't the case with classes in a named package). > > "... will not be accessible by this JarFile" hints of access control or even security manager. Would it clearer to re-word this to something like "will not be located by methods such as getEntry" ? > > "returned depends whether" -> "returned depends on whether". > > In the javadoc for entries() and stream() then it mentions "the constructor" many times. I would be tempted to replace many of these - for example "all entries are returned, regardless of the constructor" might be better as "all entries of returned, regards of how the JarFile is created". > > A couple of nits on the implementation: > > vze = JarFile.super.getEntry(META_INF_VERSIONS + i-- + sname); > - it would be more readable if you move the decrement to its own line. Also I assume that JarFile is not needed here. > > L942-943 looks messy too, I assume that can be cleaned up. > > JarFileFactory - "earl" will confuse readers, needs a comment or a better name. > > I think this is all that I have for now. > > -Alan. From Alan.Bateman at oracle.com Mon Feb 15 16:35:03 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 15 Feb 2016 16:35:03 +0000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C1E765.7080603@oracle.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> Message-ID: <56C1FE37.9010507@oracle.com> On 15/02/2016 14:57, Chris Hegarty wrote: > Peter, > > > > http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/ > > Given the changes in 6857566, I think your proposal to remove > jdk.internal.ref.Cleaner should be good. ( Note: third party libraries > digging into internals of direct buffers will need to change again ). > I think so too, the main think is that the fast path for cleaners has been removed (and I see that it has). -Alan From brian.goetz at oracle.com Mon Feb 15 16:57:52 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 15 Feb 2016 11:57:52 -0500 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <56C123D3.8000608@oracle.com> References: <6728448.20160214205343@gmail.com> <56C123D3.8000608@oracle.com> Message-ID: The model that Tagir implemented has a natural analogue: the for loop. for (T v = seed; predicate.test(v); v = op.apply(v)) { ? } Like a for-loop, this proposed version of iterate() can generate an empty sequence; if the predicate applied to the seed returns false, you?ll get an empty stream. There are lots of for-loops that might want to be converted to streams, but don?t fit into the existing factories: for (T p = root; p != null; p = p.next) { ? } converts naturally to Stream.iterate(root, p -> p != null; p -> p.next)? Providing an initial value is not at odds with an empty sequence; it still gets tested by the predicate. On Feb 14, 2016, at 8:03 PM, Stuart Marks wrote: > Hi Tagir, > > Thanks for picking this up. > > I'll be at a conference this week and I won't have much time to discuss this in detail until afterward. But here are some quick thoughts about the proposal. > > I'd suggest focusing on the API first before worrying about how to track the stream state with booleans, etc. Is the API convenient to use, and how well does it support the use cases we envision for it? > > In particular, I can imagine a number of cases where it would be very helpful to be able to support an empty stream, or where the computation to produce the first element is the same as the computation to produce subsequent elements. Requiring a value for the first stream element is at odds with that. > > Here are some ideas for use cases to try out: > > - a series of dice rolls representing a round of craps [1] > - elements drawn from a queue until the queue is empty or until > a sentinel is reached > - a sequence of numbers that (probably) terminates but whose length > isn't necessarily known in advance (e.g. Collatz sequence [2]) > > [1] https://en.wikipedia.org/wiki/Craps > > [2] https://en.wikipedia.org/wiki/Collatz_conjecture > > Note that in some cases the sentinel value that terminates the stream should be part of the stream, and in other cases it's not. > > I'm sure you can find more uses cases by perusing Stack Overflow. :-) > > I'm a bit skeptical of the use of "iterate" for producing a finite stream. There are the usual issues with overloading, but there's also potential confusion as some forms of iterate() are infinite and others finite. I'll suggest the name "produce" instead, but there are surely better terms. > > One thing to think about is where the state of the producer is stored. Is it expected to be in an argument that's passed to each invocation of the functional argument, or is it expected to be captured? I don't think there's an answer in isolation; examining use cases would probably shed some light here. > > Here are a few API ideas (wildcards elided): > > -- > > Stream iterate(T seed, Predicate predicate, UnaryOperator f) > > The API from your proposal, for comparison purposes. > > -- > > Stream produce(Supplier>) > > Produces elements until empty Optional is returned. This box/unboxes every element, maybe(?) alleviated by Valhalla. > > -- > > Stream produce(BooleanSupplier, Supplier) > > Calls the BooleanSupplier; if true the next stream element is what's returned by calling the Supplier. If BooleanSupplier returns false, end of stream. If you have an iterator already, this enables > > produce(iterator::hasNext, iterator::next) > > But if you don't have an iterator already, coming up with the functions to satisfy the iterator-style protocol is sometimes painful. > > -- > > Stream produce(Predicate> advancer) > > This has an odd signature, but the function is like Spliterator.tryAdvance(). It must either call the consumer once and return true, or return false without calling the consumer. > > -- > > Stream produce(Consumer> advancer) > > A variation of the above, without a boolean return. The advancer calls the consumer one or more times to add elements to the stream. End of stream occurs when the advancer doesn't call the consumer. > > -- > > Stream produce(Supplier>) > > A variation of Supplier> where the supplier returns a stream containing zero or more elements. The stream terminates if the supplier returns an empty stream. There "boxing" overhead here, but we don't seem to be bothered by this with flatMap(). > > -- > > s'marks > > > On 2/14/16 6:53 AM, Tagir F. Valeev wrote: >> Hello! >> >> I wanted to work on foldLeft, but Brian asked me to take this issue >> instead. So here's webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ >> >> I don't like iterator-based Stream source implementations, so I made >> them AbstractSpliterator-based. I also implemented manually >> forEachRemaining as, I believe, this improves the performance in >> non-short-circuiting cases. >> >> I also decided to keep two flags (started and finished) to track the >> state. Currently existing implementation of infinite iterate() does >> not use started flag, but instead reads one element ahead for >> primitive streams. This seems wrong to me and may even lead to >> unexpected exceptions (*). I could get rid of "started" flag for >> Stream.iterate() using Streams.NONE, but this would make object >> implementation different from primitive implementations. It would also >> be possible to keep single three-state variable (byte or int, >> NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve >> the performance or footprint. Having two flags looks more readable to >> me. >> >> Currently existing two-arg iterate methods can now be expressed as a >> partial case of the new method: >> >> public static Stream iterate(final T seed, final UnaryOperator f) { >> return iterate(seed, x -> true, f); >> } >> (same for primitive streams). I may do this if you think it's >> reasonable. >> >> I created new test class and added new iterate sources to existing >> data providers. >> >> Please review and sponsor! >> >> With best regards, >> Tagir Valeev. >> >> (*) Consider the following code: >> >> int[] data = {1,2,3,4,-1}; >> IntStream.iterate(0, x -> data[x]) >> .takeWhile(x -> x >= 0) >> .forEach(System.out::println); >> >> Currently this unexpectedly throws an AIOOBE, because >> IntStream.iterate unnecessarily tries to read one element ahead. >> From uschindler at apache.org Mon Feb 15 17:06:19 2016 From: uschindler at apache.org (Uwe Schindler) Date: Mon, 15 Feb 2016 18:06:19 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C1FE37.9010507@oracle.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> Message-ID: <015201d16813$333650c0$99a2f240$@apache.org> Hi, > On 15/02/2016 14:57, Chris Hegarty wrote: > > Peter, > > > > > > > http://cr.openjdk.java.net/~plevart/jdk9- > dev/removeInternalCleaner/webrev.02/ > > > > Given the changes in 6857566, I think your proposal to remove > > jdk.internal.ref.Cleaner should be good. ( Note: third party libraries > > digging into internals of direct buffers will need to change again ). > > > I think so too, the main think is that the fast path for cleaners has > been removed (and I see that it has). What do you mean with fast path? The usual code pattern for forceful unmapping did not change: - Make java.nio.DirectByteBuffer#cleaner() accessible (this is the thing where you need to fight against the Java access checks). - call clean() (Runnable#run() in previous commit) on the returned object (with correct casting to interface before calling, this is where existing legacy code must change). In preparation for these changes, I made a patch for Lucene (Solr/Elasticsearch) to "compile" the whole unmapping logic into a MethodHandle (this allows me to do all needed accesibility/security checks early on startup and before actually invoking unmapping). This allows us to tell user that forceful unmapping works *before* we actually need to call it: https://issues.apache.org/jira/secure/attachment/12787878/LUCENE-6989.patch This will be part of Lucene 6.0 which will require Java 8 as minimum. Earlier Lucene version will not be compatible to Java 9 and can only be used in "slower" non-mmap mode. Thanks, Uwe From spliterator at gmail.com Mon Feb 15 17:15:39 2016 From: spliterator at gmail.com (Stefan Zobel) Date: Mon, 15 Feb 2016 18:15:39 +0100 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <56C174CA.9050800@gmail.com> References: <6728448.20160214205343@gmail.com> <56C174CA.9050800@gmail.com> Message-ID: 2016-02-15 7:48 GMT+01:00 Peter Levart : > > > What about even more permissive: > > static Stream iterate3(S seed, Predicate > predicate, Function f) > Hi Peter, yes, Function instead of UnaryOperator is indeed tempting. On the other hand, for my taste, I find the wildcard-ridden signature of iterate3 oversteps a certain complexity budget a little (that's just my personal opinion, of course). But you may be right - we already have such complex signatures in a couple of other places. The question is, whether it would be really useful. Regards, Stefan > > public class SignatureTest { > > static Stream iterate1(T seed, Predicate predicate, > UnaryOperator f) { > ... > } > > static Stream iterate2(S seed, Predicate > predicate, UnaryOperator f) { > ... > } > > static Stream iterate3(S seed, Predicate > predicate, Function f) { > ... > } > > > public static void main(String[] args) { > > Stream si1 = iterate1(0, i -> i < 10, i -> i + 1); // OK > Stream si2 = iterate2(0, i -> i < 10, i -> i + 1); // OK > Stream si3 = iterate3(0, i -> i < 10, i -> i + 1); // OK > > Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); > > //SignatureTest.java:32: error: bad operand types for binary > operator '<' > // Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); > // ^ > //SignatureTest.java:32: error: bad operand types for binary > operator '+' > // Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); > // ^ > > > Stream sn2 = iterate2(0, i -> i < 10, i -> i + 1); // OK > Stream sn3 = iterate3(0, i -> i < 10, i -> i + 1); // OK > > > Predicate csNotEmpty = cs -> cs.length() > 0; > > Stream css1 = iterate1("abcde", csNotEmpty, (String s) > -> s.substring(1)); > > //SignatureTest.java:44: error: method iterate1 in class > SignatureTest cannot be applied to given types; > // Stream css1 = iterate1("abcde", csNotEmpty, > (String s) -> s.substring(1)); > // ^ > // required: T,Predicate,UnaryOperator > // found: String,Predicate,(String s)[...]ng(1) > // reason: inference variable T has incompatible equality > constraints String,CharSequence > // where T is a type-variable: > // T extends Object declared in method > iterate1(T,Predicate,UnaryOperator) > > > Stream css2 = iterate2("abcde", csNotEmpty, (String s) > -> s.substring(1)); > > //SignatureTest.java:54: error: method iterate2 in class > SignatureTest cannot be applied to given types; > // Stream css2 = iterate2("abcde", csNotEmpty, > (String s) -> s.substring(1)); > // ^ > // required: S,Predicate,UnaryOperator > // found: String,Predicate,(String s)[...]ng(1) > // reason: inference variable S has incompatible equality > constraints String,CharSequence > // where S,T are type-variables: > // S extends T declared in method > iterate2(S,Predicate,UnaryOperator) > // T extends Object declared in method > iterate2(S,Predicate,UnaryOperator) > > > Stream css3 = iterate3("abcde", csNotEmpty, (String s) > -> s.substring(1)); // OK > } > } > > > Regards, Peter > > > > Regards, > Stefan > > > 2016-02-14 15:53 GMT+01:00 Tagir F. Valeev : > > Hello! > > I wanted to work on foldLeft, but Brian asked me to take this issue > instead. So here's webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ > > I don't like iterator-based Stream source implementations, so I made > them AbstractSpliterator-based. I also implemented manually > forEachRemaining as, I believe, this improves the performance in > non-short-circuiting cases. > > I also decided to keep two flags (started and finished) to track the > state. Currently existing implementation of infinite iterate() does > not use started flag, but instead reads one element ahead for > primitive streams. This seems wrong to me and may even lead to > unexpected exceptions (*). I could get rid of "started" flag for > Stream.iterate() using Streams.NONE, but this would make object > implementation different from primitive implementations. It would also > be possible to keep single three-state variable (byte or int, > NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve > the performance or footprint. Having two flags looks more readable to > me. > > Currently existing two-arg iterate methods can now be expressed as a > partial case of the new method: > > public static Stream iterate(final T seed, final UnaryOperator f) { > return iterate(seed, x -> true, f); > } > (same for primitive streams). I may do this if you think it's > reasonable. > > I created new test class and added new iterate sources to existing > data providers. > > Please review and sponsor! > > With best regards, > Tagir Valeev. > > (*) Consider the following code: > > int[] data = {1,2,3,4,-1}; > IntStream.iterate(0, x -> data[x]) > .takeWhile(x -> x >= 0) > .forEach(System.out::println); > > Currently this unexpectedly throws an AIOOBE, because > IntStream.iterate unnecessarily tries to read one element ahead. > > From brian.goetz at oracle.com Mon Feb 15 17:39:26 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Mon, 15 Feb 2016 12:39:26 -0500 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <6728448.20160214205343@gmail.com> References: <6728448.20160214205343@gmail.com> Message-ID: <56C20D4E.8080509@oracle.com> Overall the implementation choices are sound and the testing story is nicely integrated into the existing test framework. For the spec, I'd like to make the connection with the for-loop, and I'd like to clarify that this may produce an empty stream (if predicate.test(seed) is false.) /** 1208 * Returns a sequential ordered {@code Stream} produced by iterative 1209 * application of a function {@code f} to an initial element {@code seed}, 1210 * producing a {@code Stream} consisting of {@code seed}, {@code f(seed)}, 1211 * {@code f(f(seed))}, etc. The stream terminates when {@code predicate} 1212 * returns false. 1213 * 1214 *

    The first element (position {@code 0}) in the {@code Stream} will be 1215 * the provided {@code seed}. For {@code n > 0}, the element at position 1216 * {@code n}, will be the result of applying the function {@code f} to the 1217 * element at position {@code n - 1}. 1218 * 1219 * @param the type of stream elements 1220 * @param seed the initial element 1221 * @param predicate a predicate to apply to elements to determine when the 1222 * stream must terminate. 1223 * @param f a function to be applied to the previous element to produce 1224 * a new element 1225 * @return a new sequential {@code Stream} 1226 * @since 9 1227 */ Something like: Returns a sequential ordered Stream produced by iterative application of a function to an initial element, conditioned on satisfying the supplied predicate. The stream terminates as soon as the predicate function returns false. Stream.iterate should produce the same sequence of elements as produced by the corresponding for-loop: for (T index=seed; predicate.test(index); index = f.apply(index)) { ... } The resulting sequence may be empty if the predicate does not hold on the seed value. Otherwise the first element will be the supplied seed value, the next element (if present) will be the result of applying the function f to the seed value, and so on iteratively until the predicate indicates that the stream should terminate. On 2/14/2016 9:53 AM, Tagir F. Valeev wrote: > Hello! > > I wanted to work on foldLeft, but Brian asked me to take this issue > instead. So here's webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ > > I don't like iterator-based Stream source implementations, so I made > them AbstractSpliterator-based. I also implemented manually > forEachRemaining as, I believe, this improves the performance in > non-short-circuiting cases. > > I also decided to keep two flags (started and finished) to track the > state. Currently existing implementation of infinite iterate() does > not use started flag, but instead reads one element ahead for > primitive streams. This seems wrong to me and may even lead to > unexpected exceptions (*). I could get rid of "started" flag for > Stream.iterate() using Streams.NONE, but this would make object > implementation different from primitive implementations. It would also > be possible to keep single three-state variable (byte or int, > NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve > the performance or footprint. Having two flags looks more readable to > me. > > Currently existing two-arg iterate methods can now be expressed as a > partial case of the new method: > > public static Stream iterate(final T seed, final UnaryOperator f) { > return iterate(seed, x -> true, f); > } > (same for primitive streams). I may do this if you think it's > reasonable. > > I created new test class and added new iterate sources to existing > data providers. > > Please review and sponsor! > > With best regards, > Tagir Valeev. > > (*) Consider the following code: > > int[] data = {1,2,3,4,-1}; > IntStream.iterate(0, x -> data[x]) > .takeWhile(x -> x >= 0) > .forEach(System.out::println); > > Currently this unexpectedly throws an AIOOBE, because > IntStream.iterate unnecessarily tries to read one element ahead. > From joe.darcy at oracle.com Mon Feb 15 17:58:00 2016 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 15 Feb 2016 09:58:00 -0800 Subject: RFR JDK-8148624: Test failure of ConstructInflaterOutput.java In-Reply-To: <56BEB7FD.305@oracle.com> References: <56BEB7FD.305@oracle.com> Message-ID: <56C211A8.70704@oracle.com> Look fine Sherman; thanks, -Joe On 2/12/2016 8:58 PM, Xueming Shen wrote: > Hi, > > Please help review the change for JDK-8148624 > > Issue: https://bugs.openjdk.java.net/browse/JDK-8148624 > Webrev: http://cr.openjdk.java.net/~sherman/8148624/webrev/ > > The tests are intended to test/verify that the Deflater/Inflater.end() > method will > not be invoked when they are passed in as the parameter to the > constructor of > the DefaultInputStream and InflateOutputStream class, when the > corresponding > close&/finish() method are invoked. The stacktrace suggests that the > end() > method is not actually being called (failure) by the close/finish(), > but by the > finalize(). It appears the deflater/inflater is somehow being > finalized by gc > between A and B showed below > > public static void main(String[] args) throws Throwable { > try {realMain(args);} catch (Throwable t) {unexpected(t);} > [A]--> > System.out.println("\nPassed = " + passed + " failed = " + > failed); > [B]--> > if (failed > 0) throw new AssertionError("Some tests failed");} > > > the problem can be easily reproduced by inserting a System.gc() in > between. > > The easy fix is just move the inflater/deflater out of realMain() to > be the static variable > to prevent them from being gc-ed. > > Thanks, > Sherman From paul.sandoz at oracle.com Mon Feb 15 18:08:46 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 15 Feb 2016 19:08:46 +0100 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <56C20D4E.8080509@oracle.com> References: <6728448.20160214205343@gmail.com> <56C20D4E.8080509@oracle.com> Message-ID: <36790320-93A7-4D4A-8C3E-9A3DCDB71D81@oracle.com> > On 15 Feb 2016, at 18:39, Brian Goetz wrote: > > Overall the implementation choices are sound and the testing story is nicely integrated into the existing test framework. > > For the spec, I'd like to make the connection with the for-loop, and I'd like to clarify that this may produce an empty stream (if predicate.test(seed) is false.) > Yes, +1, that is the behaviour i expect, but it could be misconstrued without such clarification. I will take a closer look at the webrev tomorrow, i am not sure we need such test integration within the data providers, we have to be careful when increasing the overall time of test execution. ? Note that the example in (*) is somewhat concocted :-) the function needs to work correctly for an infinite sequence, and not be "cognisant? of what operations follow downstream, in that sense it is ok to be aggressive, and parallel execution may also consume more elements than is strictly necessary. For the three arg case we need exact control; the emulation iterate(s, uo).takeWhile(p) is not always sufficient, nor necessarily efficient. Paul. > /** > 1208 * Returns a sequential ordered {@code Stream} produced by iterative > 1209 * application of a function {@code f} to an initial element {@code seed}, > 1210 * producing a {@code Stream} consisting of {@code seed}, {@code f(seed)}, > 1211 * {@code f(f(seed))}, etc. The stream terminates when {@code predicate} > 1212 * returns false. > 1213 * > 1214 *

    The first element (position {@code 0}) in the {@code Stream} will be > 1215 * the provided {@code seed}. For {@code n > 0}, the element at position > 1216 * {@code n}, will be the result of applying the function {@code f} to the > 1217 * element at position {@code n - 1}. > 1218 * > 1219 * @param the type of stream elements > 1220 * @param seed the initial element > 1221 * @param predicate a predicate to apply to elements to determine when the > 1222 * stream must terminate. > 1223 * @param f a function to be applied to the previous element to produce > 1224 * a new element > 1225 * @return a new sequential {@code Stream} > 1226 * @since 9 > 1227 */ > > > Something like: > > Returns a sequential ordered Stream produced by iterative application of a function to an initial element, conditioned on satisfying the supplied predicate. The stream terminates as soon as the predicate function returns false. Stream.iterate should produce the same sequence of elements as produced by the corresponding for-loop: > > for (T index=seed; predicate.test(index); index = f.apply(index)) { ... } > > The resulting sequence may be empty if the predicate does not hold on the seed value. Otherwise the first element will be the supplied seed value, the next element (if present) will be the result of applying the function f to the seed value, and so on iteratively until the predicate indicates that the stream should terminate. > > > On 2/14/2016 9:53 AM, Tagir F. Valeev wrote: >> Hello! >> >> I wanted to work on foldLeft, but Brian asked me to take this issue >> instead. So here's webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ >> >> I don't like iterator-based Stream source implementations, so I made >> them AbstractSpliterator-based. I also implemented manually >> forEachRemaining as, I believe, this improves the performance in >> non-short-circuiting cases. >> >> I also decided to keep two flags (started and finished) to track the >> state. Currently existing implementation of infinite iterate() does >> not use started flag, but instead reads one element ahead for >> primitive streams. This seems wrong to me and may even lead to >> unexpected exceptions (*). I could get rid of "started" flag for >> Stream.iterate() using Streams.NONE, but this would make object >> implementation different from primitive implementations. It would also >> be possible to keep single three-state variable (byte or int, >> NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve >> the performance or footprint. Having two flags looks more readable to >> me. >> >> Currently existing two-arg iterate methods can now be expressed as a >> partial case of the new method: >> >> public static Stream iterate(final T seed, final UnaryOperator f) { >> return iterate(seed, x -> true, f); >> } >> (same for primitive streams). I may do this if you think it's >> reasonable. >> >> I created new test class and added new iterate sources to existing >> data providers. >> >> Please review and sponsor! >> >> With best regards, >> Tagir Valeev. >> >> (*) Consider the following code: >> >> int[] data = {1,2,3,4,-1}; >> IntStream.iterate(0, x -> data[x]) >> .takeWhile(x -> x >= 0) >> .forEach(System.out::println); >> >> Currently this unexpectedly throws an AIOOBE, because >> IntStream.iterate unnecessarily tries to read one element ahead. >> > From sandhya.viswanathan at intel.com Mon Feb 15 18:09:11 2016 From: sandhya.viswanathan at intel.com (Viswanathan, Sandhya) Date: Mon, 15 Feb 2016 18:09:11 +0000 Subject: RFR: JDK-8031767 Support system or alternative implementations of zlib In-Reply-To: References: <56B4F031.6040501@oracle.com> <56B862FE.5000000@oracle.com> <56B86623.4090602@oracle.com> <56BB41E6.3060102@oracle.com> <56BB4932.7060105@oracle.com> <56BB4C24.7090607@oracle.com> <56BBD50F.8070803@oracle.com> <56BC63AC.9050702@oracle.com> Message-ID: <02FCFB8477C4EF43A2AD8E0C60F3DA2B75955B9F@FMSMSX112.amr.corp.intel.com> I would like to reiterate the need for system zlib support by the JVM instead of bundled zlib from the performance point of view. Compression is one of the hotspots in Big Data, Genomics and Middleware applications. There are many solutions in market today which help improve compression performance either through software or hardware acceleration. Intel has faster compression libraries as part of IPP as Sherman mentioned. Also there are hardware compression accelerators from Intel like QuickAssist. Both of these provide the acceleration through the zlib interface. Support for system zlib by the JVM would make these acceleration capabilities available to Java applications almost seamlessly through java.util.zip. Best Regards, Sandhya -----Original Message----- From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On Behalf Of Martin Buchholz Sent: Thursday, February 11, 2016 10:49 AM To: Se?n Coffey Cc: build-dev; core-libs-dev Subject: Re: RFR: JDK-8031767 Support system or alternative implementations of zlib Currently the pendulum is swinging away from multiple applications sharing common libraries towards every application being self-contained, perhaps because disk space is dirt cheap and because of the rise of "containers". It may be that much of the packaging of jdks will be picked up by third parties who package up the JDK and any of its dependencies - e.g. I already see https://hub.docker.com/_/java/ On Thu, Feb 11, 2016 at 2:34 AM, Se?n Coffey wrote: > > On 11/02/2016 00:25, Xueming Shen wrote: >> >> >> One of the benefits of moving to the system libz is actually for >> better/easy >> maintenance. Just replacing the offending version of libz with an >> earlier/later >> version that works, instead of waiting for a customized jdk/jre image with >> a >> working/bundled libz, or the next update release. Especially given the >> fact >> that we have decided not to touch the libz at source level. Having >> dependency >> on the system libz is really not that bad. The experience suggests those >> binaries are quite stable. And it should always be easier to replace the >> system libz than a java runtime, in case of problem. > > > I think people overestimate people's ability to 'just replace' a zlib > library. Adding a new system property is a struggle for some enterprises. > We'll enjoy supporting a many to one matrix of zlib:JDK scenarios now rather > than old style 1:1. It's great to have system library support - I'm just > highlighting a possible risk. A fall back option solves that but there's no > appetite for such a solution. Let's see how it goes. > > regards, > Sean. > >> >> Sherman >> >> On 02/10/2016 06:41 AM, Se?n Coffey wrote: >>> >>> >>> On 10/02/16 14:29, Alan Bateman wrote: >>>> >>>> >>>> On 10/02/2016 13:57, Se?n Coffey wrote: >>>>> >>>>> I'm all for allowing one to introduce a new version of zlib to their >>>>> JDK at runtime. I just don't think it's in the interests of enterprises and >>>>> stability to introduce a dependency to the JDK on the underlying OS zlib >>>>> libraries. Could we at least consider a runtime property to allow linking to >>>>> the (currently bundled) zlib v.1.2.8 port in case issues arise ? >>>> >>>> Don't the LD_* environment variables serve this need already? Once the >>>> JDK is using the system zlib then this is the simplest way to get it to use >>>> a different libz library at runtime. >>> >>> No - I don't see that as a solution. You've still made the default JDK >>> config become dependent on OS environment for all libzip operations. I don't >>> think we even capture the zlib version that the JDK would be operating with >>> in any diagnostics. An application wanting a tried/tested and stable libzip >>> version has extra work to do now. Letting the default be system dependent >>> has just increased risk for QA teams also. A system property just makes this >>> all go away. In fact - I would say that for JDK9, the default should be the >>> JDK bundled libzip library. For those looking for libzip experimenting and >>> performance benefits, they could take the system property approach. >>> >>> regards, >>> Sean. >>>> >>>> >>>> -Alan >>> >>> >>> >>> On 08/02/16 09:55, Alan Bateman wrote: >>>> >>>> >>>> On 08/02/2016 10:42, Se?n Coffey wrote: >>>>> >>>>> Is there an option to fall back to the older v.1.2.8 implementation if >>>>> necessary ? It would certainly alleviate issues for any applications that >>>>> might run into issues with the latest and greatest zlib libraries. >>>>> JDK-8133206 would be one such example. >>>> >>>> Just at build time >>> >>> so - we introduce a runtime dependency on the underlying zlib libraries >>> on the OS by default. I would be very concerned with this approach. We've >>> run JDK 6 for 10+ years with zlib v1.1.3. It was consistent and reliable for >>> the most part. When we moved JDK7/8 to zlib v1.2.5, we encountered an >>> inflation issue[1]. When JDK was upgraded to zlib v1.2.8, we received >>> reports of performance/OOM issues [2]. >>>> >>>> but if the zlib on the platform is broken then it impacts tools and >>>> probably lots of things. I would assume the OS would patch such issues >>>> quickly. In the case of JDK-8133206 then was the issue addressed in the >>>> libzip wrapper or in the zlib code? I thought it was the former. >>> >>> The code change is proposed in the libzip wrapper but the issue was >>> triggered by the zlib library update. >>>> >>>> >>>> On a fallback or some way to configure at launch time then Sandhya >>>> Viswanathan (Intel) has a proposal here last year. I think we mostly agreed >>>> on that thread that switching the build to use the system zlib by default >>>> would be better. >>> >>> I'm all for allowing one to introduce a new version of zlib to their JDK >>> at runtime. I just don't think it's in the interests of enterprises and >>> stability to introduce a dependency to the JDK on the underlying OS zlib >>> libraries. Could we at least consider a runtime property to allow linking to >>> the (currently bundled) zlib v.1.2.8 port in case issues arise ? >>> >>> regards, >>> Sean. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8044725 >>> [2] https://bugs.openjdk.java.net/browse/JDK-8133206 >>> >> > From vladimir.x.ivanov at oracle.com Mon Feb 15 19:01:47 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 15 Feb 2016 22:01:47 +0300 Subject: [9] RFR (XS): 8148518: Unsafe.getCharUnaligned() loads aren't folded in case of -XX:-UseUnalignedAccesses Message-ID: <56C2209B.8090604@oracle.com> http://cr.openjdk.java.net/~vlivanov/8148518/webrev.00/ https://bugs.openjdk.java.net/browse/JDK-8148518 Current Unsafe::getCharUnaligned Java implementation isn't fully optimized by C2: it never constant folds such loads from char fields. C2 matches the types of a load and a location and does constant folding only if they match. For getCharUnaligned it sees a load of a short value, but the field is of type char. The fix is to call Unsafe::getChar when offset is aligned. It doesn't matter what is used for unaligned case, so I decided to keep makeShort(). Testing: failing test, JPRT. Best regards, Vladimir Ivanov PS: I don't update the test because it was refactored in Jigsaw [1]. To avoid unnecessary conflicts during merges, I filed an RFE [2] to adjust the test once the fix and Jake are integrated. [1] http://hg.openjdk.java.net/jigsaw/jake/hotspot/file/f6daf3633512/test/compiler/unsafe/UnsafeGetConstantField.java [2] https://bugs.openjdk.java.net/browse/JDK-8149844 From vladimir.kozlov at oracle.com Mon Feb 15 19:45:58 2016 From: vladimir.kozlov at oracle.com (Vladimir Kozlov) Date: Mon, 15 Feb 2016 11:45:58 -0800 Subject: [9] RFR (XS): 8148518: Unsafe.getCharUnaligned() loads aren't folded in case of -XX:-UseUnalignedAccesses In-Reply-To: <56C2209B.8090604@oracle.com> References: <56C2209B.8090604@oracle.com> Message-ID: <56C22AF6.10007@oracle.com> Looks good to me. Thanks, Vladimir K On 2/15/16 11:01 AM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8148518/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8148518 > > Current Unsafe::getCharUnaligned Java implementation isn't fully optimized by C2: it never constant folds such loads > from char fields. > > C2 matches the types of a load and a location and does constant folding only if they match. For getCharUnaligned it sees > a load of a short value, but the field is of type char. > > The fix is to call Unsafe::getChar when offset is aligned. It doesn't matter what is used for unaligned case, so I > decided to keep makeShort(). > > Testing: failing test, JPRT. > > Best regards, > Vladimir Ivanov > > PS: I don't update the test because it was refactored in Jigsaw [1]. To avoid unnecessary conflicts during merges, I > filed an RFE [2] to adjust the test once the fix and Jake are integrated. > > [1] http://hg.openjdk.java.net/jigsaw/jake/hotspot/file/f6daf3633512/test/compiler/unsafe/UnsafeGetConstantField.java > > [2] https://bugs.openjdk.java.net/browse/JDK-8149844 From mandy.chung at oracle.com Mon Feb 15 20:05:09 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Mon, 15 Feb 2016 12:05:09 -0800 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <015201d16813$333650c0$99a2f240$@apache.org> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> Message-ID: > On Feb 15, 2016, at 9:06 AM, Uwe Schindler wrote: > > Hi, > >> On 15/02/2016 14:57, Chris Hegarty wrote: >>> Peter, >>> >>> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/ >>> This patch looks correct to me. The innocuous thread created for common cleaner is of Thread.MAX_PRIORITY - 2 priority whereas the reference handler thread is of MAX_PRIORITY priority. I don?t know if this would impact any real-world application or whether worth having a dedicated thread with MAX_PRIORTY (David Holmes may have recommendation to this). Another subtle difference is no more package access check whether running with security manager. On the other hand, ?suppressAccessChecks? permission is still checked. I don?t have issue with this. >>> Given the changes in 6857566, I think your proposal to remove >>> jdk.internal.ref.Cleaner should be good. ( Note: third party libraries >>> digging into internals of direct buffers will need to change again ). >>> >> I think so too, the main think is that the fast path for cleaners has >> been removed (and I see that it has). > > What do you mean with fast path? I believe Alan refers to the fast path handling jdk.internal.ref.Cleaner via the Reference Handler thread. What this means to existing code is that the java.nio.DirectByteBuffer::cleaner method still exists but with java.lang.ref.Cleaner.Cleanable return type. You still need to call setAccessible(true) on the cleaner method. You can then statically reference java.lang.ref.Cleaner.Cleanable::clean method Mandy > The usual code pattern for forceful unmapping did not change: > - Make java.nio.DirectByteBuffer#cleaner() accessible (this is the thing where you need to fight against the Java access checks). > - call clean() (Runnable#run() in previous commit) on the returned object (with correct casting to interface before calling, this is where existing legacy code must change). > > In preparation for these changes, I made a patch for Lucene (Solr/Elasticsearch) to "compile" the whole unmapping logic into a MethodHandle (this allows me to do all needed accesibility/security checks early on startup and before actually invoking unmapping). This allows us to tell user that forceful unmapping works *before* we actually need to call it: https://issues.apache.org/jira/secure/attachment/12787878/LUCENE-6989.patch > > This will be part of Lucene 6.0 which will require Java 8 as minimum. Earlier Lucene version will not be compatible to Java 9 and can only be used in "slower" non-mmap mode. > > Thanks, > Uwe > From aleksey.shipilev at oracle.com Mon Feb 15 20:06:31 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Mon, 15 Feb 2016 23:06:31 +0300 Subject: [9] RFR (XS): 8148518: Unsafe.getCharUnaligned() loads aren't folded in case of -XX:-UseUnalignedAccesses In-Reply-To: <56C2209B.8090604@oracle.com> References: <56C2209B.8090604@oracle.com> Message-ID: <56C22FC7.30808@oracle.com> On 02/15/2016 10:01 PM, Vladimir Ivanov wrote: > http://cr.openjdk.java.net/~vlivanov/8148518/webrev.00/ > https://bugs.openjdk.java.net/browse/JDK-8148518 +1 > Current Unsafe::getCharUnaligned Java implementation isn't fully > optimized by C2: it never constant folds such loads from char fields. > C2 matches the types of a load and a location and does constant folding > only if they match. For getCharUnaligned it sees a load of a short > value, but the field is of type char. Does the same affect putCharUnaligned? Cheers, -Aleksey From xueming.shen at oracle.com Mon Feb 15 22:45:48 2016 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 15 Feb 2016 14:45:48 -0800 Subject: RFR: 8135108: java/util/zip/TestLocalTime.java fails intermittently with Invalid value for NanoOfSecond Message-ID: <56C2551C.8030805@oracle.com> Hi, Please help review the changes for JDK-8135108 issue: https://bugs.openjdk.java.net/browse/JDK-8135108 webrev: http://cr.openjdk.java.net/~sherman/8135108 It's a overflow of the low 32-bit of xdostime when the year is > 0x70 (which will be shifting << 25). We have a similar patch at ZipUtils.javaToDosTime(). Obviously ZipEntry.setTimeLocale() needs one as well. /** * Converts Java time to DOS time. */ private static long javaToDosTime(long time) { Instant instant = Instant.ofEpochMilli(time); LocalDateTime ldt = LocalDateTime.ofInstant( instant, ZoneId.systemDefault()); int year = ldt.getYear() - 1980; if (year < 0) { return (1 << 21) | (1 << 16); } return (year << 25 | ldt.getMonthValue() << 21 | ldt.getDayOfMonth() << 16 | ldt.getHour() << 11 | ldt.getMinute() << 5 | ldt.getSecond() >> 1) & 0xffffffffL; } Thanks! Sherman From joe.darcy at oracle.com Tue Feb 16 01:25:47 2016 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 15 Feb 2016 17:25:47 -0800 Subject: RFR: 8135108: java/util/zip/TestLocalTime.java fails intermittently with Invalid value for NanoOfSecond In-Reply-To: <56C2551C.8030805@oracle.com> References: <56C2551C.8030805@oracle.com> Message-ID: <56C27A9B.6090801@oracle.com> Looks fine Sherman; thanks, -Joe On 2/15/2016 2:45 PM, Xueming Shen wrote: > Hi, > > Please help review the changes for JDK-8135108 > > issue: https://bugs.openjdk.java.net/browse/JDK-8135108 > webrev: http://cr.openjdk.java.net/~sherman/8135108 > > It's a overflow of the low 32-bit of xdostime when the year is > 0x70 > (which will be shifting << 25). We have a similar patch at > ZipUtils.javaToDosTime(). Obviously ZipEntry.setTimeLocale() needs > one as well. > > /** > * Converts Java time to DOS time. > */ > private static long javaToDosTime(long time) { > Instant instant = Instant.ofEpochMilli(time); > LocalDateTime ldt = LocalDateTime.ofInstant( > instant, ZoneId.systemDefault()); > int year = ldt.getYear() - 1980; > if (year < 0) { > return (1 << 21) | (1 << 16); > } > return (year << 25 | > ldt.getMonthValue() << 21 | > ldt.getDayOfMonth() << 16 | > ldt.getHour() << 11 | > ldt.getMinute() << 5 | > ldt.getSecond() >> 1) & 0xffffffffL; > } > > Thanks! > Sherman From david.holmes at oracle.com Tue Feb 16 05:24:49 2016 From: david.holmes at oracle.com (David Holmes) Date: Tue, 16 Feb 2016 15:24:49 +1000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> Message-ID: <56C2B2A1.6020200@oracle.com> Hi Mandy, On 16/02/2016 6:05 AM, Mandy Chung wrote: > >> On Feb 15, 2016, at 9:06 AM, Uwe Schindler wrote: >> >> Hi, >> >>> On 15/02/2016 14:57, Chris Hegarty wrote: >>>> Peter, >>>> >>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/ >>>> > > This patch looks correct to me. The innocuous thread created for common cleaner is of Thread.MAX_PRIORITY - 2 priority whereas the reference handler thread is of MAX_PRIORITY priority. I don?t know if this would impact any real-world application or whether worth having a dedicated thread with MAX_PRIORTY (David Holmes may have recommendation to this). How did you know I would read this? :) Thread priority has no meaning on Solaris, Linux or BSD/OSX by default. Only Windows actually applies thread priorities under normal conditions. The difference between MAX_PRIORITY and MAX_PRIORITY-2 is the former uses THREAD_PRIORITY_HIGHEST and the latter THREAD_PRIORITY_ABOVE_NORMAL. Without any real/reasonable workloads for benchmarking it is all somewhat arbitrary. Arguably the Reference handler thread has more work to do in general so might be better given the higher priority. Cheers, David ----- > Another subtle difference is no more package access check whether running with security manager. On the other hand, ?suppressAccessChecks? permission is still checked. I don?t have issue with this. > >>>> Given the changes in 6857566, I think your proposal to remove >>>> jdk.internal.ref.Cleaner should be good. ( Note: third party libraries >>>> digging into internals of direct buffers will need to change again ). >>>> >>> I think so too, the main think is that the fast path for cleaners has >>> been removed (and I see that it has). >> >> What do you mean with fast path? > > I believe Alan refers to the fast path handling jdk.internal.ref.Cleaner via the Reference Handler thread. > > What this means to existing code is that the java.nio.DirectByteBuffer::cleaner method still exists but with java.lang.ref.Cleaner.Cleanable return type. You still need to call setAccessible(true) on the cleaner method. You can then statically reference java.lang.ref.Cleaner.Cleanable::clean method > > > Mandy > >> The usual code pattern for forceful unmapping did not change: >> - Make java.nio.DirectByteBuffer#cleaner() accessible (this is the thing where you need to fight against the Java access checks). >> - call clean() (Runnable#run() in previous commit) on the returned object (with correct casting to interface before calling, this is where existing legacy code must change). >> >> In preparation for these changes, I made a patch for Lucene (Solr/Elasticsearch) to "compile" the whole unmapping logic into a MethodHandle (this allows me to do all needed accesibility/security checks early on startup and before actually invoking unmapping). This allows us to tell user that forceful unmapping works *before* we actually need to call it: https://issues.apache.org/jira/secure/attachment/12787878/LUCENE-6989.patch >> >> This will be part of Lucene 6.0 which will require Java 8 as minimum. Earlier Lucene version will not be compatible to Java 9 and can only be used in "slower" non-mmap mode. >> >> Thanks, >> Uwe >> > From joe.darcy at oracle.com Tue Feb 16 06:23:19 2016 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 15 Feb 2016 22:23:19 -0800 Subject: JDK 9 RFR of JDK-8149896: Remove unnecessary values in FloatConsts and DoubleConsts Message-ID: <56C2C057.4090000@oracle.com> Hello, The the FloatConsts and DoubleConsts classes, while moved to an internal package recently (JDK-8145990), contain constants now available via the public API. All such uses of the redundant values should be removed as well as the redundant constants themselves. A quick note on the 2d changes, several constants (and a copy from a package-private method from java.lang) were used to initialize a double value POWER_2_TO_32; this can be accomplished more directly using a hexadecimal floating-point literal. Please review the webrev http://cr.openjdk.java.net/~darcy/8149896.0/ Thanks, -Joe From aleksey.shipilev at oracle.com Tue Feb 16 08:59:28 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 16 Feb 2016 11:59:28 +0300 Subject: JDK 9 RFR of JDK-8149896: Remove unnecessary values in FloatConsts and DoubleConsts In-Reply-To: <56C2C057.4090000@oracle.com> References: <56C2C057.4090000@oracle.com> Message-ID: <56C2E4F0.5030700@oracle.com> On 02/16/2016 09:23 AM, joe darcy wrote: > Please review the webrev > > http://cr.openjdk.java.net/~darcy/8149896.0/ +1, nice cleanup. -Aleksey (not a Reviewer) From paul.sandoz at oracle.com Tue Feb 16 09:05:04 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 16 Feb 2016 10:05:04 +0100 Subject: JDK 9 RFR of JDK-8149896: Remove unnecessary values in FloatConsts and DoubleConsts In-Reply-To: <56C2C057.4090000@oracle.com> References: <56C2C057.4090000@oracle.com> Message-ID: <85D3D7D7-2D23-4A10-947A-05D4872B117E@oracle.com> > On 16 Feb 2016, at 07:23, joe darcy wrote: > > Hello, > > The the FloatConsts and DoubleConsts classes, while moved to an internal package recently (JDK-8145990), contain constants now available via the public API. All such uses of the redundant values should be removed as well as the redundant constants themselves. > > A quick note on the 2d changes, several constants (and a copy from a package-private method from java.lang) were used to initialize a double value POWER_2_TO_32; this can be accomplished more directly using a hexadecimal floating-point literal. > > Please review the webrev > > http://cr.openjdk.java.net/~darcy/8149896.0/ > +1 My favourite change is this one: - private static final double POWER_2_TO_32 = FloatMath.powerOfTwoD(32); + private static final double POWER_2_TO_32 = 0x1.0p32; and the removal of the method on FloatMath. Paul. From bourges.laurent at gmail.com Tue Feb 16 10:08:53 2016 From: bourges.laurent at gmail.com (=?UTF-8?Q?Laurent_Bourg=C3=A8s?=) Date: Tue, 16 Feb 2016 11:08:53 +0100 Subject: [OpenJDK 2D-Dev] JDK 9 RFR of JDK-8149896: Remove unnecessary values in FloatConsts and DoubleConsts In-Reply-To: <85D3D7D7-2D23-4A10-947A-05D4872B117E@oracle.com> References: <56C2C057.4090000@oracle.com> <85D3D7D7-2D23-4A10-947A-05D4872B117E@oracle.com> Message-ID: Hello, Joe, I tested your changes in the Marlin renderer as it works well ! > A quick note on the 2d changes, several constants (and a copy from a > package-private method from java.lang) were used to initialize a double > value POWER_2_TO_32; this can be accomplished more directly using a > hexadecimal floating-point literal. > > > > Please review the webrev > > > > http://cr.openjdk.java.net/~darcy/8149896.0/ > ... > My favourite change is this one: > > - private static final double POWER_2_TO_32 = FloatMath.powerOfTwoD(32); > + private static final double POWER_2_TO_32 = 0x1.0p32; > > and the removal of the method on FloatMath. > I agree it is simpler and exactly what I needed; I wasn't aware of such literals: https://blogs.oracle.com/darcy/entry/hexadecimal_floating_point_literals (I am not a reviewer) Thanks for the fix, Laurent From peter.firmstone at zeus.net.au Tue Feb 16 10:40:25 2016 From: peter.firmstone at zeus.net.au (Peter Firmstone) Date: Tue, 16 Feb 2016 20:40:25 +1000 Subject: DeserializationPermission Proposal Message-ID: <56C2FC99.3030800@zeus.net.au> Hi Dave, You've pretty much got it right, although I'm not proposing making the permissions fine grained to class name level. There are different DeSerializationPermission's however: "PROXY" - controls dynamic proxy creation (Some basic invariants are checked prior, the InvocationHandler field object is passed to the dynamic proxy's constructor). "ATOMIC" - controls which domains are allowed to use atomic deserialization constructor. "MARSHAL" - allows deserialization of MarshalledObject instances (Again invariants are checked prior). "EXTERNAL" - allows deserialization of or Externalizable objects. Serializable classes that are stateless or have only primitive fields don't require any permission and remain unchanged. Serializable classes that contain Object fields or reads information directly from the stream require an atomic constructor, identified with @AtomicSerial, otherwise they are not Serializable. There is also an atomic constructor available for Externalizable objects, identified with @AtomicExternal, however this isn't required. AtomicMarshalInputStream also provides a T readObject(Class type) method that reads ahead and prevents deserialization if the object isn't the correct type. There's another annotation for readResolve methods to identify the replacement object type, as readResolve doesn't allow covariant return types. Some debugging output: NonActGrp-out: access: access allowed ("java.util.PropertyPermission" "sun.net.maxDatagramSockets" "read") NonActGrp-out: access: access allowed ("java.net.SocketPermission" "localhost:4160" "listen,resolve") NonActGrp-out: access: access allowed ("java.net.SocketPermission" "224.0.1.84" "connect,accept,resolve") NonActGrp-out: access: access allowed ("java.io.FilePermission" "\C:\Users\peter\Documents\NetBeansProjects\input-validation-for-serialization\lib\mahalo.jar" "read") NonActGrp-out: access: access allowed ("java.lang.RuntimePermission" "accessClassInPackage.sun.reflect") NonActGrp-out: access: access allowed ("java.net.SocketPermission" "localhost:0" "listen,resolve") NonActGrp-out: access: access allowed ("java.lang.RuntimePermission" "accessDeclaredMembers") NonActGrp-out: access: access allowed ("net.jini.discovery.DiscoveryPermission" "QATestDefaultGroup_medusa_1455617825772") NonActGrp-out: access: access allowed ("java.io.SerializablePermission" "enableSubclassImplementation") NonActGrp-out: access: access allowed ("java.io.SerializablePermission" "enableSubstitution") NonActGrp-out: access: AccessControlContext invoking the Combiner NonActGrp-out: access: access allowed ("org.apache.river.api.io.DeSerializationPermission" "ATOMIC") NonActGrp-out: access: AccessControlContext invoking the Combiner NonActGrp-out: access: access allowed ("org.apache.river.api.io.DeSerializationPermission" "PROXY") NonActGrp-out: access: AccessControlContext invoking the Combiner NonActGrp-out: access: access allowed ("org.apache.river.api.io.DeSerializationPermission" "ATOMIC") NonActGrp-out: access: access allowed ("java.lang.RuntimePermission" "createSecurityManager") NonActGrp-out: access: access allowed ("org.apache.river.api.io.DeSerializationPermission" "ATOMIC") NonActGrp-out: access: access allowed ("java.io.SerializablePermission" "enableSubclassImplementation") NonActGrp-out: access: access allowed ("java.io.SerializablePermission" "enableSubstitution") Regards, Peter. > Things are getting confused (for me anyway) so I'm going to try and > spell out each idea separately. > > Given a class hierarchy H of: > > C -> B -> A -> NonSerializableClass -> Object > > where C, B, and A are serializable classes, and protection domain M > which is initiating deserialization, I think the following requirements > definitely apply: > > * When M deserializes an instance of A, both M and the protection domain > of A must be sufficient to allow the deserialization of A > * When M deserializes an instance of B, both M and the protection domain > of B must be sufficient to allow the deserialization of B > * When M deserializes an instance of C, both M and the protection domain > of C must be sufficient to allow the deserialization of C > * Extrapolate generally > > The following requirements have definite pros and cons, which is what I > think we're touching on in this thread: > > * When M deserializes an instance of B (or any subtype thereof), the > protection domain of A must also be sufficient to allow the > deserialization of B > * When M deserializes an instance of C (or any subtype thereof), the > protection domains of both A and B must also be sufficient to allow the > deserialization of C > * Extrapolate generally > > Based on the above, the following requirements might or might not make > sense: > > * When M deserializes an instance of A, the protection domains of > NonSerializableClass and also Object must also be sufficient to allow > the deserialization of A > * Likewise for B& C, extrapolate generally > > Regardless of which is the best set of requirements, one of the major > complicating questions is, what is the ideal representation of the > permission(s) needed to allow the above? I can see several use cases: > > 1. Grant the ability for a serializable class S to deserialize itself > 2. Grant the ability for M to deserialize a specific class (by name?) > 3. Grant the ability to any (serializable?) class X to allow > deserialization of a specific subtype (by name?) > 4. Grant the ability to any (serializable?) class X to allow > deserialization of any subtype (think java.lang.Object, or general (but > harmless) base classes like java.util.EventObject) > > The big problem with permissions by name is that names are completely > relative. It is totally valid (and happens in the wild today) that a > serialization graph contains classes from a variety of class loaders > which are isolated from one another - in fact, it's even possible to > have more than one class with the same class name and the same > superclass in the same graph! Given this fact, a flat class namespace > *cannot* be assumed. > > I think that the *only* possible remedy for this is that it should be up > to each class' class loader (and/or module) to determine what permission > is required in order to deserialize that class. In this way, the JDK > (9) can use a permission of the form class+module name for installed > module classes, Java EE systems could use permissions in the form of > application+module+jar+class name for deployment, OSGi systems could use > permissions including bundle identification information, etc. > > To be secure, the default class loader implementation of this method > would have to produce a result which causes all checks to fail, > preventing deserialization in a security manager environment of classes > whose class loaders do not explicitly allow for it. > > On 02/12/2016 12:30 AM, Peter Firmstone wrote: > >/ Thanks David, > />/ > />/ I'd originally considered something like that, but I later realised I'd > />/ need to grant DeserializationPermission to other domains I might not > />/ want allow objects to be deserialized with, simply because their domain > />/ is included in the call stack. It looked like it would widen the scope, > />/ which was contrary to the intended goal of minimising the attack > />/ surface. In the end I decided to limit the context to only the domains > />/ of classes involved in deserialization. > />/ > />/ Cheers, > />/ > />/ Peter. > />/ > />>/ On 02/11/2016 03:52 AM, Peter Firmstone wrote: > />>/ >/ An example might be more useful. > />>/ />/ > />>/ />/ Traditionally, when the first non serializable superclass zero > />>/ argument > />>/ />/ constructor is called, the domains of the subclasses aren't > />>/ present on > />>/ />/ the call stack. Any security checks performed in the constructor > />>/ of the > />>/ />/ superclass will not include the subclasses domains. > />>/ />/ > />>/ />/ In the proposed case, prior to construction, all domains in the > />>/ class > />>/ />/ heirarchy of the to be deserialized object via the local > />>/ />/ ObjectStreamClass, are added to an AccessControlContext, which is > />>/ then > />>/ />/ passed to the SecurityManager two argument permission check. > />>/ / > />>/ Sure, that makes sense; in fact this could be a very good > />>/ standalone/incremental enhancement. > />>/ > />>/ >/ Now an attacker will not be able to construct this object unless all > />>/ />/ domains have DeserializationPermission. > />>/ />/ > />>/ />/ If we use the stack context, it won't contain the yet to be > />>/ deserialized > />>/ />/ classes. > />>/ / > />>/ True; perhaps the ideal solution would use the initial context plus a > />>/ per-deserializing-class context, so that both the original caller and > />>/ the class being deserialized have permissions. This would have the > />>/ advantage of consistent behavior, and would also allow the PD of each > />>/ class to restrict whether it can be deserialized (which would apply > />>/ globally no matter who was doing the deserialization in the VM). > />>/ > />>/ -- > />>/ - DML > / > -- > - DML From vladimir.x.ivanov at oracle.com Tue Feb 16 10:56:26 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 16 Feb 2016 13:56:26 +0300 Subject: [9] RFR (XS): 8148518: Unsafe.getCharUnaligned() loads aren't folded in case of -XX:-UseUnalignedAccesses In-Reply-To: <56C22FC7.30808@oracle.com> References: <56C2209B.8090604@oracle.com> <56C22FC7.30808@oracle.com> Message-ID: <56C3005A.4010109@oracle.com> Vladimir, Aleksey, thanks for the reviews. >> Current Unsafe::getCharUnaligned Java implementation isn't fully >> optimized by C2: it never constant folds such loads from char fields. >> C2 matches the types of a load and a location and does constant folding >> only if they match. For getCharUnaligned it sees a load of a short >> value, but the field is of type char. > > Does the same affect putCharUnaligned? No, the logic is specific to unsafe loads. Best regards, Vladimir Ivanov From christoph.langer at sap.com Tue Feb 16 12:49:26 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 16 Feb 2016 12:49:26 +0000 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE Message-ID: Hi, can you please review the following change for JDK-8149915: http://cr.openjdk.java.net/~clanger/webrevs/8149915.0/ I'm not sure if it is the right thing to set the default for XML_SECURITY_PROPERTY_MANAGER to a new XMLSecurityPropertyManager() object instead of the String "all". But it must not be a String, I think. And also I don't know if my approach to add a field "fSecurityManager" to XML11Configuration. Please advise and let me know how I can do it better. Thanks in advance and best regards Christoph From paul.sandoz at oracle.com Tue Feb 16 13:32:33 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 16 Feb 2016 14:32:33 +0100 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <6728448.20160214205343@gmail.com> References: <6728448.20160214205343@gmail.com> Message-ID: <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> Hi Tagir, The implementation approach looks good. I agree with Brian on the documentation, esp. about the connection to a for loop. Testing-wise adding to the spliterator data sources is ok. Because you have done that there is no need to explicitly use SpliteratorTestHelper in IterateTest, as that is reasonably covered by SpliteratorTest. Since you have the data providers for streams just run it through exerciseStream instead. Regarding wild cards we could clean this up later and include other methods [1]. It?s easy to go overboard on this, recommend sticking to ? extends/? super where appropriate. The existing iterate methods use UnaryOperator, thus there is no wiggle room there, am i ok with reusing that for the 3-arg method. Regarding the existing two-args method, my inclination is to keep the implementations separate from the 3-args, but feel free to change to using abstract spliterator (reduce layering) using a similar code pattern (no need for "finished" or overriding forEachRemaining). Then you can remove Streams.NONE and it?s sneaky erased usage :-) Paul. [1] https://bugs.openjdk.java.net/browse/JDK-8132097 > On 14 Feb 2016, at 15:53, Tagir F. Valeev wrote: > > Hello! > > I wanted to work on foldLeft, but Brian asked me to take this issue > instead. So here's webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ > > I don't like iterator-based Stream source implementations, so I made > them AbstractSpliterator-based. I also implemented manually > forEachRemaining as, I believe, this improves the performance in > non-short-circuiting cases. > > I also decided to keep two flags (started and finished) to track the > state. Currently existing implementation of infinite iterate() does > not use started flag, but instead reads one element ahead for > primitive streams. This seems wrong to me and may even lead to > unexpected exceptions (*). I could get rid of "started" flag for > Stream.iterate() using Streams.NONE, but this would make object > implementation different from primitive implementations. It would also > be possible to keep single three-state variable (byte or int, > NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve > the performance or footprint. Having two flags looks more readable to > me. > > Currently existing two-arg iterate methods can now be expressed as a > partial case of the new method: > > public static Stream iterate(final T seed, final UnaryOperator f) { > return iterate(seed, x -> true, f); > } > (same for primitive streams). I may do this if you think it's > reasonable. > > I created new test class and added new iterate sources to existing > data providers. > > Please review and sponsor! > > With best regards, > Tagir Valeev. > > (*) Consider the following code: > > int[] data = {1,2,3,4,-1}; > IntStream.iterate(0, x -> data[x]) > .takeWhile(x -> x >= 0) > .forEach(System.out::println); > > Currently this unexpectedly throws an AIOOBE, because > IntStream.iterate unnecessarily tries to read one element ahead. > From chris.hegarty at oracle.com Tue Feb 16 14:02:03 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 16 Feb 2016 14:02:03 +0000 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> Message-ID: On 15 Feb 2016, at 20:05, Mandy Chung wrote: > >> On Feb 15, 2016, at 9:06 AM, Uwe Schindler wrote: >> >> Hi, >> >>> On 15/02/2016 14:57, Chris Hegarty wrote: >>>> Peter, >>>> >>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/ >>>> > > This patch looks correct to me. The innocuous thread created for common cleaner is of Thread.MAX_PRIORITY - 2 priority whereas the reference handler thread is of MAX_PRIORITY priority. I see your point, but I don?t see this as an issue because threads failing to allocate memory themselves get involved in cleaning. So pressure is somewhat off the reference handler thread. -Chris. From miroslav.kos at oracle.com Tue Feb 16 15:42:06 2016 From: miroslav.kos at oracle.com (Miroslav Kos) Date: Tue, 16 Feb 2016 16:42:06 +0100 Subject: [9] RFR: 8149923 invalid javadoc in javax.xml.bind.JAXBContext (introduced by 8138699) Message-ID: <56C3434E.6080504@oracle.com> Hi Lance, could you review this minor patch? It is just minor overlook I didn't catch while doing 8138699. Filed a new JBS issue: https://bugs.openjdk.java.net/browse/JDK-8149923 webrev: http://cr.openjdk.java.net/~mkos/8149923/jaxws.01/ Thanks Miran From peter.levart at gmail.com Tue Feb 16 16:15:23 2016 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 16 Feb 2016 17:15:23 +0100 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> Message-ID: <56C34B1B.8050001@gmail.com> Hello everybody, Thanks for looking into this and for all your comments. Now that it seems we have a consensus that replacing internal Cleaner with public Cleaner is not a wrong idea, I created an issue for it [1] so that we may officially review the implementation. I also created an updated webrev [2] which fixes some comments in usages that were not correct any more after the change. I also took the liberty to modify the CleanerImpl.InnocuousThreadFactory to give names to threads in accordance to the style used in some other thread factories (Timer-0, Timer-1, ..., Cleaner-0, Cleaner-1, ..., etc.). This gives the thread of internal Cleaner instance, which is now constructed as part of the boot-up sequence, a stable name: "Cleaner-0". If you feel that internal Cleaner instance should have a Thread with MAX_PRIORITY, I can incorporate that too. [1] https://bugs.openjdk.java.net/browse/JDK-8149925 [2] http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.03/ I re-ran the java/lang/ref and java/nio tests and all pass except two ignored tests: java/nio/Buffer/LimitDirectMemory.sh: Test option to limit direct memory allocation java/nio/file/spi/SetDefaultProvider.java: Unit test for java.nio.file.spi.FileSystemProvider and the following two, which seem to not like my network config. or something like that: java/nio/channels/DatagramChannel/MulticastSendReceiveTests.java: Unit test for DatagramChannel's multicast support java/nio/channels/DatagramChannel/Promiscuous.java: Test for interference when two sockets are bound to the same port but joined to different multicast groups Regards, Peter On 02/16/2016 03:02 PM, Chris Hegarty wrote: > On 15 Feb 2016, at 20:05, Mandy Chung wrote: > >>> On Feb 15, 2016, at 9:06 AM, Uwe Schindler wrote: >>> >>> Hi, >>> >>>> On 15/02/2016 14:57, Chris Hegarty wrote: >>>>> Peter, >>>>> >>>>> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.02/ >>>>> >> This patch looks correct to me. The innocuous thread created for common cleaner is of Thread.MAX_PRIORITY - 2 priority whereas the reference handler thread is of MAX_PRIORITY priority. > I see your point, but I don?t see this as an issue because threads failing to > allocate memory themselves get involved in cleaning. So pressure is > somewhat off the reference handler thread. > > -Chris. From openjdk at duigou.org Tue Feb 16 16:58:18 2016 From: openjdk at duigou.org (Mike Duigou) Date: Tue, 16 Feb 2016 08:58:18 -0800 Subject: JDK 9 RFR of JDK-8149896: Remove unnecessary values in FloatConsts and DoubleConsts In-Reply-To: References: Message-ID: > From: joe darcy > Hello, > > The the FloatConsts and DoubleConsts classes, while moved to an > internal > package recently (JDK-8145990), contain constants now available via the > public API. All such uses of the redundant values should be removed as > well as the redundant constants themselves. > > http://cr.openjdk.java.net/~darcy/8149896.0/ The changes look good to me and it is very tempting to suggest just retiring DoubleConstants and FloatConstants altogether and move the remaining quite useful constants in to Double and Float. Mike From joe.darcy at oracle.com Tue Feb 16 16:59:17 2016 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 16 Feb 2016 08:59:17 -0800 Subject: [OpenJDK 2D-Dev] JDK 9 RFR of JDK-8149896: Remove unnecessary values in FloatConsts and DoubleConsts In-Reply-To: References: <56C2C057.4090000@oracle.com> <85D3D7D7-2D23-4A10-947A-05D4872B117E@oracle.com> Message-ID: <56C35565.2050605@oracle.com> Hi Laurent, On 2/16/2016 2:08 AM, Laurent Bourg?s wrote: > Hello, > > Joe, I tested your changes in the Marlin renderer as it works well ! > > > > A quick note on the 2d changes, several constants (and a copy > from a package-private method from java.lang) were used to > initialize a double value POWER_2_TO_32; this can be accomplished > more directly using a hexadecimal floating-point literal. > > > > Please review the webrev > > > > http://cr.openjdk.java.net/~darcy/8149896.0/ > > ... > My favourite change is this one: > > - private static final double POWER_2_TO_32 = > FloatMath.powerOfTwoD(32); > + private static final double POWER_2_TO_32 = 0x1.0p32; > > and the removal of the method on FloatMath. > > > I agree it is simpler and exactly what I needed; I wasn't aware of > such literals: > https://blogs.oracle.com/darcy/entry/hexadecimal_floating_point_literals > > Most people don't have cause to use hexadecimal floating-point literals very often, but when you need them, they are exactly what you need :-) Thanks for the review, -Joe From joel.borggren.franck at gmail.com Tue Feb 16 17:48:04 2016 From: joel.borggren.franck at gmail.com (=?UTF-8?Q?Joel_Borggr=C3=A9n=2DFranck?=) Date: Tue, 16 Feb 2016 17:48:04 +0000 Subject: RFR 7183985: Class.getAnnotation() throws an ArrayStoreException when the annotation class not present In-Reply-To: References: Message-ID: Hi Liam, Sorry for the delay, On Sat, 30 Jan 2016 at 04:45 Liam Miller-Cushon wrote: > On Thu, Jan 28, 2016 at 11:50 AM, Joel Borggr?n-Franck < > joel.borggren.franck at gmail.com> wrote: > > But, the reason we didn't fix this the last two times we looked at it >> (that I am aware of) is the compatibility story. In order to fix his >> you need to figure out two things: >> >> - Is this is a spec change, that is, can we get away with throwing an >> AnnotationFormatError where we would now do? Should we clarify the >> spec? >> > > Can you clarify which parts of the spec might need to be updated? I can't > find anything relevant in the jls or jvms. The javadoc for AnnotatedElement > mentions some conditions under which TypeNotPresentException is thrown: > > "If an annotation returned by a method in this interface contains > (directly or indirectly) a Class-valued member referring to a class that is > not accessible in this VM, attempting to read the class by calling the > relevant Class-returning method on the returned annotation will result in a > TypeNotPresentException." > > So throwing TypeNotPresentException when an array-valued annotation > indirectly references an inaccessible class sounds like the right > behaviour, and is consistent with how the implementation currently handles > similar cases. > I think you answered my concerns. By the spec I meant either the Java Language Specification or the normative parts of the javadoc. This seems to be in line with what is currently specified. > - Since this is a behaviorally incompatible change, how big is the >> impact? This is of course a hard question to answer, but if one could >> do a corpus analysis over a large code base and look for catches of >> ArrayStoreExceptions when reflecting over annotations, that could be >> useful. If it turns out that "a lot" of users have adopted to this >> bug, perhaps it isn't worth fixing? On the other hand I can imagine >> that this is so uncommon that no one catches either type of error. >> > > I'm working on evaluating the impact. A review of our code base showed > that handling ArrayStoreException is fairly uncommon. Of the instances I > found, none of them were in code that was inspecting annotations. > > The next step is to start using the patch internally and see if anything > breaks. I'll update the thread when I have results on that. > Great, if the experiment works out I'll help formulate a change request for compatibility review. cheers /Joel From martinrb at google.com Tue Feb 16 18:15:50 2016 From: martinrb at google.com (Martin Buchholz) Date: Tue, 16 Feb 2016 10:15:50 -0800 Subject: RFR: jsr166 jdk9 integration wave 4 In-Reply-To: <28FC2496-E44B-46A4-941F-1772C52466B4@oracle.com> References: <56BB15F7.1060606@oracle.com> <4C0F0336-BC90-4652-9D84-A58CDC1C8590@oracle.com> <28FC2496-E44B-46A4-941F-1772C52466B4@oracle.com> Message-ID: Finally integrated! Nothing in the jsr166 queue at the moment. (I'm sure we'll fix that ...) On Thu, Feb 11, 2016 at 2:56 AM, Chris Hegarty wrote: > > On 11 Feb 2016, at 01:37, Martin Buchholz wrote: > >> and ... webrev regenerated > > Thank you Martin, this is better. > > For completeness, the specdiff has been updated too: > http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html > > -Chris. > >> On Wed, Feb 10, 2016 at 3:32 PM, Martin Buchholz wrote: >>> The specdiff is very helpful. It was much easier to see that the >>> cross-method links from whenComplete to handle could be improved, so >>> we'll change (modulo reflow) >>> >>> --- src/main/java/util/concurrent/CompletionStage.java 24 Jan 2016 >>> 21:22:16 -0000 1.37 >>> +++ src/main/java/util/concurrent/CompletionStage.java 10 Feb 2016 >>> 23:29:49 -0000 >>> @@ -736,7 +736,7 @@ >>> * {@code null} if none) of this stage as arguments. The returned >>> * stage is completed when the action returns. >>> * >>> - *

    Unlike method {@link #handle}, this method is not designed >>> + *

    Unlike method {@link #handle handle}, this method is not designed >>> * to translate completion outcomes, so the supplied action should >>> * not throw an exception. However, if it does, the following >>> * rules apply: if this stage completed normally but the supplied >>> @@ -762,7 +762,7 @@ >>> * if none) of this stage as arguments. The returned stage is completed >>> * when the action returns. >>> * >>> - *

    Unlike method {@link #handle}, this method is not designed >>> + *

    Unlike method {@link #handleAsync(BiFunction) handleAsync}, >>> this method is not designed >>> * to translate completion outcomes, so the supplied action should >>> * not throw an exception. However, if it does, the following >>> * rules apply: If this stage completed normally but the supplied >>> @@ -788,7 +788,7 @@ >>> * if none) of this stage as arguments. The returned stage is completed >>> * when the action returns. >>> * >>> - *

    Unlike method {@link #handle}, this method is not designed >>> + *

    Unlike method {@link #handleAsync(BiFunction,Executor) >>> handleAsync}, this method is not designed >>> * to translate completion outcomes, so the supplied action should >>> * not throw an exception. However, if it does, the following >>> * rules apply: If this stage completed normally but the supplied >>> >>> On Wed, Feb 10, 2016 at 2:51 PM, Chris Hegarty wrote: >>>> >>>> On 10 Feb 2016, at 15:53, Martin Buchholz wrote: >>>> >>>> Thanks for creating the specdiff, but ... it looks reversed; the green >>>> is the old and the red is the new! >>>> >>>> >>>> D?oh, of course. Updated in-place. >>>> >>>> http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html >>>> >>>> Sorry for our "endless fiddling"; we do have future changes in mind, >>>> but no spec changes. >>>> >>>> >>>> No problem. >>>> >>>> -Chris. >>>> >>>> On Wed, Feb 10, 2016 at 2:50 AM, Chris Hegarty >>>> wrote: >>>> >>>> On 02/02/16 15:23, Martin Buchholz wrote: >>>> >>>> >>>> On Tue, Feb 2, 2016 at 6:37 AM, Chris Hegarty >>>> wrote: >>>> >>>> >>>> >>>> On 1 Feb 2016, at 18:45, Martin Buchholz wrote: >>>> >>>> After much debate on what to do when CompleteableFuture.whenComplete >>>> encounters an exception in both the source and the action, we chose >>>> what was acceptable to the most people - add the action's exception to >>>> the source exception as a suppressed exception. And added usage >>>> guidelines. And gave handle "top billing" over whenComplete. >>>> >>>> >>>> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/ >>>> >>>> >>>> >>>> This all looks fine to me. >>>> >>>> So I assume you only need a small CCC request for CompletionStage, right? >>>> Everything else is implementation. >>>> >>>> >>>> >>>> If you squint you might argue that CompletionStage's contract hasn't >>>> actually changed, >>>> but yeah, go ahead and do a CCC for CompletionStage. Publishing a >>>> specdiff would be nice - method reordering (for "top billing") has >>>> made the diffs harder to review. Thanks. >>>> >>>> >>>> >>>> Here are the specdiffs that will be used for the CCC, unless there are >>>> any last minute changes. >>>> >>>> http://cr.openjdk.java.net/~chegar/jsr166-jdk9-integration-CompletableFuture/CompletionStage.html >>>> >>>> -Chris. >>>> >>>> > From joe.darcy at oracle.com Tue Feb 16 18:35:10 2016 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 16 Feb 2016 10:35:10 -0800 Subject: JDK 9 RFR of JDK-8149981: java/util/Formatter/Basic.java fails after JDK-8149896 Message-ID: <56C36BDE.3060700@oracle.com> Hello, *sigh* The one set of regression tests related in JDK-8149896 I forgot run run before pushing had a dependency on the removed constants in DoubleConsts. Please review this straightforward change to fix up the Formatter test template and regenerate the tests: http://cr.openjdk.java.net/~darcy/8149981.0/ Actual semantic changes in test/java/util/Formatter/Basic-X.java.template test/java/util/Formatter/Basic.java test/java/util/Formatter/BasicDouble.java Thanks, -Joe From lance.andersen at oracle.com Tue Feb 16 18:55:18 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Tue, 16 Feb 2016 13:55:18 -0500 Subject: [9] RFR: 8149923 invalid javadoc in javax.xml.bind.JAXBContext (introduced by 8138699) In-Reply-To: <56C3434E.6080504@oracle.com> References: <56C3434E.6080504@oracle.com> Message-ID: <8BF5625E-D715-479D-8751-03FA577C2F35@oracle.com> Hi Miran Looks OK though you might want to check the spacing on line 454 as it should probably be moved over but no additional review needed Best Lance On Feb 16, 2016, at 10:42 AM, Miroslav Kos wrote: > Hi Lance, > could you review this minor patch? It is just minor overlook I didn't catch while doing 8138699. > > Filed a new JBS issue: https://bugs.openjdk.java.net/browse/JDK-8149923 > webrev: http://cr.openjdk.java.net/~mkos/8149923/jaxws.01/ > > Thanks > Miran > > > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From Roger.Riggs at Oracle.com Tue Feb 16 18:59:00 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 16 Feb 2016 13:59:00 -0500 Subject: JDK 9 RFR of JDK-8149981: java/util/Formatter/Basic.java fails after JDK-8149896 In-Reply-To: <56C36BDE.3060700@oracle.com> References: <56C36BDE.3060700@oracle.com> Message-ID: <56C37174.60406@Oracle.com> +1 Roger On 2/16/2016 1:35 PM, joe darcy wrote: > Hello, > > *sigh* The one set of regression tests related in JDK-8149896 I forgot > run run before pushing had a dependency on the removed constants in > DoubleConsts. Please review this straightforward change to fix up the > Formatter test template and regenerate the tests: > > http://cr.openjdk.java.net/~darcy/8149981.0/ > > Actual semantic changes in > > test/java/util/Formatter/Basic-X.java.template > test/java/util/Formatter/Basic.java > test/java/util/Formatter/BasicDouble.java > > Thanks, > > -Joe From brian.burkhalter at oracle.com Tue Feb 16 19:01:51 2016 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 16 Feb 2016 11:01:51 -0800 Subject: JDK 9 RFR of JDK-8149981: java/util/Formatter/Basic.java fails after JDK-8149896 In-Reply-To: <56C37174.60406@Oracle.com> References: <56C36BDE.3060700@oracle.com> <56C37174.60406@Oracle.com> Message-ID: <73371C4C-FA7E-438E-B953-8F39A0E029A6@oracle.com> +2 On Feb 16, 2016, at 10:59 AM, Roger Riggs wrote: > +1 > > Roger > > On 2/16/2016 1:35 PM, joe darcy wrote: >> Hello, >> >> *sigh* The one set of regression tests related in JDK-8149896 I forgot run run before pushing had a dependency on the removed constants in DoubleConsts. Please review this straightforward change to fix up the Formatter test template and regenerate the tests: >> >> http://cr.openjdk.java.net/~darcy/8149981.0/ >> >> Actual semantic changes in >> >> test/java/util/Formatter/Basic-X.java.template >> test/java/util/Formatter/Basic.java >> test/java/util/Formatter/BasicDouble.java >> >> Thanks, >> >> -Joe > From chris.hegarty at oracle.com Tue Feb 16 19:19:35 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 16 Feb 2016 19:19:35 +0000 Subject: RFR[9] 8068686: Remove meta-index support Message-ID: <08F34760-D34C-441C-80AB-DCB0E744466C@oracle.com> The Modular Run-Time image, integrated in b41, no longer has JAR files, see JEP 220 [1], The meta-index is no longer generated or used. This issue proposes to remove the meta-index support from the runtime and the build. http://cr.openjdk.java.net/~chegar/8068686/ https://bugs.openjdk.java.net/browse/JDK-8068686 -Chris. [1] https://bugs.openjdk.java.net/browse/JDK-8061971 From huizhe.wang at oracle.com Tue Feb 16 20:02:40 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 16 Feb 2016 12:02:40 -0800 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: References: Message-ID: <56C38060.2030703@oracle.com> Hi Christoph, At initialization of XSDHandler (in reset), note there are two local variables for the security / property manager, change them to instance variables, then when you need to create annotation validator, set both on the validator, e.g.: fAnnotationValidator.setProperty(SECURITY_MANAGER, fSecurityManager); fAnnotationValidator.setProperty(XML_SECURITY_PROPERTY_MANAGER, fSecurityPropertyMgr); That should fix the issue. There are a bunch of rawtypes/unchecked warnings in this class, bonus but not required :-) For the test, in the past (jaxp standalone), we've created tests for each issue, nowadays we'd prefer to consolidate them. We don't have "SchemaTest" yet, so you may name the test "SchemaTest", then for the @summary, make it general, e.g. "Test Schema creation", and move the specific comment to the test case and make it specific to the issue: /* * @bug 8149915 * Verifies that the annotation validator is initialized with the security manager for schema * creation with http://apache.org/xml/features/validate-annotations=true. */ @Test public void testValidation() throws Exception { The test is new, the copyright year would be "2016," instead of "2014, 2016,". Best, Joe On 2/16/2016 4:49 AM, Langer, Christoph wrote: > Hi, > > can you please review the following change for JDK-8149915: > > http://cr.openjdk.java.net/~clanger/webrevs/8149915.0/ > > I'm not sure if it is the right thing to set the default for XML_SECURITY_PROPERTY_MANAGER to a new XMLSecurityPropertyManager() object instead of the String "all". But it must not be a String, I think. > > And also I don't know if my approach to add a field "fSecurityManager" to XML11Configuration. > > Please advise and let me know how I can do it better. > > Thanks in advance and best regards > Christoph > From aleksey.shipilev at oracle.com Tue Feb 16 20:16:46 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Tue, 16 Feb 2016 23:16:46 +0300 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class Message-ID: <56C383AE.3000702@oracle.com> Hi, Please review a StringConcatFactory fix that makes generated String concat stubs to be named "$$StringConcat/", instead of "java/lang/String$$Concat/": https://bugs.openjdk.java.net/browse/JDK-8149835 Among other things, this helps to dodge the failure caught by stricter access controls in Jake (JDK-8149165), and hopefully caters for future stricter Unsafe.defineAnonymousClass access controls (JDK-8058575). Webrevs: http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.01/ http://cr.openjdk.java.net/~shade/8149835/webrev.langtools.01/ Testing: JPRT, java/lang/String; + copying SCF code to jake forest and running with it. Cheers, -Aleksey From mandy.chung at oracle.com Tue Feb 16 20:39:33 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Feb 2016 12:39:33 -0800 Subject: RFR[9] 8068686: Remove meta-index support In-Reply-To: <08F34760-D34C-441C-80AB-DCB0E744466C@oracle.com> References: <08F34760-D34C-441C-80AB-DCB0E744466C@oracle.com> Message-ID: <2A663FD8-2169-4331-956B-F224B6D12762@oracle.com> On Feb 16, 2016, at 11:19 AM, Chris Hegarty wrote: > > The Modular Run-Time image, integrated in b41, no longer has JAR files, > see JEP 220 [1], The meta-index is no longer generated or used. This > issue proposes to remove the meta-index support from the runtime and > the build. > > http://cr.openjdk.java.net/~chegar/8068686/ > https://bugs.openjdk.java.net/browse/JDK-8068686 Looks good. I assume at some point the classlist under jdk/make/tools/classlist will be updated. They currently reference several sun.misc types that you have removed or renamed. Mandy From andrej.golovnin at gmail.com Tue Feb 16 20:59:14 2016 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Tue, 16 Feb 2016 21:59:14 +0100 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class In-Reply-To: <56C383AE.3000702@oracle.com> References: <56C383AE.3000702@oracle.com> Message-ID: <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> Hi Aleksey, > http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.01/ 701 return (pkg != null ? pkg.getName().replace(".", "/") + "/" : "") + "Stubs$$StringConcat"; 702 } else { 703 return hostClass.getName().replace(".", "/") + "$$StringConcat?; Maybe you should use here the character based String#replace()-method as it is faster and does not produce as much garbage as the CharSequence based method does. Best regards, Andrej Golovnin From brian.goetz at oracle.com Tue Feb 16 21:20:26 2016 From: brian.goetz at oracle.com (Brian Goetz) Date: Tue, 16 Feb 2016 16:20:26 -0500 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> References: <6728448.20160214205343@gmail.com> <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> Message-ID: <56C3929A.3010802@oracle.com> Regarding wildcards: you definitely want Predicte, which is what we use elsewhere. For UnaryOperator, you're stuck with invariance. On 2/16/2016 8:32 AM, Paul Sandoz wrote: > Hi Tagir, > > The implementation approach looks good. I agree with Brian on the documentation, esp. about the connection to a for loop. > > Testing-wise adding to the spliterator data sources is ok. Because you have done that there is no need to explicitly use SpliteratorTestHelper in IterateTest, as that is reasonably covered by SpliteratorTest. Since you have the data providers for streams just run it through exerciseStream instead. > > Regarding wild cards we could clean this up later and include other methods [1]. It?s easy to go overboard on this, recommend sticking to ? extends/? super where appropriate. The existing iterate methods use UnaryOperator, thus there is no wiggle room there, am i ok with reusing that for the 3-arg method. > > Regarding the existing two-args method, my inclination is to keep the implementations separate from the 3-args, but feel free to change to using abstract spliterator (reduce layering) using a similar code pattern (no need for "finished" or overriding forEachRemaining). Then you can remove Streams.NONE and it?s sneaky erased usage :-) > > Paul. > > [1] https://bugs.openjdk.java.net/browse/JDK-8132097 > >> On 14 Feb 2016, at 15:53, Tagir F. Valeev wrote: >> >> Hello! >> >> I wanted to work on foldLeft, but Brian asked me to take this issue >> instead. So here's webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ >> >> I don't like iterator-based Stream source implementations, so I made >> them AbstractSpliterator-based. I also implemented manually >> forEachRemaining as, I believe, this improves the performance in >> non-short-circuiting cases. >> >> I also decided to keep two flags (started and finished) to track the >> state. Currently existing implementation of infinite iterate() does >> not use started flag, but instead reads one element ahead for >> primitive streams. This seems wrong to me and may even lead to >> unexpected exceptions (*). I could get rid of "started" flag for >> Stream.iterate() using Streams.NONE, but this would make object >> implementation different from primitive implementations. It would also >> be possible to keep single three-state variable (byte or int, >> NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve >> the performance or footprint. Having two flags looks more readable to >> me. >> >> Currently existing two-arg iterate methods can now be expressed as a >> partial case of the new method: >> >> public static Stream iterate(final T seed, final UnaryOperator f) { >> return iterate(seed, x -> true, f); >> } >> (same for primitive streams). I may do this if you think it's >> reasonable. >> >> I created new test class and added new iterate sources to existing >> data providers. >> >> Please review and sponsor! >> >> With best regards, >> Tagir Valeev. >> >> (*) Consider the following code: >> >> int[] data = {1,2,3,4,-1}; >> IntStream.iterate(0, x -> data[x]) >> .takeWhile(x -> x >= 0) >> .forEach(System.out::println); >> >> Currently this unexpectedly throws an AIOOBE, because >> IntStream.iterate unnecessarily tries to read one element ahead. >> From Roger.Riggs at Oracle.com Tue Feb 16 21:37:59 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 16 Feb 2016 16:37:59 -0500 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56C15A52.1050403@oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> <56C15A52.1050403@oracle.com> Message-ID: <56C396B7.8000803@Oracle.com> Hi David, Webrev updated in place: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ On 2/14/2016 11:55 PM, David Holmes wrote: > Hi Roger, > > A few mostly minor comments ... > > On 13/02/2016 3:47 AM, Roger Riggs wrote: >> Please review moving the functionality of sun.misc.Signal to >> jdk.internal.misc and >> creating a wrapper sun.misc.Signal for existing external uses. >> >> +++ This time including the hotspot changes to update the target of the >> upcalls. >> >> A new replacement API will be considered separately. >> >> The update includes a test that passes with or without the changes. >> (Except for an NPE instead of SEGV if null is passed). >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ > > I take it we don't care about reorder files any more? There does not seem to be any particular guidance about how/when to use them on Solaris. So it seemed as reasonable to remove the entries as to update them. > > --- > > src/java.base/share/classes/sun/misc/Signal.java > > The @since should not be changed to 9. You only changed the > implementation not the API. Conversely the renamed class should > arguably be @since 9 and not @since 1.2. But it is wrong to say > sun.misc.Signal is @since 9. The new sun.misc.Signal is a wrapper around the original implementation now in jdk.internal.misc. I had modified the mercurial commands to rename sun.misc.Signal to jdk.internal.misc to preserve the history of the implementation. I will correct sun.misc.Signal to retain the original @since and use @since 9 for the jdk.internal.misc > > The *Handler.of method name reads strangely to me - "for" would be > better but I presume that is not allowed. The xx.of () pattern has been used recently. > > --- > > src/java.base/share/classes/jdk/internal/misc/Signal.java > > Not sure I understand the role of NativeSignalHandler any more - given > it can't actually be used ?? They are used to retain the previous handler and can be used to restore that handler. The ability to invoke the handler directly was removed as a simplification. > > --- > > test/sun/misc/SunMiscSignalTest.java > > Based on our email discussion this test can not be testing that the > handler actually gets invoked - as we know it does not in some cases. > I have doubts that sun.misc.Signal ever intended to support all the > signals you are testing. There were no tests for sun.misc.Signal. I created the tests based on the current implementations. They can be updated to reflect current support for handle, raise, and whether a signal handler is called and of course -Xrs. It should allow detection of unintended changes in behavior. > >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ > > Hotspot changes are fine. Thanks, Roger > > Thanks, > David > ----- > >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8149750 >> >> Thanks, Roger >> >> JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 >> >> From Roger.Riggs at Oracle.com Tue Feb 16 21:38:41 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 16 Feb 2016 16:38:41 -0500 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56C1B4DD.2060903@oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> <56C1B4DD.2060903@oracle.com> Message-ID: <56C396E1.3020305@Oracle.com> Hi Chris, Webrev updated in place: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ On 2/15/2016 6:22 AM, Chris Hegarty wrote: > On 12/02/16 17:47, Roger Riggs wrote: >> Please review moving the functionality of sun.misc.Signal to >> jdk.internal.misc and >> creating a wrapper sun.misc.Signal for existing external uses. >> >> +++ This time including the hotspot changes to update the target of the >> upcalls. >> >> A new replacement API will be considered separately. >> >> The update includes a test that passes with or without the changes. >> (Except for an NPE instead of SEGV if null is passed). >> >> Webrev: >> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ > > Overall looks ok, and satisfies the requirement of JEP 260. > > It took me a while to satisfy myself that it is ok to "ignore" > the passed Signal in the wrapper's 'handle' methods. The assumption > is that this wrapper's signal field and the passes signal are, MUST, > represent the same underlying signal. Maybe an assert to make this > explicit? The jdk.internal.misc.Signal passed to the wrapper's methods needs to be mapped to the corresponding sun.misc.Signal but instead of maintaining a map the s.m.Signal instance is kept in the wrapper of the s.m.SignalHandler. > > Looking at j.i.m.Signal., I can see that you explicitly check > for the 'SIG' prefix and prepend it if not there, but toString() also > prepends it. Will getName also be impacted by this ( it may now have > the name prepended with 'SIG', where it previously was not ). Yes, there was a bug there; reverting to backward compatible behavior. HotSpot recognizes different forms of signal names on Windows and Posix. [1] To be compatible with previous versions, the "SIG" prefix allowed by the HotSpot change (only on Posix) is not supported and throws IllegalArgumentException. Thanks, Roger [1] https://bugs.openjdk.java.net/browse/JDK-8149748 > >> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ > > Looks fine. > > -Chris. From christoph.langer at sap.com Tue Feb 16 22:13:28 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Tue, 16 Feb 2016 22:13:28 +0000 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: <56C38060.2030703@oracle.com> References: <56C38060.2030703@oracle.com> Message-ID: <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> Hi Joe, thanks for your great suggestions. I'll come up with a new webrev tomorrow. However, I'm still wondering if the change for the default value of XML_SECURITY_PROPERTY_MANAGER in XMLDocumentFragmentScannerImpl should be done anyway as I guess an XMLObject is more expected rather than a String from other places in the code that could make use of such a default...? Is that true? Best Christoph -----Original Message----- From: huizhe wang [mailto:huizhe.wang at oracle.com] Sent: Dienstag, 16. Februar 2016 21:03 To: Langer, Christoph Cc: core-libs-dev at openjdk.java.net Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE Hi Christoph, At initialization of XSDHandler (in reset), note there are two local variables for the security / property manager, change them to instance variables, then when you need to create annotation validator, set both on the validator, e.g.: fAnnotationValidator.setProperty(SECURITY_MANAGER, fSecurityManager); fAnnotationValidator.setProperty(XML_SECURITY_PROPERTY_MANAGER, fSecurityPropertyMgr); That should fix the issue. There are a bunch of rawtypes/unchecked warnings in this class, bonus but not required :-) For the test, in the past (jaxp standalone), we've created tests for each issue, nowadays we'd prefer to consolidate them. We don't have "SchemaTest" yet, so you may name the test "SchemaTest", then for the @summary, make it general, e.g. "Test Schema creation", and move the specific comment to the test case and make it specific to the issue: /* * @bug 8149915 * Verifies that the annotation validator is initialized with the security manager for schema * creation with http://apache.org/xml/features/validate-annotations=true. */ @Test public void testValidation() throws Exception { The test is new, the copyright year would be "2016," instead of "2014, 2016,". Best, Joe On 2/16/2016 4:49 AM, Langer, Christoph wrote: > Hi, > > can you please review the following change for JDK-8149915: > > http://cr.openjdk.java.net/~clanger/webrevs/8149915.0/ > > I'm not sure if it is the right thing to set the default for XML_SECURITY_PROPERTY_MANAGER to a new XMLSecurityPropertyManager() object instead of the String "all". But it must not be a String, I think. > > And also I don't know if my approach to add a field "fSecurityManager" to XML11Configuration. > > Please advise and let me know how I can do it better. > > Thanks in advance and best regards > Christoph > From james.graham at oracle.com Tue Feb 16 22:28:27 2016 From: james.graham at oracle.com (Jim Graham) Date: Tue, 16 Feb 2016 14:28:27 -0800 Subject: [OpenJDK 2D-Dev] JDK 9 RFR of JDK-8149896: Remove unnecessary values in FloatConsts and DoubleConsts In-Reply-To: References: <56C2C057.4090000@oracle.com> <85D3D7D7-2D23-4A10-947A-05D4872B117E@oracle.com> Message-ID: <56C3A28B.7070100@oracle.com> Laurent may not technically be a reviewer, but he pretty much owns that code so if he's happy then I'm happy (but I didn't look at any of the other code outside of Marlin if you are keeping count of "full reviewers"...). ...jim On 2/16/2016 2:08 AM, Laurent Bourg?s wrote: > Hello, > > Joe, I tested your changes in the Marlin renderer as it works well ! > > > > A quick note on the 2d changes, several constants (and a copy from a package-private method from java.lang) were used to initialize a double value POWER_2_TO_32; this can be accomplished more directly using a hexadecimal floating-point literal. > > > > Please review the webrev > > > >http://cr.openjdk.java.net/~darcy/8149896.0/ > ... > My favourite change is this one: > > - private static final double POWER_2_TO_32 = > FloatMath.powerOfTwoD(32); > + private static final double POWER_2_TO_32 = 0x1.0p32; > > and the removal of the method on FloatMath. > > > I agree it is simpler and exactly what I needed; I wasn't aware of such > literals: > https://blogs.oracle.com/darcy/entry/hexadecimal_floating_point_literals > > (I am not a reviewer) > > Thanks for the fix, > Laurent From huizhe.wang at oracle.com Tue Feb 16 23:21:59 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 16 Feb 2016 15:21:59 -0800 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> References: <56C38060.2030703@oracle.com> <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> Message-ID: <56C3AF17.6050103@oracle.com> On 2/16/2016 2:13 PM, Langer, Christoph wrote: > Hi Joe, > > thanks for your great suggestions. I'll come up with a new webrev tomorrow. > > However, I'm still wondering if the change for the default value of XML_SECURITY_PROPERTY_MANAGER in XMLDocumentFragmentScannerImpl should be done anyway as I guess an XMLObject is more expected rather than a String from other places in the code that could make use of such a default...? Is that true? You're right, the default value was incorrectly set to the default value of a property manager. "null" would be good in this case since there's no need to create a new instance here. Best, Joe > > Best > Christoph > > -----Original Message----- > From: huizhe wang [mailto:huizhe.wang at oracle.com] > Sent: Dienstag, 16. Februar 2016 21:03 > To: Langer, Christoph > Cc: core-libs-dev at openjdk.java.net > Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE > > Hi Christoph, > > At initialization of XSDHandler (in reset), note there are two local > variables for the security / property manager, change them to instance > variables, then when you need to create annotation validator, set both > on the validator, e.g.: > > fAnnotationValidator.setProperty(SECURITY_MANAGER, > fSecurityManager); > fAnnotationValidator.setProperty(XML_SECURITY_PROPERTY_MANAGER, > fSecurityPropertyMgr); > > That should fix the issue. > > There are a bunch of rawtypes/unchecked warnings in this class, bonus > but not required :-) > > For the test, in the past (jaxp standalone), we've created tests for > each issue, nowadays we'd prefer to consolidate them. We don't have > "SchemaTest" yet, so you may name the test "SchemaTest", then for the > @summary, make it general, e.g. "Test Schema creation", and move the > specific comment to the test case and make it specific to the issue: > > /* > * @bug 8149915 > * Verifies that the annotation validator is initialized with the security manager for schema > * creation with http://apache.org/xml/features/validate-annotations=true. > */ > > @Test > public void testValidation() throws Exception { > > > The test is new, the copyright year would be "2016," instead of "2014, > 2016,". > > Best, > Joe > > On 2/16/2016 4:49 AM, Langer, Christoph wrote: >> Hi, >> >> can you please review the following change for JDK-8149915: >> >> http://cr.openjdk.java.net/~clanger/webrevs/8149915.0/ >> >> I'm not sure if it is the right thing to set the default for XML_SECURITY_PROPERTY_MANAGER to a new XMLSecurityPropertyManager() object instead of the String "all". But it must not be a String, I think. >> >> And also I don't know if my approach to add a field "fSecurityManager" to XML11Configuration. >> >> Please advise and let me know how I can do it better. >> >> Thanks in advance and best regards >> Christoph >> From mandy.chung at oracle.com Tue Feb 16 23:52:20 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Tue, 16 Feb 2016 15:52:20 -0800 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C2B2A1.6020200@oracle.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C2B2A1.6020200@oracle.com> Message-ID: > On Feb 15, 2016, at 9:24 PM, David Holmes wrote: > >> This patch looks correct to me. The innocuous thread created for common cleaner is of Thread.MAX_PRIORITY - 2 priority whereas the reference handler thread is of MAX_PRIORITY priority. I don?t know if this would impact any real-world application or whether worth having a dedicated thread with MAX_PRIORTY (David Holmes may have recommendation to this). > > How did you know I would read this? :) Magic ball :) > > Thread priority has no meaning on Solaris, Linux or BSD/OSX by default. > > Only Windows actually applies thread priorities under normal conditions. The difference between MAX_PRIORITY and MAX_PRIORITY-2 is the former uses THREAD_PRIORITY_HIGHEST and the latter THREAD_PRIORITY_ABOVE_NORMAL. Without any real/reasonable workloads for benchmarking it is all somewhat arbitrary. Arguably the Reference handler thread has more work to do in general so might be better given the higher priority. Thanks that?s useful. I never spent the time to look at the implementation. Mandy From kim.barrett at oracle.com Wed Feb 17 00:17:54 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 16 Feb 2016 19:17:54 -0500 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56B871F3.8070504@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B871F3.8070504@gmail.com> Message-ID: > On Feb 8, 2016, at 5:46 AM, Peter Levart wrote: > If special-casing in ReferenceHandler is removed then an opportunity opens to get rid of ReferenceHandler thread altogether. VM/GC could enqueue Reference(s) to their respective ReferenceQueue(s) directly, so everyone could enjoy the benefits sun.misc.Cleaner has? I don't think that's likely. That would require the VM/GC to acquire all the queue locks, and we certainly wouldn't want to do that in a STW pause, so an intermediate thread is kind of unavoidable. [GC already needs to acquire the pending list lock, for interaction with the reference queue thread.] [Or change to lock-free enqueuing and wakeup of waiters, but I think the current preference would be to avoid pulling even something like that into a STW phase.] From kim.barrett at oracle.com Wed Feb 17 00:20:24 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 16 Feb 2016 19:20:24 -0500 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C34B1B.8050001@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> Message-ID: > On Feb 16, 2016, at 11:15 AM, Peter Levart wrote: > > Hello everybody, > > Thanks for looking into this and for all your comments. Now that it seems we have a consensus that replacing internal Cleaner with public Cleaner is not a wrong idea, I created an issue for it [1] so that we may officially review the implementation. I also created an updated webrev [2] which fixes some comments in usages that were not correct any more after the change. I also took the liberty to modify the CleanerImpl.InnocuousThreadFactory to give names to threads in accordance to the style used in some other thread factories (Timer-0, Timer-1, ..., Cleaner-0, Cleaner-1, ..., etc.). This gives the thread of internal Cleaner instance, which is now constructed as part of the boot-up sequence, a stable name: "Cleaner-0". > > If you feel that internal Cleaner instance should have a Thread with MAX_PRIORITY, I can incorporate that too. > > [1] https://bugs.openjdk.java.net/browse/JDK-8149925 > [2] http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.03/ > > I re-ran the java/lang/ref and java/nio tests and all pass except two ignored tests: I think some of the existing uses of sun.misc.Cleaner were really just a matter of convenience and the previous lack of java.lang.ref.Cleaner. An example of this that I have personal knowledge of the history for is the CallSite cleanup in MethodHandleNatives. It's good to see those converted. As for the others, if nobody wants to defend the special handling by the reference processing thread, I'm certainly happy to see it removed. However, I recall Roger saying there were existing tests that failed when certain uses of sun.misc.Cleaner were replaced with java.lang.ref.Cleaner. I don't remember the details, but maybe Roger does. ------------------------------------------------------------------------------ src/java.base/share/classes/java/lang/ref/Reference.java After removal of the special handling of removing Cleaner objects from the pending list, do we still need to catch OOMEs from the pending list? If not sure, feel free to defer that in another RFE for cleanup. ------------------------------------------------------------------------------ src/java.base/share/classes/jdk/internal/ref/CleanerImpl.java I don't understand why CleanerImpl needs to be changed to public in order to provide access to the new drainQueue. Wouldn't it be better to add Cleaner.drainQueue? Some of the other changes here don't seem related to the problem at hand. Are these proposed miscellaneous cleanups? I'm specifically looking at the CleanerCleanable class. And I'm not sure why the thread's constructor argument is being changed, requiring CleanerImpl to implement Runnable. ------------------------------------------------------------------------------ src/java.base/share/classes/sun/nio/fs/NativeBuffer.java 76 Cleaner.Cleanable cleanable() { 77 return cleanable; 78 } [pre-existing, but if we're changing things anyway...] I'm kind of surprised by an accessor function (both here and in the original code) rather than a cleanup function. Is there a use case for anything other than buffer.cleanable().clean()? Similarly for the DirectBuffer interface. ------------------------------------------------------------------------------ From huizhe.wang at oracle.com Wed Feb 17 01:31:45 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Tue, 16 Feb 2016 17:31:45 -0800 Subject: RFR (JAXP) 8146237: PREFER from Features API taking precedence over catalog file Message-ID: <56C3CD81.40007@oracle.com> Hi, Please review a fix on missing property settings read from catalog files. Values from catalog files shall override any other settings including those through the API, e.g. builder().with(CatalogFeatures.Feature.PREFER, "system"). JBS: https://bugs.openjdk.java.net/browse/JDK-8146237 webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8146237/webrev/ Thanks, Joe From kim.barrett at oracle.com Wed Feb 17 04:34:46 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Tue, 16 Feb 2016 23:34:46 -0500 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> Message-ID: <3C552DE2-0ABE-4B8E-A976-A5F79DA2F703@oracle.com> > On Feb 16, 2016, at 7:20 PM, Kim Barrett wrote: > src/java.base/share/classes/java/lang/ref/Reference.java > > After removal of the special handling of removing Cleaner objects from > the pending list, do we still need to catch OOMEs from the pending > list? If not sure, feel free to defer that in another RFE for cleanup. Never mind. OOME could result from attempting to construct and throw InterruptedException. From amaembo at gmail.com Wed Feb 17 05:48:35 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Wed, 17 Feb 2016 11:48:35 +0600 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <56C123D3.8000608@oracle.com> References: <6728448.20160214205343@gmail.com> <56C123D3.8000608@oracle.com> Message-ID: <494585613.20160217114835@gmail.com> Hello, Stuart! Thank you for your comments. SM> I'd suggest focusing on the API first before worrying about how to track the SM> stream state with booleans, etc. Is the API convenient to use, and how well does SM> it support the use cases we envision for it? As Brian already noted, the most benefit of such signature is the resemblance to the good old for loop. Also it's good, because the lambdas don't need to maintain external mutable state in this case (the state is encapsulated in the current element). Most of your proposed examples, however, need to do it as they don't receive the existing state. Also I see no reason to create a method which is compatible with iterator::hasNext/iterator::next or even spliterator::tryAdvance. If you already have a spliterator, you can create a stream using StreamSupport.stream(spliterator, false). If you have an iterator, you can convert it to spliterator using Spliterators.spliterator[UnknownSize]. Well, this probably looks ugly, but more flexible and signals that some low-level stuff is performed. Supplier> is definitely bad in terms of performance. Creating a new stream is not nearly free. To illustrate this I wrote a simple benchmark which compares .flatMapToInt(OptionalInt::stream) with Java8 way .filter(OptionalInt::isPresent).mapToInt(OptionalInt::getAsInt) Here's source code and full output: http://cr.openjdk.java.net/~tvaleev/jmh/optionalstream/ Benchmark (n) Mode Cnt Score Error Units OptionalTest.testOptionalFilterGet 10 avgt 30 0,171 ? 0,011 us/op OptionalTest.testOptionalFilterGet 1000 avgt 30 6,295 ? 0,046 us/op OptionalTest.testOptionalFilterGet 1000000 avgt 30 12597,706 ? 69,214 us/op OptionalTest.testOptionalStream 10 avgt 30 0,330 ? 0,002 us/op OptionalTest.testOptionalStream 1000 avgt 30 27,552 ? 0,577 us/op OptionalTest.testOptionalStream 1000000 avgt 30 30837,240 ? 812,420 us/op Involving intermediate streams makes the thing at least twice slower. Surely this delay could become negligible in some scenarios, but I think it's inacceptable to enforce users to create new source with a bunch of streams. At least primitive specializations will become meaningless in this case: boxing would eat much less time compared to stream creation. As for elements drawn from the queue, it's much better to use existing takeWhile method: queue.stream().takeWhile(x -> x.equals(sentinel)); True, such approach will not include the sentinel element to the result, and there's no easy way to do it with current API. Probably additional method (takeWhileInclusive?) could be considered to solve such problems. Still, I think, drawing from the queue is not the problem which should be solved with new iterate() method. As for Collatz conjecture, it's quite easy to iterate without trailing one: IntStream.iterate(start, val -> val != 1, val -> val % 2 == 0 ? val / 2 : val * 3 + 1) If having one is desired, then it would be easier just to append one to the stream (even if Collatz conjecture is false we will have an infinite stream, so appended one will never appear): IntStream.concat( IntStream.iterate(start, val -> val != 1, val -> val % 2 == 0 ? val / 2 : val * 3 + 1), IntStream.of(1)) A side note: having IntStream.append(int... numbers) would be really nice: IntStream.iterate(start, val -> val != 1, val -> val % 2 == 0 ? val / 2 : val * 3 + 1).append(1) Another approach would be to introduce a special stop value (for example, -1): IntStream.iterate(start, val -> val != -1, val -> val == 1 ? -1 : val % 2 == 0 ? val / 2 : val * 3 + 1) This stream produces Collatz series, including the trailing one. As for Craps, I never heard about such game. If I understood the rules correctly, it's good to represent the state as separate object and define state transition via its method. Something like this should work: Random r = new Random(); IntSupplier dice = () -> r.nextInt(6)+r.nextInt(6)+2; class State { int roll, point; State(int roll, int point) { this.roll = roll; this.point = point; } State() { this(dice.getAsInt(), 0); } boolean isStopRound() { return roll == 7 || (point == 0 && (roll > 10 || roll < 4)) || (point != 0 && roll == point); } State next() { return isStopRound() ? null : new State(dice.getAsInt(), point == 0 ? roll : point); } } Stream.iterate(new State(), Objects::nonNull, State::next) .mapToInt(state -> state.roll) .forEach(System.out::println); With best regards, Tagir Valeev. SM> In particular, I can imagine a number of cases where it would be very helpful to SM> be able to support an empty stream, or where the computation to produce the SM> first element is the same as the computation to produce subsequent elements. SM> Requiring a value for the first stream element is at odds with that. SM> Here are some ideas for use cases to try out: SM> - a series of dice rolls representing a round of craps [1] SM> - elements drawn from a queue until the queue is empty or until SM> a sentinel is reached SM> - a sequence of numbers that (probably) terminates but whose length SM> isn't necessarily known in advance (e.g. Collatz sequence [2]) SM> [1] https://en.wikipedia.org/wiki/Craps SM> [2] https://en.wikipedia.org/wiki/Collatz_conjecture SM> Note that in some cases the sentinel value that terminates the stream should be SM> part of the stream, and in other cases it's not. SM> I'm sure you can find more uses cases by perusing Stack Overflow. :-) SM> I'm a bit skeptical of the use of "iterate" for producing a finite stream. There SM> are the usual issues with overloading, but there's also potential confusion as SM> some forms of iterate() are infinite and others finite. I'll suggest the name SM> "produce" instead, but there are surely better terms. SM> One thing to think about is where the state of the producer is stored. Is it SM> expected to be in an argument that's passed to each invocation of the functional SM> argument, or is it expected to be captured? I don't think there's an answer in SM> isolation; examining use cases would probably shed some light here. SM> Here are a few API ideas (wildcards elided): SM> -- SM> Stream iterate(T seed, Predicate predicate, UnaryOperator f) SM> The API from your proposal, for comparison purposes. SM> -- SM> Stream produce(Supplier>) SM> Produces elements until empty Optional is returned. This box/unboxes every SM> element, maybe(?) alleviated by Valhalla. SM> -- SM> Stream produce(BooleanSupplier, Supplier) SM> Calls the BooleanSupplier; if true the next stream element is what's returned by SM> calling the Supplier. If BooleanSupplier returns false, end of stream. If you SM> have an iterator already, this enables SM> produce(iterator::hasNext, iterator::next) SM> But if you don't have an iterator already, coming up with the functions to SM> satisfy the iterator-style protocol is sometimes painful. SM> -- SM> Stream produce(Predicate> advancer) SM> This has an odd signature, but the function is like Spliterator.tryAdvance(). It SM> must either call the consumer once and return true, or return false without SM> calling the consumer. SM> -- SM> Stream produce(Consumer> advancer) SM> A variation of the above, without a boolean return. The advancer calls the SM> consumer one or more times to add elements to the stream. End of stream occurs SM> when the advancer doesn't call the consumer. SM> -- SM> Stream produce(Supplier>) SM> A variation of Supplier> where the supplier returns a stream SM> containing zero or more elements. The stream terminates if the supplier returns SM> an empty stream. There "boxing" overhead here, but we don't seem to be bothered SM> by this with flatMap(). SM> -- SM> s'marks SM> On 2/14/16 6:53 AM, Tagir F. Valeev wrote: >> Hello! >> >> I wanted to work on foldLeft, but Brian asked me to take this issue >> instead. So here's webrev: >> http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ >> >> I don't like iterator-based Stream source implementations, so I made >> them AbstractSpliterator-based. I also implemented manually >> forEachRemaining as, I believe, this improves the performance in >> non-short-circuiting cases. >> >> I also decided to keep two flags (started and finished) to track the >> state. Currently existing implementation of infinite iterate() does >> not use started flag, but instead reads one element ahead for >> primitive streams. This seems wrong to me and may even lead to >> unexpected exceptions (*). I could get rid of "started" flag for >> Stream.iterate() using Streams.NONE, but this would make object >> implementation different from primitive implementations. It would also >> be possible to keep single three-state variable (byte or int, >> NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve >> the performance or footprint. Having two flags looks more readable to >> me. >> >> Currently existing two-arg iterate methods can now be expressed as a >> partial case of the new method: >> >> public static Stream iterate(final T seed, final UnaryOperator f) { >> return iterate(seed, x -> true, f); >> } >> (same for primitive streams). I may do this if you think it's >> reasonable. >> >> I created new test class and added new iterate sources to existing >> data providers. >> >> Please review and sponsor! >> >> With best regards, >> Tagir Valeev. >> >> (*) Consider the following code: >> >> int[] data = {1,2,3,4,-1}; >> IntStream.iterate(0, x -> data[x]) >> .takeWhile(x -> x >= 0) >> .forEach(System.out::println); >> >> Currently this unexpectedly throws an AIOOBE, because >> IntStream.iterate unnecessarily tries to read one element ahead. >> From peter.levart at gmail.com Wed Feb 17 07:50:24 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Feb 2016 08:50:24 +0100 Subject: We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B871F3.8070504@gmail.com> Message-ID: <56C42640.1020903@gmail.com> On 02/17/2016 01:17 AM, Kim Barrett wrote: >> On Feb 8, 2016, at 5:46 AM, Peter Levart wrote: >> If special-casing in ReferenceHandler is removed then an opportunity opens to get rid of ReferenceHandler thread altogether. VM/GC could enqueue Reference(s) to their respective ReferenceQueue(s) directly, so everyone could enjoy the benefits sun.misc.Cleaner has? > I don't think that's likely. > > That would require the VM/GC to acquire all the queue locks, and we > certainly wouldn't want to do that in a STW pause, so an intermediate > thread is kind of unavoidable. [GC already needs to acquire the > pending list lock, for interaction with the reference queue thread.] > > [Or change to lock-free enqueuing and wakeup of waiters, but I think > the current preference would be to avoid pulling even something like > that into a STW phase.] > Yeah, probably this would be to much to do in a STW pause. It's better to leave as much processing to a background thread as possible because it can be performed concurrently with application threads. Regards, Peter From david.holmes at oracle.com Wed Feb 17 08:33:48 2016 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Feb 2016 18:33:48 +1000 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56C396B7.8000803@Oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> <56C15A52.1050403@oracle.com> <56C396B7.8000803@Oracle.com> Message-ID: <56C4306C.6060206@oracle.com> Hi Roger, On 17/02/2016 7:37 AM, Roger Riggs wrote: > Hi David, > > Webrev updated in place: > http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ Thanks for those tweaks. I'm still perplexed by the test. We established that for USR2, PIPE and XFSZ the handler is not actually invoked - yet you are testing that case. ?? David > On 2/14/2016 11:55 PM, David Holmes wrote: >> Hi Roger, >> >> A few mostly minor comments ... >> >> On 13/02/2016 3:47 AM, Roger Riggs wrote: >>> Please review moving the functionality of sun.misc.Signal to >>> jdk.internal.misc and >>> creating a wrapper sun.misc.Signal for existing external uses. >>> >>> +++ This time including the hotspot changes to update the target of the >>> upcalls. >>> >>> A new replacement API will be considered separately. >>> >>> The update includes a test that passes with or without the changes. >>> (Except for an NPE instead of SEGV if null is passed). >>> >>> Webrev: >>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ >> >> I take it we don't care about reorder files any more? > There does not seem to be any particular guidance about how/when to use > them on Solaris. > So it seemed as reasonable to remove the entries as to update them. >> >> --- >> >> src/java.base/share/classes/sun/misc/Signal.java >> >> The @since should not be changed to 9. You only changed the >> implementation not the API. Conversely the renamed class should >> arguably be @since 9 and not @since 1.2. But it is wrong to say >> sun.misc.Signal is @since 9. > The new sun.misc.Signal is a wrapper around the original implementation > now in jdk.internal.misc. > I had modified the mercurial commands to rename sun.misc.Signal to > jdk.internal.misc to preserve the history > of the implementation. > > I will correct sun.misc.Signal to retain the original @since and use > @since 9 for the jdk.internal.misc >> >> The *Handler.of method name reads strangely to me - "for" would be >> better but I presume that is not allowed. > The xx.of () pattern has been used recently. >> >> --- >> >> src/java.base/share/classes/jdk/internal/misc/Signal.java >> >> Not sure I understand the role of NativeSignalHandler any more - given >> it can't actually be used ?? > They are used to retain the previous handler and can be used to restore > that handler. > The ability to invoke the handler directly was removed as a simplification. >> >> --- >> >> test/sun/misc/SunMiscSignalTest.java >> >> Based on our email discussion this test can not be testing that the >> handler actually gets invoked - as we know it does not in some cases. >> I have doubts that sun.misc.Signal ever intended to support all the >> signals you are testing. > There were no tests for sun.misc.Signal. I created the tests based on > the current implementations. > They can be updated to reflect current support for handle, raise, and > whether a signal handler is called > and of course -Xrs. It should allow detection of unintended changes in > behavior. > >> >>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ >> >> Hotspot changes are fine. > > Thanks, Roger >> >> Thanks, >> David >> ----- >> >>> Issue: >>> https://bugs.openjdk.java.net/browse/JDK-8149750 >>> >>> Thanks, Roger >>> >>> JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 >>> >>> > From peter.levart at gmail.com Wed Feb 17 09:06:31 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Feb 2016 10:06:31 +0100 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> Message-ID: <56C43817.7060805@gmail.com> Hi Kim, Thanks for looking into this. Answers inline... On 02/17/2016 01:20 AM, Kim Barrett wrote: >> On Feb 16, 2016, at 11:15 AM, Peter Levart wrote: >> >> Hello everybody, >> >> Thanks for looking into this and for all your comments. Now that it seems we have a consensus that replacing internal Cleaner with public Cleaner is not a wrong idea, I created an issue for it [1] so that we may officially review the implementation. I also created an updated webrev [2] which fixes some comments in usages that were not correct any more after the change. I also took the liberty to modify the CleanerImpl.InnocuousThreadFactory to give names to threads in accordance to the style used in some other thread factories (Timer-0, Timer-1, ..., Cleaner-0, Cleaner-1, ..., etc.). This gives the thread of internal Cleaner instance, which is now constructed as part of the boot-up sequence, a stable name: "Cleaner-0". >> >> If you feel that internal Cleaner instance should have a Thread with MAX_PRIORITY, I can incorporate that too. >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8149925 >> [2] http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.03/ >> >> I re-ran the java/lang/ref and java/nio tests and all pass except two ignored tests: > I think some of the existing uses of sun.misc.Cleaner were really just > a matter of convenience and the previous lack of > java.lang.ref.Cleaner. An example of this that I have personal > knowledge of the history for is the CallSite cleanup in > MethodHandleNatives. It's good to see those converted. > > As for the others, if nobody wants to defend the special handling by > the reference processing thread, I'm certainly happy to see it > removed. However, I recall Roger saying there were existing tests that > failed when certain uses of sun.misc.Cleaner were replaced with > java.lang.ref.Cleaner. I don't remember the details, but maybe Roger > does. If the failing test was the following: java/nio/Buffer/DirectBufferAllocTest.java ...then it has been taken care of (see Bits.java). All java/lang/ref and java/nio tests pass on my PC. Are there any internal Oracle tests that fail? > > ------------------------------------------------------------------------------ > src/java.base/share/classes/java/lang/ref/Reference.java > > After removal of the special handling of removing Cleaner objects from > the pending list, do we still need to catch OOMEs from the pending > list? If not sure, feel free to defer that in another RFE for cleanup. As you have already noticed it is still possible for a lock.wait() to throw OOME because of not being able to allocate InterruptedException. The other place where OOME could still be thrown (and has not been fixed yet) is from sun.misc.Cleaner.clean() special handling. I even constructed a reproducer for it (see the last comment in JDK-8066859). So removing special handling of sun.misc.Cleaner would finally put the JDK-8066859 to the rest. > > ------------------------------------------------------------------------------ > src/java.base/share/classes/jdk/internal/ref/CleanerImpl.java > > I don't understand why CleanerImpl needs to be changed to public in > order to provide access to the new drainQueue. Wouldn't it be better > to add Cleaner.drainQueue? An interesting idea. But I don't know if such functionality is generally useful enough to commit to it in a public API. I have another idea where java.lang.ref.Cleaner would use an Executor instead of ThreadFactory. The default would be an instance of a single-threaded executor per Cleaner instance, but if user passed in a ForkJoinPool, he could use it's method ForkJoinPool.awaitQuiescence() to help the executor's thread(s) do the cleaning. > > Some of the other changes here don't seem related to the problem at > hand. Are these proposed miscellaneous cleanups? I'm specifically > looking at the CleanerCleanable class. And I'm not sure why the > thread's constructor argument is being changed, requiring CleanerImpl > to implement Runnable. One thing that this change unfortunately requires is to get rid of lambdas and method references in the implementation and dependencies of java.land.ref.Cleaner API, because it gets used early in the boot-up sequence when java.lang.invoke infrastructure is not ready yet. The alternative to CleanerCleanable is a no-op Runnable implementation passed to PhantomCleanableRef constructor. I opted for a subclass of abstract PhantomCleanable class with an empty performCleanup() method. Both approaches require a new class, but the CleanerCleanable is a more direct way to express the intent. Making CleanerImpl implement Runnable and consequently making its run() method public is also just a way to avoid introducing another anonymous inner class as an adapter between Runnable.run() and CleanerImpl.run(). CleanerImpl is in an internal concealed package, so there's no danger of abuse. But I can revert it back to using an anonymous inner adapter class (as was done in webrev.02) if desired or I can just add a comment to public CleanerImpl.run() to warn against it's direct use. > > ------------------------------------------------------------------------------ > src/java.base/share/classes/sun/nio/fs/NativeBuffer.java > 76 Cleaner.Cleanable cleanable() { > 77 return cleanable; > 78 } > > [pre-existing, but if we're changing things anyway...] > > I'm kind of surprised by an accessor function (both here and in the > original code) rather than a cleanup function. Is there a use case > for anything other than buffer.cleanable().clean()? It can't be, since both old Cleaner and new Cleanable have only got a single method - clean(). > > Similarly for the DirectBuffer interface. This one is a critical method, used by various 3rd party softwares. In order to make it less painful to support the change from sun.misc.Cleaner to java.lang.ref.Cleaner.Cleanable, I think the method should retain its name and semantics - it only changes it's return type from a public type to another public type that both contain a method with same signature - clean(). This way one can construct a simple reflective adapter that is compatible with both JDK 8- and JDK 9+. Regards, Peter From aleksey.shipilev at oracle.com Wed Feb 17 09:36:42 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 17 Feb 2016 12:36:42 +0300 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class In-Reply-To: <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> References: <56C383AE.3000702@oracle.com> <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> Message-ID: <56C43F2A.4030205@oracle.com> On 02/16/2016 11:59 PM, Andrej Golovnin wrote: > Hi Aleksey, > >> http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.01/ > > 701 return (pkg != null ? pkg.getName().replace(".", "/") + "/" : "") + "Stubs$$StringConcat"; > 702 } else { > 703 return hostClass.getName().replace(".", "/") + "$$StringConcat?; > > Maybe you should use here the character based String#replace()-method as it is faster and > does not produce as much garbage as the CharSequence based method does. Yes: http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.02/ http://cr.openjdk.java.net/~shade/8149835/webrev.langtools.01/ I have also changed to Lookup.IMPL_LOOKUP when looking up the method from U.defineAnonymousClass-loaded bytecode stub. This make the load sequence consistent with LambdaMetafactory. It still passes JPRT, java/lang/String and jake build. Cheers, -Aleksey From paul.sandoz at oracle.com Wed Feb 17 10:48:21 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 17 Feb 2016 11:48:21 +0100 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class In-Reply-To: <56C43F2A.4030205@oracle.com> References: <56C383AE.3000702@oracle.com> <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> <56C43F2A.4030205@oracle.com> Message-ID: <742A6645-6DF2-46A4-87C7-C36E0C94A870@oracle.com> > On 17 Feb 2016, at 10:36, Aleksey Shipilev wrote: > > On 02/16/2016 11:59 PM, Andrej Golovnin wrote: >> Hi Aleksey, >> >>> http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.01/ >> >> 701 return (pkg != null ? pkg.getName().replace(".", "/") + "/" : "") + "Stubs$$StringConcat"; >> 702 } else { >> 703 return hostClass.getName().replace(".", "/") + "$$StringConcat?; >> >> Maybe you should use here the character based String#replace()-method as it is faster and >> does not produce as much garbage as the CharSequence based method does. > > Yes: > http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.02/ > http://cr.openjdk.java.net/~shade/8149835/webrev.langtools.01/ > > I have also changed to Lookup.IMPL_LOOKUP when looking up the method > from U.defineAnonymousClass-loaded bytecode stub. This make the load > sequence consistent with LambdaMetafactory. > > It still passes JPRT, java/lang/String and jake build. > +1 Paul. From Alan.Bateman at oracle.com Wed Feb 17 11:30:32 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Feb 2016 11:30:32 +0000 Subject: RFR[9] 8068686: Remove meta-index support In-Reply-To: <2A663FD8-2169-4331-956B-F224B6D12762@oracle.com> References: <08F34760-D34C-441C-80AB-DCB0E744466C@oracle.com> <2A663FD8-2169-4331-956B-F224B6D12762@oracle.com> Message-ID: <56C459D8.7010607@oracle.com> On 16/02/2016 20:39, Mandy Chung wrote: > : > > I assume at some point the classlist under jdk/make/tools/classlist will be updated. They currently reference several sun.misc types that you have removed or renamed. > The classlist is out of date but I think Claes is looking at better ways to do that. In any case, the patch to remove this looks good to me too. -Alan From Alan.Bateman at oracle.com Wed Feb 17 11:53:30 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Feb 2016 11:53:30 +0000 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C34B1B.8050001@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> Message-ID: <56C45F3A.1070303@oracle.com> On 16/02/2016 16:15, Peter Levart wrote: > : > > and the following two, which seem to not like my network config. or > something like that: > > java/nio/channels/DatagramChannel/MulticastSendReceiveTests.java: Unit > test for DatagramChannel's multicast support > java/nio/channels/DatagramChannel/Promiscuous.java: Test for > interference when two sockets are bound to the same port but joined to > different multicast groups These two can be problematic when there is a firewall or VPN connection. I can't think how they could fail due to the changes that we are discussing here. -Alan From amaembo at gmail.com Wed Feb 17 11:58:32 2016 From: amaembo at gmail.com (Tagir F. Valeev) Date: Wed, 17 Feb 2016 17:58:32 +0600 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <56C174CA.9050800@gmail.com> References: <6728448.20160214205343@gmail.com> <56C174CA.9050800@gmail.com> Message-ID: <1909919814.20160217175832@gmail.com> Hello! Thank you for your comments. The thing which is absolutely necessary is to make PECS Predicate signature (for some reason I forgot it): static Stream iterate(T seed, Predicate predicate, UnaryOperator f) { ... } While you cannot readily make Stream from it, it's not a big problem if the stream has more specific type. For example, you still can collect to the List. This code works perfectly: List list = iterate(0, i -> i < 10, i -> i + 1).collect(Collectors.toList()); Predicate csNotEmpty = cs -> cs.length() > 0; List result = iterate("abcde", csNotEmpty, (String s) -> s.substring(1)) .collect(Collectors.toList()); Any functions appearing in the intermediate/terminal operations should also be fine with more concrete type (even if they are specified not as lambdas, but as actual interfaces like Function). The only problem which iterate3 would solve is when you actually need the stream of particular type (for example, you're implementing the interface method which returns exactly Stream). I think it's quite rare case and you can always add map step to solve this (if you don't like unchecked cast): Stream sn4 = iterate(0, i -> i < 10, i -> i + 1).map(Number.class::cast); Probably it's still reasonable to introduce second type variable, but for conformance with existing signatures of Stream.generate, Stream.iterate I would prefer not to stay with only. Or probably all these methods should be updated at once (this could be considered as separate issue). With best regards, Tagir Valeev. PL> On 02/14/2016 07:15 PM, Stefan Zobel wrote: PL> PL> PL> Hi Tagir, PL> this looks good. I wonder, however, if the signature should PL> accept a wider range of types, i.e., something like PL> static Stream iterate(S seed, Predicate PL> predicate, UnaryOperator f) PL> I know that the corresponding PL> static Stream iterate(T seed, UnaryOperator f) PL> is less general than that, but I don't know whether this is PL> the outcome of a thoughtful decision or just an oversight. PL> What do you think? PL> PL> PL> What about even more permissive: PL> PL> static Stream iterate3(S seed, PL> Predicate predicate, Function f) PL> PL> ... PL> PL> public class SignatureTest { PL> PL> ??? static Stream iterate1(T seed, Predicate predicate, UnaryOperator f) { PL> ??????? ... PL> ??? } PL> PL> ??? static Stream iterate2(S seed, PL> Predicate predicate, UnaryOperator f) { PL> ??????? ... PL> ??? } PL> PL> ??? static Stream iterate3(S seed, PL> Predicate predicate, Function f) { PL> ??????? ... PL> ??? } PL> PL> PL> ??? public static void main(String[] args) { PL> PL> ??????? Stream si1 = iterate1(0, i -> i < 10, i -> i + 1); // OK PL> ??????? Stream si2 = iterate2(0, i -> i < 10, i -> i + 1); // OK PL> ??????? Stream si3 = iterate3(0, i -> i < 10, i -> i + 1); // OK PL> PL> ??????? Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); PL> PL> ??????? //SignatureTest.java:32: error: bad operand types for binary operator '<' PL> ??????? //??????? Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); PL> ??????? //??????????????????????????????????????????????? ^ PL> ??????? //SignatureTest.java:32: error: bad operand types for binary operator '+' PL> ??????? //??????? Stream sn1 = iterate1(0, i -> i < 10, i -> i + 1); PL> ??????? //?????????????????????????????????????????????? PL> PL> PL> ??????? Stream sn2 = iterate2(0, i -> i < 10, i -> i + 1); // OK PL> ??????? Stream sn3 = iterate3(0, i -> i < 10, i -> i + 1); // OK PL> PL> PL> ??????? Predicate csNotEmpty = cs -> cs.length() > 0; PL> PL> ??????? Stream css1 = iterate1("abcde", PL> csNotEmpty, (String s) -> s.substring(1)); PL> PL> ??????? //SignatureTest.java:44: error: method iterate1 in PL> class SignatureTest cannot be applied to given types; PL> ??????? //??????? Stream css1 = PL> iterate1("abcde", csNotEmpty, (String s) -> s.substring(1)); PL> ??????? //??????????????????????????????????? ^ PL> ??????? //? required: T,Predicate,UnaryOperator PL> ??????? //? found: String,Predicate,(String s)[...]ng(1) PL> ??????? //? reason: inference variable T has incompatible PL> equality constraints String,CharSequence PL> ??????? //? where T is a type-variable: PL> ??????? //??? T extends Object declared in method PL> iterate1(T,Predicate,UnaryOperator) PL> PL> PL> ??????? Stream css2 = iterate2("abcde", PL> csNotEmpty, (String s) -> s.substring(1)); PL> PL> ??????? //SignatureTest.java:54: error: method iterate2 in PL> class SignatureTest cannot be applied to given types; PL> ??????? //??????? Stream css2 = PL> iterate2("abcde", csNotEmpty, (String s) -> s.substring(1)); PL> ??????? //??????????????????????????????????? ^ PL> ??????? //? required: S,Predicate,UnaryOperator PL> ??????? //? found: String,Predicate,(String s)[...]ng(1) PL> ??????? //? reason: inference variable S has incompatible PL> equality constraints String,CharSequence PL> ??????? //? where S,T are type-variables: PL> ??????? //??? S extends T declared in method PL> iterate2(S,Predicate,UnaryOperator) PL> ??????? //??? T extends Object declared in method PL> iterate2(S,Predicate,UnaryOperator) PL> PL> PL> ??????? Stream css3 = iterate3("abcde", PL> csNotEmpty, (String s) -> s.substring(1)); // OK PL> ??? } PL> } PL> PL> PL> Regards, Peter PL> PL> PL> PL> Regards, PL> Stefan PL> 2016-02-14 15:53 GMT+01:00 Tagir F. Valeev : PL> PL> PL> Hello! PL> I wanted to work on foldLeft, but Brian asked me to take this issue PL> instead. So here's webrev: PL> http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ PL> I don't like iterator-based Stream source implementations, so I made PL> them AbstractSpliterator-based. I also implemented manually PL> forEachRemaining as, I believe, this improves the performance in PL> non-short-circuiting cases. PL> I also decided to keep two flags (started and finished) to track the PL> state. Currently existing implementation of infinite iterate() does PL> not use started flag, but instead reads one element ahead for PL> primitive streams. This seems wrong to me and may even lead to PL> unexpected exceptions (*). I could get rid of "started" flag for PL> Stream.iterate() using Streams.NONE, but this would make object PL> implementation different from primitive implementations. It would also PL> be possible to keep single three-state variable (byte or int, PL> NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve PL> the performance or footprint. Having two flags looks more readable to PL> me. PL> Currently existing two-arg iterate methods can now be expressed as a PL> partial case of the new method: PL> public static Stream iterate(final T seed, final UnaryOperator f) { PL> return iterate(seed, x -> true, f); PL> } PL> (same for primitive streams). I may do this if you think it's PL> reasonable. PL> I created new test class and added new iterate sources to existing PL> data providers. PL> Please review and sponsor! PL> With best regards, PL> Tagir Valeev. PL> (*) Consider the following code: PL> int[] data = {1,2,3,4,-1}; PL> IntStream.iterate(0, x -> data[x]) PL> .takeWhile(x -> x >= 0) PL> .forEach(System.out::println); PL> Currently this unexpectedly throws an AIOOBE, because PL> IntStream.iterate unnecessarily tries to read one element ahead. PL> PL> PL> PL> From amy.lu at oracle.com Wed Feb 17 13:09:32 2016 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 17 Feb 2016 21:09:32 +0800 Subject: JDK 9 RFR of JDK-8149920: Remove intermittent key from jdk_core tests Message-ID: <56C4710C.3010609@oracle.com> sun/nio/cs/FindEncoderBugs.java sun/nio/cs/FindDecoderBugs.java java/util/concurrent/Phaser/Basic.java java/net/Inet6Address/serialize/Inet6AddressSerializationTest.java java/net/SocketPermission/SocketPermissionTest.java java/lang/ProcessHandle/InfoTest.java com/sun/jndi/ldap/LdapTimeoutTest.java For above tests, previous intermittent test failure issues are fixed, no open bug (no failure reported). This patch is to remove @key intermittent from tests. bug: https://bugs.openjdk.java.net/browse/JDK-8149920 webrev: http://cr.openjdk.java.net/~amlu/8149920/webrev.00/ Thanks, Amy From amy.lu at oracle.com Wed Feb 17 13:11:54 2016 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 17 Feb 2016 21:11:54 +0800 Subject: JDK 9 RFR of JDK-8149922: Remove intermittent key from security tests Message-ID: <56C4719A.9010006@oracle.com> sun/security/mscapi/ShortRSAKey1024.sh sun/security/mscapi/SignUsingSHA2withRSA.sh javax/net/ssl/SSLSession/SessionCacheSizeTests.java For above tests, previous intermittent test failure issues are fixed, no open bug (no failure reported). This patch is to remove @key intermittent from tests. This patch also removed SessionCacheSizeTests.java from ProblemList.txt as related bug (JDK-8146387) is resolved. bug: https://bugs.openjdk.java.net/browse/JDK-8149922 https://bugs.openjdk.java.net/browse/JDK-8150047 webrev: http://cr.openjdk.java.net/~amlu/8149922/webrev.00/ Thanks, Amy From aleksey.shipilev at oracle.com Wed Feb 17 13:41:37 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 17 Feb 2016 16:41:37 +0300 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class In-Reply-To: <742A6645-6DF2-46A4-87C7-C36E0C94A870@oracle.com> References: <56C383AE.3000702@oracle.com> <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> <56C43F2A.4030205@oracle.com> <742A6645-6DF2-46A4-87C7-C36E0C94A870@oracle.com> Message-ID: <56C47891.40506@oracle.com> On 02/17/2016 01:48 PM, Paul Sandoz wrote: >> On 17 Feb 2016, at 10:36, Aleksey Shipilev wrote: >> Yes: >> http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.02/ >> http://cr.openjdk.java.net/~shade/8149835/webrev.langtools.01/ >> >> I have also changed to Lookup.IMPL_LOOKUP when looking up the method >> from U.defineAnonymousClass-loaded bytecode stub. This make the load >> sequence consistent with LambdaMetafactory. >> >> It still passes JPRT, java/lang/String and jake build. >> > > +1 Thanks Paul! Alan, Mandy, would you like to review this as well? This should fix Jake compatibility problems. Cheers, -Aleksey From shilpi.rastogi at oracle.com Wed Feb 17 14:47:13 2016 From: shilpi.rastogi at oracle.com (shilpi rastogi) Date: Wed, 17 Feb 2016 20:17:13 +0530 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes Message-ID: <56C487F1.3030609@oracle.com> Hi All, Please review fix for the following bug- https://bugs.openjdk.java.net/browse/JDK-8144931 http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ Thanks, Shilpi From cheleswer.sahu at oracle.com Wed Feb 17 15:17:06 2016 From: cheleswer.sahu at oracle.com (cheleswer sahu) Date: Wed, 17 Feb 2016 20:47:06 +0530 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <569E27A2.4070207@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> Message-ID: <56C48EF2.3030403@oracle.com> Hi, I have made changes in the property name (jdk.lang.processReaperUseDefaultStackSize) and code as suggested in the earlier reviews. --- old/src/java.base/share/classes/java/lang/ProcessHandleImpl.java 2016-02-17 18:48:10.545639999 +0530 +++ new/src/java.base/share/classes/java/lang/ProcessHandleImpl.java 2016-02-17 18:48:10.081639999 +0530 @@ -81,9 +81,8 @@ ThreadGroup systemThreadGroup = tg; ThreadFactory threadFactory = grimReaper -> { - // Our thread stack requirement is quite modest. - Thread t = new Thread(systemThreadGroup, grimReaper, - "process reaper", 32768); + long stackSize = Boolean.getBoolean("jdk.lang.processReaperUseDefaultStackSize") ? 0 : 32768; + Thread t = new Thread(systemThreadGroup, grimReaper, "process reaper", stackSize); t.setDaemon(true); // A small attempt (probably futile) to avoid priority inversion t.setPriority(Thread.MAX_PRIORITY); I would really like to thanks "Martin" for the patch (http://cr.openjdk.java.net/~martin/webrevs/openjdk9/tls-size-guarantee/ ) which he has provided. Since we support a wider range of glibc versions and platform using patch will require more testing and work. I think the use of system property for this issue will be safer at this point of time. Regards, Cheleswer On 1/19/2016 5:40 PM, David Holmes wrote: > On 19/01/2016 9:53 PM, Kevin Walls wrote: >> | >> Hi Cheleswer, I think Martin is suggesting something like: >> | >> >> // Use a modest stack size, unless requested otherwise: >> long stackSize = >> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; > > Someone from core-libs needs to chime on what the appropriate > namespace for such a property would be. > > David > ----- > >> Thread t = new Thread(systemThreadGroup, grimReaper, "process >> reaper", stackSize); >> >> ||| >> If that tests OK for you, it may be the way we can go ahead with the >> point fix for this. >> >> Regards >> Kevin >> | >> On 18/01/2016 16:52, Martin Buchholz wrote: >>> Historical note - I never liked having a reaper thread for each >>> subprocess, but no other reliable implementation is known. Given the >>> potential for many reaper threads, I at least wanted to keep the >>> memory waste low. >>> >>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>> wrote: >>> >>>> + Thread t = null; >>>> + if >>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>> + t = new Thread(systemThreadGroup, grimReaper, >>>> "process reaper"); >>>> + } else { >>>> + // Our thread stack requirement is quite >>>> modest. >>>> + t = new Thread(systemThreadGroup, grimReaper, >>>> "process reaper", 32768); >>>> + } >>> If we do end up using this strategy, cleaner would be to use >>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>> >>> stackSize - the desired stack size for the new thread, or zero to >>> indicate that this parameter is to be ignored. >> -------------- next part -------------- --- old/src/java.base/share/classes/java/lang/ProcessHandleImpl.java 2016-02-17 18:48:10.545639999 +0530 +++ new/src/java.base/share/classes/java/lang/ProcessHandleImpl.java 2016-02-17 18:48:10.081639999 +0530 @@ -81,9 +81,8 @@ ThreadGroup systemThreadGroup = tg; ThreadFactory threadFactory = grimReaper -> { - // Our thread stack requirement is quite modest. - Thread t = new Thread(systemThreadGroup, grimReaper, - "process reaper", 32768); + long stackSize = Boolean.getBoolean("jdk.lang.processReaperUseDefaultStackSize") ? 0 : 32768; + Thread t = new Thread(systemThreadGroup, grimReaper, "process reaper", stackSize); t.setDaemon(true); // A small attempt (probably futile) to avoid priority inversion t.setPriority(Thread.MAX_PRIORITY); From Roger.Riggs at Oracle.com Wed Feb 17 15:49:05 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 17 Feb 2016 10:49:05 -0500 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56C4306C.6060206@oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> <56C15A52.1050403@oracle.com> <56C396B7.8000803@Oracle.com> <56C4306C.6060206@oracle.com> Message-ID: <56C49671.3060107@Oracle.com> Hi David, On 2/17/2016 3:33 AM, David Holmes wrote: > Hi Roger, > > On 17/02/2016 7:37 AM, Roger Riggs wrote: >> Hi David, >> >> Webrev updated in place: >> http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ > > Thanks for those tweaks. > > I'm still perplexed by the test. We established that for USR2, PIPE > and XFSZ the handler is not actually invoked - yet you are testing > that case. ?? I'll have to dig a bit more. When that test is run, the SignalHandler does get called with the expected signal. The test succeeds on Linux with jdk 8 and jdk 9. Roger > > David > >> On 2/14/2016 11:55 PM, David Holmes wrote: >>> Hi Roger, >>> >>> A few mostly minor comments ... >>> >>> On 13/02/2016 3:47 AM, Roger Riggs wrote: >>>> Please review moving the functionality of sun.misc.Signal to >>>> jdk.internal.misc and >>>> creating a wrapper sun.misc.Signal for existing external uses. >>>> >>>> +++ This time including the hotspot changes to update the target of >>>> the >>>> upcalls. >>>> >>>> A new replacement API will be considered separately. >>>> >>>> The update includes a test that passes with or without the changes. >>>> (Except for an NPE instead of SEGV if null is passed). >>>> >>>> Webrev: >>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ >>> >>> I take it we don't care about reorder files any more? >> There does not seem to be any particular guidance about how/when to use >> them on Solaris. >> So it seemed as reasonable to remove the entries as to update them. >>> >>> --- >>> >>> src/java.base/share/classes/sun/misc/Signal.java >>> >>> The @since should not be changed to 9. You only changed the >>> implementation not the API. Conversely the renamed class should >>> arguably be @since 9 and not @since 1.2. But it is wrong to say >>> sun.misc.Signal is @since 9. >> The new sun.misc.Signal is a wrapper around the original implementation >> now in jdk.internal.misc. >> I had modified the mercurial commands to rename sun.misc.Signal to >> jdk.internal.misc to preserve the history >> of the implementation. >> >> I will correct sun.misc.Signal to retain the original @since and use >> @since 9 for the jdk.internal.misc >>> >>> The *Handler.of method name reads strangely to me - "for" would be >>> better but I presume that is not allowed. >> The xx.of () pattern has been used recently. >>> >>> --- >>> >>> src/java.base/share/classes/jdk/internal/misc/Signal.java >>> >>> Not sure I understand the role of NativeSignalHandler any more - given >>> it can't actually be used ?? >> They are used to retain the previous handler and can be used to restore >> that handler. >> The ability to invoke the handler directly was removed as a >> simplification. >>> >>> --- >>> >>> test/sun/misc/SunMiscSignalTest.java >>> >>> Based on our email discussion this test can not be testing that the >>> handler actually gets invoked - as we know it does not in some cases. >>> I have doubts that sun.misc.Signal ever intended to support all the >>> signals you are testing. >> There were no tests for sun.misc.Signal. I created the tests based on >> the current implementations. >> They can be updated to reflect current support for handle, raise, and >> whether a signal handler is called >> and of course -Xrs. It should allow detection of unintended changes in >> behavior. >> >>> >>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ >>> >>> Hotspot changes are fine. >> >> Thanks, Roger >>> >>> Thanks, >>> David >>> ----- >>> >>>> Issue: >>>> https://bugs.openjdk.java.net/browse/JDK-8149750 >>>> >>>> Thanks, Roger >>>> >>>> JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 >>>> >>>> >> From mandy.chung at oracle.com Wed Feb 17 16:00:01 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Feb 2016 08:00:01 -0800 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class In-Reply-To: <56C47891.40506@oracle.com> References: <56C383AE.3000702@oracle.com> <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> <56C43F2A.4030205@oracle.com> <742A6645-6DF2-46A4-87C7-C36E0C94A870@oracle.com> <56C47891.40506@oracle.com> Message-ID: <94F053D0-0DC9-4480-9D7B-3C1FA4B5310B@oracle.com> > On Feb 17, 2016, at 5:41 AM, Aleksey Shipilev wrote: > > On 02/17/2016 01:48 PM, Paul Sandoz wrote: >>> On 17 Feb 2016, at 10:36, Aleksey Shipilev wrote: >>> Yes: >>> http://cr.openjdk.java.net/~shade/8149835/webrev.jdk.02/ >>> http://cr.openjdk.java.net/~shade/8149835/webrev.langtools.01/ >>> >>> I have also changed to Lookup.IMPL_LOOKUP when looking up the method >>> from U.defineAnonymousClass-loaded bytecode stub. This make the load >>> sequence consistent with LambdaMetafactory. >>> >>> It still passes JPRT, java/lang/String and jake build. >>> >> >> +1 > > Thanks Paul! > > Alan, Mandy, would you like to review this as well? This should fix Jake > compatibility problems. Looks okay to me. Mandy From Roger.Riggs at Oracle.com Wed Feb 17 16:00:32 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 17 Feb 2016 11:00:32 -0500 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <56C48EF2.3030403@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <56C48EF2.3030403@oracle.com> Message-ID: <56C49920.8030006@Oracle.com> Hi, ok. A release note will be needed to document the new property. Please add the label 'release-note' and a separate comment with the proposed release note text. Thanks, Roger On 2/17/2016 10:17 AM, cheleswer sahu wrote: > Hi, > I have made changes in the property name > (jdk.lang.processReaperUseDefaultStackSize) and code as suggested in > the earlier reviews. > > --- old/src/java.base/share/classes/java/lang/ProcessHandleImpl.java > 2016-02-17 18:48:10.545639999 +0530 > +++ new/src/java.base/share/classes/java/lang/ProcessHandleImpl.java > 2016-02-17 18:48:10.081639999 +0530 > @@ -81,9 +81,8 @@ > ThreadGroup systemThreadGroup = tg; > > ThreadFactory threadFactory = grimReaper -> { > - // Our thread stack requirement is quite modest. > - Thread t = new Thread(systemThreadGroup, grimReaper, > - "process reaper", 32768); > + long stackSize = > Boolean.getBoolean("jdk.lang.processReaperUseDefaultStackSize") ? 0 : > 32768; > + Thread t = new Thread(systemThreadGroup, > grimReaper, "process reaper", stackSize); > t.setDaemon(true); > // A small attempt (probably futile) to avoid > priority inversion > t.setPriority(Thread.MAX_PRIORITY); > > I would really like to thanks "Martin" for the patch > (http://cr.openjdk.java.net/~martin/webrevs/openjdk9/tls-size-guarantee/) > which he has provided. Since we support a wider range of glibc > versions and platform using patch will > require more testing and work. I think the use of system property for > this issue will be safer at this point of time. > > Regards, > Cheleswer > > > On 1/19/2016 5:40 PM, David Holmes wrote: >> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>> | >>> Hi Cheleswer, I think Martin is suggesting something like: >>> | >>> >>> // Use a modest stack size, unless requested otherwise: >>> long stackSize = >>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >> >> Someone from core-libs needs to chime on what the appropriate >> namespace for such a property would be. >> >> David >> ----- >> >>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>> reaper", stackSize); >>> >>> ||| >>> If that tests OK for you, it may be the way we can go ahead with the >>> point fix for this. >>> >>> Regards >>> Kevin >>> | >>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>> Historical note - I never liked having a reaper thread for each >>>> subprocess, but no other reliable implementation is known. Given the >>>> potential for many reaper threads, I at least wanted to keep the >>>> memory waste low. >>>> >>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>> wrote: >>>> >>>>> + Thread t = null; >>>>> + if >>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>> "process reaper"); >>>>> + } else { >>>>> + // Our thread stack requirement is quite >>>>> modest. >>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>> "process reaper", 32768); >>>>> + } >>>> If we do end up using this strategy, cleaner would be to use >>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>> >>>> stackSize - the desired stack size for the new thread, or zero to >>>> indicate that this parameter is to be ignored. >>> > From Alan.Bateman at oracle.com Wed Feb 17 16:10:17 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Feb 2016 16:10:17 +0000 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class In-Reply-To: <56C47891.40506@oracle.com> References: <56C383AE.3000702@oracle.com> <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> <56C43F2A.4030205@oracle.com> <742A6645-6DF2-46A4-87C7-C36E0C94A870@oracle.com> <56C47891.40506@oracle.com> Message-ID: <56C49B69.8040001@oracle.com> On 17/02/2016 13:41, Aleksey Shipilev wrote: > : > > Alan, Mandy, would you like to review this as well? This should fix Jake > compatibility problems. > > Yes, I think this is okay although we'll need to look at the IMPL_LOOKUP usage when we get this to jake. -Alan From aleksey.shipilev at oracle.com Wed Feb 17 16:11:26 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Wed, 17 Feb 2016 19:11:26 +0300 Subject: RFR (S) 8149835: StringConcatFactory should emit classes with the same package as the host class In-Reply-To: <56C49B69.8040001@oracle.com> References: <56C383AE.3000702@oracle.com> <8A52E683-27C4-485B-90DE-920825B8068F@gmail.com> <56C43F2A.4030205@oracle.com> <742A6645-6DF2-46A4-87C7-C36E0C94A870@oracle.com> <56C47891.40506@oracle.com> <56C49B69.8040001@oracle.com> Message-ID: <56C49BAE.9090902@oracle.com> On 02/17/2016 07:10 PM, Alan Bateman wrote: > > > On 17/02/2016 13:41, Aleksey Shipilev wrote: >> : >> >> Alan, Mandy, would you like to review this as well? This should fix Jake >> compatibility problems. >> >> > Yes, I think this is okay although we'll need to look at the IMPL_LOOKUP > usage when we get this to jake. Thanks Alan and Mandy! Lookup.IMPL_LOOKUP is used by LambdaMetafactory as well, so you would need to see into all JDK BSMs. -Aleksey From peter.levart at gmail.com Wed Feb 17 16:16:23 2016 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 17 Feb 2016 17:16:23 +0100 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C43817.7060805@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> <56C43817.7060805@gmail.com> Message-ID: <56C49CD7.8050807@gmail.com> On 02/17/2016 10:06 AM, Peter Levart wrote: On 02/17/2016 01:20 AM, Kim Barrett wrote: >> src/java.base/share/classes/jdk/internal/ref/CleanerImpl.java >> >> I don't understand why CleanerImpl needs to be changed to public in >> order to provide access to the new drainQueue. Wouldn't it be better >> to add Cleaner.drainQueue? > > An interesting idea. But I don't know if such functionality is > generally useful enough to commit to it in a public API. > > I have another idea where java.lang.ref.Cleaner would use an Executor > instead of ThreadFactory. The default would be an instance of a > single-threaded executor per Cleaner instance, but if user passed in a > ForkJoinPool, he could use it's method ForkJoinPool.awaitQuiescence() > to help the executor's thread(s) do the cleaning. ...well, this would not be a good idea as ReferenceHandler thread would then have to feed the Executor (invoke Executor::execute(Runnable)) and a user-specified Executor could abuse that. There has to be a queue API between ReferenceHandler and custom user code. Regards, Peter From christoph.langer at sap.com Wed Feb 17 16:27:43 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Wed, 17 Feb 2016 16:27:43 +0000 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: <56C3AF17.6050103@oracle.com> References: <56C38060.2030703@oracle.com> <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> <56C3AF17.6050103@oracle.com> Message-ID: Hi Joe, here's my next version, trying to implement your suggestions: http://cr.openjdk.java.net/~clanger/webrevs/8149915.1/ I also took the chance to do some cleanups and type check fixes. Hope that's ok? I'm also wondering why some files have a copyright header like this, e.g. src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11DTDScannerImpl.java: /* * reserved comment block * DO NOT REMOVE OR ALTER! */ Shouldn't there be the standard copyright header or is there some reason behind it? Thanks in advance and best regards Christoph -----Original Message----- From: huizhe wang [mailto:huizhe.wang at oracle.com] Sent: Mittwoch, 17. Februar 2016 00:22 To: Langer, Christoph Cc: core-libs-dev at openjdk.java.net Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE On 2/16/2016 2:13 PM, Langer, Christoph wrote: > Hi Joe, > > thanks for your great suggestions. I'll come up with a new webrev tomorrow. > > However, I'm still wondering if the change for the default value of XML_SECURITY_PROPERTY_MANAGER in XMLDocumentFragmentScannerImpl should be done anyway as I guess an XMLObject is more expected rather than a String from other places in the code that could make use of such a default...? Is that true? You're right, the default value was incorrectly set to the default value of a property manager. "null" would be good in this case since there's no need to create a new instance here. Best, Joe > > Best > Christoph > > -----Original Message----- > From: huizhe wang [mailto:huizhe.wang at oracle.com] > Sent: Dienstag, 16. Februar 2016 21:03 > To: Langer, Christoph > Cc: core-libs-dev at openjdk.java.net > Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE > > Hi Christoph, > > At initialization of XSDHandler (in reset), note there are two local > variables for the security / property manager, change them to instance > variables, then when you need to create annotation validator, set both > on the validator, e.g.: > > fAnnotationValidator.setProperty(SECURITY_MANAGER, > fSecurityManager); > fAnnotationValidator.setProperty(XML_SECURITY_PROPERTY_MANAGER, > fSecurityPropertyMgr); > > That should fix the issue. > > There are a bunch of rawtypes/unchecked warnings in this class, bonus > but not required :-) > > For the test, in the past (jaxp standalone), we've created tests for > each issue, nowadays we'd prefer to consolidate them. We don't have > "SchemaTest" yet, so you may name the test "SchemaTest", then for the > @summary, make it general, e.g. "Test Schema creation", and move the > specific comment to the test case and make it specific to the issue: > > /* > * @bug 8149915 > * Verifies that the annotation validator is initialized with the security manager for schema > * creation with http://apache.org/xml/features/validate-annotations=true. > */ > > @Test > public void testValidation() throws Exception { > > > The test is new, the copyright year would be "2016," instead of "2014, > 2016,". > > Best, > Joe > > On 2/16/2016 4:49 AM, Langer, Christoph wrote: >> Hi, >> >> can you please review the following change for JDK-8149915: >> >> http://cr.openjdk.java.net/~clanger/webrevs/8149915.0/ >> >> I'm not sure if it is the right thing to set the default for XML_SECURITY_PROPERTY_MANAGER to a new XMLSecurityPropertyManager() object instead of the String "all". But it must not be a String, I think. >> >> And also I don't know if my approach to add a field "fSecurityManager" to XML11Configuration. >> >> Please advise and let me know how I can do it better. >> >> Thanks in advance and best regards >> Christoph >> From joe.darcy at oracle.com Wed Feb 17 16:33:13 2016 From: joe.darcy at oracle.com (joe darcy) Date: Wed, 17 Feb 2016 08:33:13 -0800 Subject: JDK 9 RFR of JDK-8149920: Remove intermittent key from jdk_core tests In-Reply-To: <56C4710C.3010609@oracle.com> References: <56C4710C.3010609@oracle.com> Message-ID: <56C4A0C9.5010808@oracle.com> Hi Amy, Looks fine; thanks, -Joe On 2/17/2016 5:09 AM, Amy Lu wrote: > sun/nio/cs/FindEncoderBugs.java > sun/nio/cs/FindDecoderBugs.java > java/util/concurrent/Phaser/Basic.java > java/net/Inet6Address/serialize/Inet6AddressSerializationTest.java > java/net/SocketPermission/SocketPermissionTest.java > java/lang/ProcessHandle/InfoTest.java > com/sun/jndi/ldap/LdapTimeoutTest.java > > For above tests, previous intermittent test failure issues are fixed, > no open bug (no failure reported). > > This patch is to remove @key intermittent from tests. > > bug: https://bugs.openjdk.java.net/browse/JDK-8149920 > webrev: http://cr.openjdk.java.net/~amlu/8149920/webrev.00/ > > Thanks, > Amy From mandy.chung at oracle.com Wed Feb 17 19:31:07 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Wed, 17 Feb 2016 11:31:07 -0800 Subject: Review Request: 8074069: Move JDK-specific API to jdk.* module Message-ID: This patch prepares to move closed JDK-specific APIs from java.base to jdk.net module as specified in the design principles in JEP 200 that standard modules should not non-standard APIs. Webrev: http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8074069/webrev.00/index.html Mandy From Alan.Bateman at oracle.com Wed Feb 17 19:35:14 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Feb 2016 19:35:14 +0000 Subject: Review Request: 8074069: Move JDK-specific API to jdk.* module In-Reply-To: References: Message-ID: <56C4CB72.3080305@oracle.com> On 17/02/2016 19:31, Mandy Chung wrote: > This patch prepares to move closed JDK-specific APIs from java.base to jdk.net module as specified in the design principles in JEP 200 that standard modules should not non-standard APIs. > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8074069/webrev.00/index.html > This looks okay to me. At one point we kept boot.modules in alphabetic order but it has been edited many times and this is no longer the case. -Alan From huizhe.wang at oracle.com Wed Feb 17 20:08:34 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 17 Feb 2016 12:08:34 -0800 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: References: <56C38060.2030703@oracle.com> <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> <56C3AF17.6050103@oracle.com> Message-ID: <56C4D342.9070705@oracle.com> Hi Langer, Thanks for working on the patch. On 2/17/2016 8:27 AM, Langer, Christoph wrote: > Hi Joe, > > here's my next version, trying to implement your suggestions: http://cr.openjdk.java.net/~clanger/webrevs/8149915.1/ Looks good in general. > > I also took the chance to do some cleanups and type check fixes. Hope that's ok? Yes, it's good to have some cleanups. For the generic initialization, it's preferable to have no redundant type arguments, for example, instead of: protected Stack fEntityStack = new Stack(); do: protected Stack fEntityStack = new Stack<>(); Some of the "cleanup" in the webrev were reversing <> with added type argument(s), that would be unnecessary. For the obsolete Vector, it would be good to replace it with ArrayList. Copyright year "2016" in XSAttributeChecker needs to be "2016," or else the script would complain. In XSDHandler, note that there's another call to get SECURITY_MANAGER towards the end of the reset method (at 3572 in your new version) that may be removed. You may also consolidate most of those setFeature/Property statements in one try-catch block if you want. For the test, the @bug tag is still desirable in the general comment section. For now, it's @bug 8149915. As we add more test to the class in the future, we'd then append more bug ids, @bug 8149915, xxxxxxx. > > I'm also wondering why some files have a copyright header like this, e.g. src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11DTDScannerImpl.java: > /* > * reserved comment block > * DO NOT REMOVE OR ALTER! > */ > > Shouldn't there be the standard copyright header or is there some reason behind it? The block is used by the licensing script to swap with license information required by licensees. For code that came from Xerces, we should keep the block as is with the Apache License. But when we make changes, we're required to put in the copyright header. Since the changes you made are cleanup / removing unused fields, it's okay to keep the block as is. Best, Joe > > Thanks in advance and best regards > Christoph > > > -----Original Message----- > From: huizhe wang [mailto:huizhe.wang at oracle.com] > Sent: Mittwoch, 17. Februar 2016 00:22 > To: Langer, Christoph > Cc: core-libs-dev at openjdk.java.net > Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE > > > On 2/16/2016 2:13 PM, Langer, Christoph wrote: >> Hi Joe, >> >> thanks for your great suggestions. I'll come up with a new webrev tomorrow. >> >> However, I'm still wondering if the change for the default value of XML_SECURITY_PROPERTY_MANAGER in XMLDocumentFragmentScannerImpl should be done anyway as I guess an XMLObject is more expected rather than a String from other places in the code that could make use of such a default...? Is that true? > You're right, the default value was incorrectly set to the default value > of a property manager. "null" would be good in this case since there's > no need to create a new instance here. > > Best, > Joe > >> Best >> Christoph >> >> -----Original Message----- >> From: huizhe wang [mailto:huizhe.wang at oracle.com] >> Sent: Dienstag, 16. Februar 2016 21:03 >> To: Langer, Christoph >> Cc: core-libs-dev at openjdk.java.net >> Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE >> >> Hi Christoph, >> >> At initialization of XSDHandler (in reset), note there are two local >> variables for the security / property manager, change them to instance >> variables, then when you need to create annotation validator, set both >> on the validator, e.g.: >> >> fAnnotationValidator.setProperty(SECURITY_MANAGER, >> fSecurityManager); >> fAnnotationValidator.setProperty(XML_SECURITY_PROPERTY_MANAGER, >> fSecurityPropertyMgr); >> >> That should fix the issue. >> >> There are a bunch of rawtypes/unchecked warnings in this class, bonus >> but not required :-) >> >> For the test, in the past (jaxp standalone), we've created tests for >> each issue, nowadays we'd prefer to consolidate them. We don't have >> "SchemaTest" yet, so you may name the test "SchemaTest", then for the >> @summary, make it general, e.g. "Test Schema creation", and move the >> specific comment to the test case and make it specific to the issue: >> >> /* >> * @bug 8149915 >> * Verifies that the annotation validator is initialized with the security manager for schema >> * creation with http://apache.org/xml/features/validate-annotations=true. >> */ >> >> @Test >> public void testValidation() throws Exception { >> >> >> The test is new, the copyright year would be "2016," instead of "2014, >> 2016,". >> >> Best, >> Joe >> >> On 2/16/2016 4:49 AM, Langer, Christoph wrote: >>> Hi, >>> >>> can you please review the following change for JDK-8149915: >>> >>> http://cr.openjdk.java.net/~clanger/webrevs/8149915.0/ >>> >>> I'm not sure if it is the right thing to set the default for XML_SECURITY_PROPERTY_MANAGER to a new XMLSecurityPropertyManager() object instead of the String "all". But it must not be a String, I think. >>> >>> And also I don't know if my approach to add a field "fSecurityManager" to XML11Configuration. >>> >>> Please advise and let me know how I can do it better. >>> >>> Thanks in advance and best regards >>> Christoph >>> From lance.andersen at oracle.com Wed Feb 17 20:46:57 2016 From: lance.andersen at oracle.com (Lance Andersen) Date: Wed, 17 Feb 2016 15:46:57 -0500 Subject: RFR (JAXP) 8146237: PREFER from Features API taking precedence over catalog file In-Reply-To: <56C3CD81.40007@oracle.com> References: <56C3CD81.40007@oracle.com> Message-ID: <183FA680-2E5E-49E6-8C9F-E49B1443C102@oracle.com> looks OK Joe On Feb 16, 2016, at 8:31 PM, huizhe wang wrote: > Hi, > > Please review a fix on missing property settings read from catalog files. Values from catalog files shall override any other settings including those through the API, e.g. builder().with(CatalogFeatures.Feature.PREFER, "system"). > > JBS: https://bugs.openjdk.java.net/browse/JDK-8146237 > webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8146237/webrev/ > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From rachel.protacio at oracle.com Wed Feb 17 21:21:46 2016 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Wed, 17 Feb 2016 16:21:46 -0500 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56A94C8C.9050306@oracle.com> References: <56A94C8C.9050306@oracle.com> Message-ID: <56C4E46A.9050702@oracle.com> Hello, everyone, We are moving forward with "JNI_VERSION_9". If it later turns out that it should be "9_0", we will file a separate bug, but the plain 9 is most likely. There is currently a compatibility request in the approval process. Here is the updated code, which builds properly and still prints the correct version from my own private test. hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot.01/ jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk.01/ Thank you, Rachel On 1/27/2016 6:02 PM, Rachel Protacio wrote: > Hello! > > Small but important change for review: updating the JNI_VERSION and in > so doing, changing the format from JNI_VERSION_1_x to JNI_VERSION_x_y > (see code/bug for details). > > Bug: https://bugs.openjdk.java.net/browse/JDK-8145098 > > hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot/ > jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk/ > > I grep'ed through the code to find references to the current JNI > version and believe I have caught all the ones that needed changing, > plus the fact that all these tests pass: > > * local hotspot jtreg tests > * my own sample JNI test to print and visually inspect the version > (essentially what is performed by the updated > hotspot/test/native_sanity/JniVersion.java test) > * jck vm tests > * local rbt colcated and noncolocated tests, especially for the > purpose of hitting tonga/src/nsk/share/jvmti tests where > JNI_GetVersion() is used > > Thank you, > Rachel From huizhe.wang at oracle.com Wed Feb 17 21:33:28 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 17 Feb 2016 13:33:28 -0800 Subject: RFR (JAXP) 8146237: PREFER from Features API taking precedence over catalog file In-Reply-To: <183FA680-2E5E-49E6-8C9F-E49B1443C102@oracle.com> References: <56C3CD81.40007@oracle.com> <183FA680-2E5E-49E6-8C9F-E49B1443C102@oracle.com> Message-ID: <56C4E728.7000807@oracle.com> Thanks Lance! On 2/17/2016 12:46 PM, Lance Andersen wrote: > looks OK Joe > > On Feb 16, 2016, at 8:31 PM, huizhe wang > wrote: > >> Hi, >> >> Please review a fix on missing property settings read from catalog >> files. Values from catalog files shall override any other settings >> including those through the API, e.g. >> builder().with(CatalogFeatures.Feature.PREFER, "system"). >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8146237 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk9/8146237/webrev/ >> >> >> Thanks, >> Joe > > > > Lance > Andersen| Principal Member of Technical Staff | +1.781.442.2037 > Oracle Java Engineering > 1 Network Drive > Burlington, MA 01803 > Lance.Andersen at oracle.com > > > From kim.barrett at oracle.com Wed Feb 17 22:34:32 2016 From: kim.barrett at oracle.com (Kim Barrett) Date: Wed, 17 Feb 2016 17:34:32 -0500 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C43817.7060805@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> <56C43817.7060805@gmail.com> Message-ID: <7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com> > On Feb 17, 2016, at 4:06 AM, Peter Levart wrote: > > Hi Kim, > > Thanks for looking into this. Answers inline? Peter, Thanks for the explanations. > On 02/17/2016 01:20 AM, Kim Barrett wrote: >>> However, I recall Roger saying there were existing tests that >> failed when certain uses of sun.misc.Cleaner were replaced with >> java.lang.ref.Cleaner. I don't remember the details, but maybe Roger >> does. > > If the failing test was the following: > > java/nio/Buffer/DirectBufferAllocTest.java > > ...then it has been taken care of (see Bits.java). That looks familiar. And yes, I see what you did there, and I don't think Roger's initial prototype testing did anything similar, so indeed this is likely the failure he encountered. Though I'm still inclined to object to that form of drainQueue (see below). >> I don't understand why CleanerImpl needs to be changed to public in >> order to provide access to the new drainQueue. Wouldn't it be better >> to add Cleaner.drainQueue? > > An interesting idea. But I don't know if such functionality is generally useful enough to commit to it in a public API. java.desktop:sun.java2d.Disposer.pollRemove seems to me to be addressing an essentially similar requirement, with java.desktop:sun.font.StrikeCache being the user of that API. Of course, that's already got all the scaffolding for using phantom references, and there's no need to rewrite it to use java.lang.ref.Cleaner. But maybe there are others like this? In any case, I really dislike the approach of exposing the CleanerImpl object to get at this functionality. >> Some of the other changes here don't seem related to the problem at >> hand. ... > > One thing that this change unfortunately requires is to get rid of lambdas and method references in the implementation and dependencies of java.land.ref.Cleaner API, because it gets used early in the boot-up sequence when java.lang.invoke infrastructure is not ready yet. Oh, right! Bootstrapping issues! > The alternative to CleanerCleanable is a no-op Runnable implementation passed to PhantomCleanableRef constructor. ? OK. Now I understand. > Making CleanerImpl implement Runnable ? I'd be fine with this if the CleanerImpl wasn't exposed as part of the drainQueue functionality. >> ------------------------------------------------------------------------------ >> src/java.base/share/classes/sun/nio/fs/NativeBuffer.java >> 76 Cleaner.Cleanable cleanable() { >> 77 return cleanable; >> 78 } >> >> [pre-existing, but if we're changing things anyway...] >> >> I'm kind of surprised by an accessor function (both here and in the >> original code) rather than a cleanup function. Is there a use case >> for anything other than buffer.cleanable().clean()? > > It can't be, since both old Cleaner and new Cleanable have only got a single method - clean(). So this could be replaced with void clean() { cleanable.clean(); } To me, that seems better. >> Similarly for the DirectBuffer interface. > > This one is a critical method, used by various 3rd party softwares... I want to cover my ears when people start talking about some of the things that have done? OK. From david.holmes at oracle.com Thu Feb 18 01:15:11 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Feb 2016 11:15:11 +1000 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56C4E46A.9050702@oracle.com> References: <56A94C8C.9050306@oracle.com> <56C4E46A.9050702@oracle.com> Message-ID: <56C51B1F.5030502@oracle.com> Hi Rachel, On 18/02/2016 7:21 AM, Rachel Protacio wrote: > Hello, everyone, > > We are moving forward with "JNI_VERSION_9". If it later turns out that > it should be "9_0", we will file a separate bug, but the plain 9 is most > likely. There is currently a compatibility request in the approval process. > > Here is the updated code, which builds properly and still prints the > correct version from my own private test. > hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot.01/ > jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk.01/ Looks good! Thanks, David ----- > Thank you, > Rachel > > > On 1/27/2016 6:02 PM, Rachel Protacio wrote: >> Hello! >> >> Small but important change for review: updating the JNI_VERSION and in >> so doing, changing the format from JNI_VERSION_1_x to JNI_VERSION_x_y >> (see code/bug for details). >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8145098 >> >> hotspot repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_hotspot/ >> jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk/ >> >> I grep'ed through the code to find references to the current JNI >> version and believe I have caught all the ones that needed changing, >> plus the fact that all these tests pass: >> >> * local hotspot jtreg tests >> * my own sample JNI test to print and visually inspect the version >> (essentially what is performed by the updated >> hotspot/test/native_sanity/JniVersion.java test) >> * jck vm tests >> * local rbt colcated and noncolocated tests, especially for the >> purpose of hitting tonga/src/nsk/share/jvmti tests where >> JNI_GetVersion() is used >> >> Thank you, >> Rachel > From david.holmes at oracle.com Thu Feb 18 01:54:25 2016 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Feb 2016 11:54:25 +1000 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56C49671.3060107@Oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> <56C15A52.1050403@oracle.com> <56C396B7.8000803@Oracle.com> <56C4306C.6060206@oracle.com> <56C49671.3060107@Oracle.com> Message-ID: <56C52451.6040108@oracle.com> On 18/02/2016 1:49 AM, Roger Riggs wrote: > Hi David, > > On 2/17/2016 3:33 AM, David Holmes wrote: >> Hi Roger, >> >> On 17/02/2016 7:37 AM, Roger Riggs wrote: >>> Hi David, >>> >>> Webrev updated in place: >>> http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ >> >> Thanks for those tweaks. >> >> I'm still perplexed by the test. We established that for USR2, PIPE >> and XFSZ the handler is not actually invoked - yet you are testing >> that case. ?? > I'll have to dig a bit more. > When that test is run, the SignalHandler does get called with the > expected signal. > The test succeeds on Linux with jdk 8 and jdk 9. In my local testing (with the original java.util.Signal changes) raise() throws IllegalStateException for USR2, PIPE and XFSZ (even though register succeeded). If I can get the time I'll try applying these exact changes and testing them. Thanks, David > Roger > >> >> David >> >>> On 2/14/2016 11:55 PM, David Holmes wrote: >>>> Hi Roger, >>>> >>>> A few mostly minor comments ... >>>> >>>> On 13/02/2016 3:47 AM, Roger Riggs wrote: >>>>> Please review moving the functionality of sun.misc.Signal to >>>>> jdk.internal.misc and >>>>> creating a wrapper sun.misc.Signal for existing external uses. >>>>> >>>>> +++ This time including the hotspot changes to update the target of >>>>> the >>>>> upcalls. >>>>> >>>>> A new replacement API will be considered separately. >>>>> >>>>> The update includes a test that passes with or without the changes. >>>>> (Except for an NPE instead of SEGV if null is passed). >>>>> >>>>> Webrev: >>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ >>>> >>>> I take it we don't care about reorder files any more? >>> There does not seem to be any particular guidance about how/when to use >>> them on Solaris. >>> So it seemed as reasonable to remove the entries as to update them. >>>> >>>> --- >>>> >>>> src/java.base/share/classes/sun/misc/Signal.java >>>> >>>> The @since should not be changed to 9. You only changed the >>>> implementation not the API. Conversely the renamed class should >>>> arguably be @since 9 and not @since 1.2. But it is wrong to say >>>> sun.misc.Signal is @since 9. >>> The new sun.misc.Signal is a wrapper around the original implementation >>> now in jdk.internal.misc. >>> I had modified the mercurial commands to rename sun.misc.Signal to >>> jdk.internal.misc to preserve the history >>> of the implementation. >>> >>> I will correct sun.misc.Signal to retain the original @since and use >>> @since 9 for the jdk.internal.misc >>>> >>>> The *Handler.of method name reads strangely to me - "for" would be >>>> better but I presume that is not allowed. >>> The xx.of () pattern has been used recently. >>>> >>>> --- >>>> >>>> src/java.base/share/classes/jdk/internal/misc/Signal.java >>>> >>>> Not sure I understand the role of NativeSignalHandler any more - given >>>> it can't actually be used ?? >>> They are used to retain the previous handler and can be used to restore >>> that handler. >>> The ability to invoke the handler directly was removed as a >>> simplification. >>>> >>>> --- >>>> >>>> test/sun/misc/SunMiscSignalTest.java >>>> >>>> Based on our email discussion this test can not be testing that the >>>> handler actually gets invoked - as we know it does not in some cases. >>>> I have doubts that sun.misc.Signal ever intended to support all the >>>> signals you are testing. >>> There were no tests for sun.misc.Signal. I created the tests based on >>> the current implementations. >>> They can be updated to reflect current support for handle, raise, and >>> whether a signal handler is called >>> and of course -Xrs. It should allow detection of unintended changes in >>> behavior. >>> >>>> >>>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ >>>> >>>> Hotspot changes are fine. >>> >>> Thanks, Roger >>>> >>>> Thanks, >>>> David >>>> ----- >>>> >>>>> Issue: >>>>> https://bugs.openjdk.java.net/browse/JDK-8149750 >>>>> >>>>> Thanks, Roger >>>>> >>>>> JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>> >>>>> >>> > From Alan.Bateman at oracle.com Thu Feb 18 07:48:51 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 Feb 2016 07:48:51 +0000 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56C4E46A.9050702@oracle.com> References: <56A94C8C.9050306@oracle.com> <56C4E46A.9050702@oracle.com> Message-ID: <56C57763.4050907@oracle.com> On 17/02/2016 21:21, Rachel Protacio wrote: > Hello, everyone, > > We are moving forward with "JNI_VERSION_9". If it later turns out that > it should be "9_0", we will file a separate bug, but the plain 9 is > most likely. There is currently a compatibility request in the > approval process. > > Here is the updated code, which builds properly and still prints the > correct version from my own private test. > hotspot repo webrev: > http://cr.openjdk.java.net/~rprotacio/JNI_hotspot.01/ > jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk.01/ This looks good to me. Iris - I assume you'll add a section to JEP 223 for this, it seems like the right place to capture this. -Alan From vladimir.x.ivanov at oracle.com Thu Feb 18 08:28:15 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 18 Feb 2016 11:28:15 +0300 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C487F1.3030609@oracle.com> References: <56C487F1.3030609@oracle.com> Message-ID: <56C5809F.3080506@oracle.com> Shilpi, _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM to LF: LF_HIDDEN_SIG LF_COMPILED_SIG FORCEINLINE_SIG DONTINLINE_SIG Otherwise, looks fine. Best regards, Vladimir Ivanov On 2/17/16 5:47 PM, shilpi rastogi wrote: > Hi All, > > Please review fix for the following bug- > > https://bugs.openjdk.java.net/browse/JDK-8144931 > http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ > > > Thanks, > Shilpi From kevin.walls at oracle.com Thu Feb 18 10:24:03 2016 From: kevin.walls at oracle.com (Kevin Walls) Date: Thu, 18 Feb 2016 10:24:03 +0000 Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in executables with 32k tbss/tdata In-Reply-To: <56C48EF2.3030403@oracle.com> References: <56962632.7080404@oracle.com> <569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com> <56C48EF2.3030403@oracle.com> Message-ID: <56C59BC3.6040207@oracle.com> Hi Cheleswer, Looks good to me. Thanks Kevin (Also, as one of the comments was that there may be no real cost to using default stack sizes here (on most systems...?), having jdk.lang.processReaperUseDefaultStackSize gives us a way to test that widely, and one day the 32k may be able to disappear.) On 17/02/2016 15:17, cheleswer sahu wrote: > Hi, > I have made changes in the property name > (jdk.lang.processReaperUseDefaultStackSize) and code as suggested in > the earlier reviews. > > --- old/src/java.base/share/classes/java/lang/ProcessHandleImpl.java > 2016-02-17 18:48:10.545639999 +0530 > +++ new/src/java.base/share/classes/java/lang/ProcessHandleImpl.java > 2016-02-17 18:48:10.081639999 +0530 > @@ -81,9 +81,8 @@ > ThreadGroup systemThreadGroup = tg; > > ThreadFactory threadFactory = grimReaper -> { > - // Our thread stack requirement is quite modest. > - Thread t = new Thread(systemThreadGroup, grimReaper, > - "process reaper", 32768); > + long stackSize = > Boolean.getBoolean("jdk.lang.processReaperUseDefaultStackSize") ? 0 : > 32768; > + Thread t = new Thread(systemThreadGroup, > grimReaper, "process reaper", stackSize); > t.setDaemon(true); > // A small attempt (probably futile) to avoid > priority inversion > t.setPriority(Thread.MAX_PRIORITY); > > I would really like to thanks "Martin" for the patch > (http://cr.openjdk.java.net/~martin/webrevs/openjdk9/tls-size-guarantee/) > which he has provided. Since we support a wider range of glibc > versions and platform using patch will > require more testing and work. I think the use of system property for > this issue will be safer at this point of time. > > Regards, > Cheleswer > > > On 1/19/2016 5:40 PM, David Holmes wrote: >> On 19/01/2016 9:53 PM, Kevin Walls wrote: >>> | >>> Hi Cheleswer, I think Martin is suggesting something like: >>> | >>> >>> // Use a modest stack size, unless requested otherwise: >>> long stackSize = >>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768; >> >> Someone from core-libs needs to chime on what the appropriate >> namespace for such a property would be. >> >> David >> ----- >> >>> Thread t = new Thread(systemThreadGroup, grimReaper, "process >>> reaper", stackSize); >>> >>> ||| >>> If that tests OK for you, it may be the way we can go ahead with the >>> point fix for this. >>> >>> Regards >>> Kevin >>> | >>> On 18/01/2016 16:52, Martin Buchholz wrote: >>>> Historical note - I never liked having a reaper thread for each >>>> subprocess, but no other reliable implementation is known. Given the >>>> potential for many reaper threads, I at least wanted to keep the >>>> memory waste low. >>>> >>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu >>>> wrote: >>>> >>>>> + Thread t = null; >>>>> + if >>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) { >>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>> "process reaper"); >>>>> + } else { >>>>> + // Our thread stack requirement is quite >>>>> modest. >>>>> + t = new Thread(systemThreadGroup, grimReaper, >>>>> "process reaper", 32768); >>>>> + } >>>> If we do end up using this strategy, cleaner would be to use >>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long- >>>> >>>> stackSize - the desired stack size for the new thread, or zero to >>>> indicate that this parameter is to be ignored. >>> > From shilpi.rastogi at oracle.com Thu Feb 18 11:18:12 2016 From: shilpi.rastogi at oracle.com (shilpi.rastogi at oracle.com) Date: Thu, 18 Feb 2016 16:48:12 +0530 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C5809F.3080506@oracle.com> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> Message-ID: <56C5A874.90606@oracle.com> Thank You Vladimir! I have done the changes. Please review the updated patch- http://cr.openjdk.java.net/~srastogi/8144931/webrev.02/ Regards, Shilpi On 2/18/2016 1:58 PM, Vladimir Ivanov wrote: > Shilpi, > > _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM to LF: > LF_HIDDEN_SIG > LF_COMPILED_SIG > FORCEINLINE_SIG > DONTINLINE_SIG > > Otherwise, looks fine. > > Best regards, > Vladimir Ivanov > > On 2/17/16 5:47 PM, shilpi rastogi wrote: >> Hi All, >> >> Please review fix for the following bug- >> >> https://bugs.openjdk.java.net/browse/JDK-8144931 >> http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ >> >> >> Thanks, >> Shilpi From chris.hegarty at oracle.com Thu Feb 18 11:22:16 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Feb 2016 11:22:16 +0000 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56C396E1.3020305@Oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> <56C1B4DD.2060903@oracle.com> <56C396E1.3020305@Oracle.com> Message-ID: On 16 Feb 2016, at 21:38, Roger Riggs wrote: > Hi Chris, > > Webrev updated in place: > http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ Thanks Roger, I?m happy with the source changes. -Chris. > On 2/15/2016 6:22 AM, Chris Hegarty wrote: >> On 12/02/16 17:47, Roger Riggs wrote: >>> Please review moving the functionality of sun.misc.Signal to >>> jdk.internal.misc and >>> creating a wrapper sun.misc.Signal for existing external uses. >>> >>> +++ This time including the hotspot changes to update the target of the >>> upcalls. >>> >>> A new replacement API will be considered separately. >>> >>> The update includes a test that passes with or without the changes. >>> (Except for an NPE instead of SEGV if null is passed). >>> >>> Webrev: >>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ >> >> Overall looks ok, and satisfies the requirement of JEP 260. >> >> It took me a while to satisfy myself that it is ok to "ignore" >> the passed Signal in the wrapper's 'handle' methods. The assumption >> is that this wrapper's signal field and the passes signal are, MUST, >> represent the same underlying signal. Maybe an assert to make this >> explicit? > The jdk.internal.misc.Signal passed to the wrapper's methods needs to be > mapped to the corresponding sun.misc.Signal but instead of maintaining a map > the s.m.Signal instance is kept in the wrapper of the s.m.SignalHandler. >> >> Looking at j.i.m.Signal., I can see that you explicitly check >> for the 'SIG' prefix and prepend it if not there, but toString() also >> prepends it. Will getName also be impacted by this ( it may now have >> the name prepended with 'SIG', where it previously was not ). > Yes, there was a bug there; reverting to backward compatible behavior. > HotSpot recognizes different forms of signal names on Windows and Posix. [1] > To be compatible with previous versions, the "SIG" prefix allowed by the HotSpot change > (only on Posix) is not supported and throws IllegalArgumentException. > > Thanks, Roger > > [1] https://bugs.openjdk.java.net/browse/JDK-8149748 > >> >>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ >> >> Looks fine. >> >> -Chris. > From Alan.Bateman at oracle.com Thu Feb 18 11:33:54 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 Feb 2016 11:33:54 +0000 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: References: <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC.7080102@oracle.com> <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> <56C1C4F3.9000505@oracle.com> Message-ID: <56C5AC22.7010609@oracle.com> On 15/02/2016 16:30, Steve Drach wrote: > Thank you Alan. I?ll address the issues you bring up before integration. Thanks. Are you planning to update the webrev too as it would be nice to see the final javadoc? -Alan From chris.hegarty at oracle.com Thu Feb 18 12:56:10 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Feb 2016 12:56:10 +0000 Subject: RFR [9] 8150163: JarFileSystem support for MRJARs should use the JDK specific Version API Message-ID: <3B35D44C-A88F-4725-8782-F5CF7CA84E35@oracle.com> To support MRJARs, currently JarFileSystem uses sun.misc.Version to retrieve the major version. It should be updated to use the new JDK specific Version API. http://cr.openjdk.java.net/~chegar/8150163/ https://bugs.openjdk.java.net/browse/JDK-8150163 -Chris. From vladimir.x.ivanov at oracle.com Thu Feb 18 13:09:25 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 18 Feb 2016 16:09:25 +0300 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C5A874.90606@oracle.com> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> Message-ID: <56C5C285.4070509@oracle.com> Reviewed. Best regards, Vladimir Ivanov On 2/18/16 2:18 PM, shilpi.rastogi at oracle.com wrote: > Thank You Vladimir! > > I have done the changes. Please review the updated patch- > > http://cr.openjdk.java.net/~srastogi/8144931/webrev.02/ > > Regards, > Shilpi > > On 2/18/2016 1:58 PM, Vladimir Ivanov wrote: >> Shilpi, >> >> _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM to LF: >> LF_HIDDEN_SIG >> LF_COMPILED_SIG >> FORCEINLINE_SIG >> DONTINLINE_SIG >> >> Otherwise, looks fine. >> >> Best regards, >> Vladimir Ivanov >> >> On 2/17/16 5:47 PM, shilpi rastogi wrote: >>> Hi All, >>> >>> Please review fix for the following bug- >>> >>> https://bugs.openjdk.java.net/browse/JDK-8144931 >>> http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ >>> >>> >>> Thanks, >>> Shilpi > From christoph.langer at sap.com Thu Feb 18 13:15:02 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 18 Feb 2016 13:15:02 +0000 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: <56C4D342.9070705@oracle.com> References: <56C38060.2030703@oracle.com> <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> <56C3AF17.6050103@oracle.com> <56C4D342.9070705@oracle.com> Message-ID: <56c7676f64ad49bd9dc9cf8cf1ed617f@DEWDFE13DE11.global.corp.sap> Hi Joe, here is the next version: http://cr.openjdk.java.net/~clanger/webrevs/8149915.2/ Hopefully the final one :-) > Yes, it's good to have some cleanups. For the generic initialization, > it's preferable to have no redundant type arguments, for example, > instead of: > > protected Stack fEntityStack = new Stack(); > > do: > protected Stack fEntityStack = new Stack<>(); > > Some of the "cleanup" in the webrev were reversing <> with added type > argument(s), that would be unnecessary. I think I did that on all the places that I touched now. > For the obsolete Vector, it would be good to replace it with ArrayList. There are so many Vector instances in this code that it's out of scope for me now to touch them all... > Copyright year "2016" in XSAttributeChecker needs to be "2016," or else > the script would complain. For that one I thought that it would look like "2015,2016" if the initial copyright was done in 2015. But I changed it as you suggested. > In XSDHandler, note that there's another call to get SECURITY_MANAGER > towards the end of the reset method (at 3572 in your new version) that > may be removed. You may also consolidate most of those > setFeature/Property statements in one try-catch block if you want. OK, I did some further refacturing here. However, I've left the try/catch blocks as I think it's the intention to silently catch some exceptions but then try to go on with the next property. > For the test, the @bug tag is still desirable in the general comment > section. For now, it's @bug 8149915. As we add more test to the class in > the future, we'd then append more bug ids, @bug 8149915, xxxxxxx. Done. I also left the @bug in the comments for the test method. > > I'm also wondering why some files have a copyright header like this, e.g. > src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11D > TDScannerImpl.java: > > /* > > * reserved comment block > > * DO NOT REMOVE OR ALTER! > > */ > > > > Shouldn't there be the standard copyright header or is there some reason > behind it? > > The block is used by the licensing script to swap with license > information required by licensees. For code that came from Xerces, we > should keep the block as is with the Apache License. But when we make > changes, we're required to put in the copyright header. Since the > changes you made are cleanup / removing unused fields, it's okay to keep > the block as is. OK :) Thanks again for reviewing. All the Best Christoph From paul.sandoz at oracle.com Thu Feb 18 13:25:33 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 18 Feb 2016 14:25:33 +0100 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C5A874.90606@oracle.com> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> Message-ID: <826628CF-7D45-4B9F-B5D9-BE9B86C9BFA7@oracle.com> > On 18 Feb 2016, at 12:18, shilpi.rastogi at oracle.com wrote: > > Thank You Vladimir! > > I have done the changes. Please review the updated patch- > > http://cr.openjdk.java.net/~srastogi/8144931/webrev.02/ > Looking good. There are also a few other cases in InvokerBytecodeGenerator you can update: 679 mv.visitAnnotation("Ljava/lang/invoke/InjectedProfile;", true); 1328 // Suppress this method in backtraces displayed to the user. 1329 mv.visitAnnotation("Ljava/lang/invoke/LambdaForm$Hidden;", true); 1330 1331 // Don't inline the interpreter entry. 1332 mv.visitAnnotation("Ljdk/internal/vm/annotation/DontInline;", true); 1387 // Suppress this method in backtraces displayed to the user. 1388 mv.visitAnnotation("Ljava/lang/invoke/LambdaForm$Hidden;", true); 1389 1390 // Force inlining of this invoker method. 1391 mv.visitAnnotation("Ljdk/internal/vm/annotation/ForceInline;", true); 1392 That might be all the rest but i have not double checked. As a follow on task can you check other classes in j.l.i where this technique might be applied? if so we can log another issue for that e.g search using the regex \"L[\w\/]+;\?. Paul. > Regards, > Shilpi > > On 2/18/2016 1:58 PM, Vladimir Ivanov wrote: >> Shilpi, >> >> _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM to LF: >> LF_HIDDEN_SIG >> LF_COMPILED_SIG >> FORCEINLINE_SIG >> DONTINLINE_SIG >> >> Otherwise, looks fine. >> >> Best regards, >> Vladimir Ivanov >> >> On 2/17/16 5:47 PM, shilpi rastogi wrote: >>> Hi All, >>> >>> Please review fix for the following bug- >>> >>> https://bugs.openjdk.java.net/browse/JDK-8144931 >>> http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ >>> >>> >>> Thanks, >>> Shilpi > From uschindler at apache.org Thu Feb 18 13:54:23 2016 From: uschindler at apache.org (Uwe Schindler) Date: Thu, 18 Feb 2016 14:54:23 +0100 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C5A874.90606@oracle.com> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> Message-ID: <011201d16a53$e1e6e390$a5b4aab0$@apache.org> Hi, just a stupid question from somebody outside the OpenJDK developers: You are already using ASM to generate the class files. Why not also use the Type class in ASM to generate the signatures of a class constant?: Instead of: static final String LF_HIDDEN_SIG = className("Ljava/lang/invoke/LambdaForm$Hidden;"); Use the following to define the constant: import jdk.internal.org.objectweb.asm.Type; import java.lang.invoke.LambdaForm.Hidden; static final String LF_HIDDEN_SIG = Type.getDescriptor(Hidden.class); This is compile-time checked, because of the .class notation. Thanks, Uwe ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of shilpi.rastogi at oracle.com > Sent: Thursday, February 18, 2016 12:18 PM > To: Vladimir Ivanov ; core-libs- > dev at openjdk.java.net > Subject: Re: RFR 8144931: Assert class signatures are correct and refer to valid > classes > > Thank You Vladimir! > > I have done the changes. Please review the updated patch- > > http://cr.openjdk.java.net/~srastogi/8144931/webrev.02/ > > Regards, > Shilpi > > On 2/18/2016 1:58 PM, Vladimir Ivanov wrote: > > Shilpi, > > > > _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM to > LF: > > LF_HIDDEN_SIG > > LF_COMPILED_SIG > > FORCEINLINE_SIG > > DONTINLINE_SIG > > > > Otherwise, looks fine. > > > > Best regards, > > Vladimir Ivanov > > > > On 2/17/16 5:47 PM, shilpi rastogi wrote: > >> Hi All, > >> > >> Please review fix for the following bug- > >> > >> https://bugs.openjdk.java.net/browse/JDK-8144931 > >> http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ > >> > >> > >> Thanks, > >> Shilpi From paul.sandoz at oracle.com Thu Feb 18 14:05:02 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 18 Feb 2016 15:05:02 +0100 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <011201d16a53$e1e6e390$a5b4aab0$@apache.org> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> Message-ID: Hi Uwe Not a stupid question. We took the conservative approach to preserve the existing costs (avoid linkage and string generation). Paul. > On 18 Feb 2016, at 14:54, Uwe Schindler wrote: > > Hi, > > just a stupid question from somebody outside the OpenJDK developers: > You are already using ASM to generate the class files. Why not also use the Type class in ASM to generate the signatures of a class constant?: > > Instead of: > > static final String LF_HIDDEN_SIG = className("Ljava/lang/invoke/LambdaForm$Hidden;"); > > Use the following to define the constant: > > import jdk.internal.org.objectweb.asm.Type; > import java.lang.invoke.LambdaForm.Hidden; > static final String LF_HIDDEN_SIG = Type.getDescriptor(Hidden.class); > > This is compile-time checked, because of the .class notation. > > Thanks, > Uwe > > ----- > Uwe Schindler > uschindler at apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of shilpi.rastogi at oracle.com >> Sent: Thursday, February 18, 2016 12:18 PM >> To: Vladimir Ivanov ; core-libs- >> dev at openjdk.java.net >> Subject: Re: RFR 8144931: Assert class signatures are correct and refer to valid >> classes >> >> Thank You Vladimir! >> >> I have done the changes. Please review the updated patch- >> >> http://cr.openjdk.java.net/~srastogi/8144931/webrev.02/ >> >> Regards, >> Shilpi >> >> On 2/18/2016 1:58 PM, Vladimir Ivanov wrote: >>> Shilpi, >>> >>> _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM to >> LF: >>> LF_HIDDEN_SIG >>> LF_COMPILED_SIG >>> FORCEINLINE_SIG >>> DONTINLINE_SIG >>> >>> Otherwise, looks fine. >>> >>> Best regards, >>> Vladimir Ivanov >>> >>> On 2/17/16 5:47 PM, shilpi rastogi wrote: >>>> Hi All, >>>> >>>> Please review fix for the following bug- >>>> >>>> https://bugs.openjdk.java.net/browse/JDK-8144931 >>>> http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ >>>> >>>> >>>> Thanks, >>>> Shilpi > From uschindler at apache.org Thu Feb 18 14:20:50 2016 From: uschindler at apache.org (Uwe Schindler) Date: Thu, 18 Feb 2016 15:20:50 +0100 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> Message-ID: <012401d16a57$93deb480$bb9c1d80$@apache.org> Hi, Thanks! I was expecting something like this. You just want to check the class name in an assertion - that?s fine. Kind regards, Uwe P.S.: You could improve the assertion with the Type class, so it would also fail on otherwise broken descriptor strings (missing "L" or missing ";"): static boolean checkClassName(String cn) { Type tp = Type.getType(cn); // additional sanity so only valid "L;" descriptors work if (tp.getSort() != Type.OBJECT) { return false; } try { Class c = Class.forName(tp.getClassName(), false, null); return true; } catch (ClassNotFoundException e) { return false; } } ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/ > -----Original Message----- > From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > Behalf Of Paul Sandoz > Sent: Thursday, February 18, 2016 3:05 PM > Cc: Java Core Libs > Subject: Re: RFR 8144931: Assert class signatures are correct and refer to valid > classes > > Hi Uwe > > Not a stupid question. > > We took the conservative approach to preserve the existing costs (avoid > linkage and string generation). > > Paul. > > > On 18 Feb 2016, at 14:54, Uwe Schindler wrote: > > > > Hi, > > > > just a stupid question from somebody outside the OpenJDK developers: > > You are already using ASM to generate the class files. Why not also use the > Type class in ASM to generate the signatures of a class constant?: > > > > Instead of: > > > > static final String LF_HIDDEN_SIG = > className("Ljava/lang/invoke/LambdaForm$Hidden;"); > > > > Use the following to define the constant: > > > > import jdk.internal.org.objectweb.asm.Type; > > import java.lang.invoke.LambdaForm.Hidden; > > static final String LF_HIDDEN_SIG = Type.getDescriptor(Hidden.class); > > > > This is compile-time checked, because of the .class notation. > > > > Thanks, > > Uwe > > > > ----- > > Uwe Schindler > > uschindler at apache.org > > ASF Member, Apache Lucene PMC / Committer > > Bremen, Germany > > http://lucene.apache.org/ > > > >> -----Original Message----- > >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On > >> Behalf Of shilpi.rastogi at oracle.com > >> Sent: Thursday, February 18, 2016 12:18 PM > >> To: Vladimir Ivanov ; core-libs- > >> dev at openjdk.java.net > >> Subject: Re: RFR 8144931: Assert class signatures are correct and refer to > valid > >> classes > >> > >> Thank You Vladimir! > >> > >> I have done the changes. Please review the updated patch- > >> > >> http://cr.openjdk.java.net/~srastogi/8144931/webrev.02/ > >> > >> Regards, > >> Shilpi > >> > >> On 2/18/2016 1:58 PM, Vladimir Ivanov wrote: > >>> Shilpi, > >>> > >>> _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM > to > >> LF: > >>> LF_HIDDEN_SIG > >>> LF_COMPILED_SIG > >>> FORCEINLINE_SIG > >>> DONTINLINE_SIG > >>> > >>> Otherwise, looks fine. > >>> > >>> Best regards, > >>> Vladimir Ivanov > >>> > >>> On 2/17/16 5:47 PM, shilpi rastogi wrote: > >>>> Hi All, > >>>> > >>>> Please review fix for the following bug- > >>>> > >>>> https://bugs.openjdk.java.net/browse/JDK-8144931 > >>>> http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ > >>>> > >>>> > >>>> Thanks, > >>>> Shilpi > > From Alan.Bateman at oracle.com Thu Feb 18 14:22:07 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 18 Feb 2016 14:22:07 +0000 Subject: RFR [9] 8150163: JarFileSystem support for MRJARs should use the JDK specific Version API In-Reply-To: <3B35D44C-A88F-4725-8782-F5CF7CA84E35@oracle.com> References: <3B35D44C-A88F-4725-8782-F5CF7CA84E35@oracle.com> Message-ID: <56C5D38F.6070806@oracle.com> On 18/02/2016 12:56, Chris Hegarty wrote: > To support MRJARs, currently JarFileSystem uses sun.misc.Version to retrieve the > major version. It should be updated to use the new JDK specific Version API. > > http://cr.openjdk.java.net/~chegar/8150163/ > https://bugs.openjdk.java.net/browse/JDK-8150163 > This looks okay for now but I assume it may have to change again soon once a home for the new Version API is found. -Alan. From vladimir.x.ivanov at oracle.com Thu Feb 18 14:55:15 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Thu, 18 Feb 2016 17:55:15 +0300 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <012401d16a57$93deb480$bb9c1d80$@apache.org> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> <012401d16a57$93deb480$bb9c1d80$@apache.org> Message-ID: <56C5DB53.1020504@oracle.com> > P.S.: You could improve the assertion with the Type class, so it would also fail on otherwise broken descriptor strings (missing "L" or missing ";"): I like how it shapes out with Type. Thanks, Uwe! Best regards, Vladimir Ivanov > > static boolean checkClassName(String cn) { > Type tp = Type.getType(cn); > // additional sanity so only valid "L;" descriptors work > if (tp.getSort() != Type.OBJECT) { > return false; > } > try { > Class c = Class.forName(tp.getClassName(), false, null); > return true; > } catch (ClassNotFoundException e) { > return false; > } > } > > ----- > Uwe Schindler > uschindler at apache.org > ASF Member, Apache Lucene PMC / Committer > Bremen, Germany > http://lucene.apache.org/ > > >> -----Original Message----- >> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >> Behalf Of Paul Sandoz >> Sent: Thursday, February 18, 2016 3:05 PM >> Cc: Java Core Libs >> Subject: Re: RFR 8144931: Assert class signatures are correct and refer to valid >> classes >> >> Hi Uwe >> >> Not a stupid question. >> >> We took the conservative approach to preserve the existing costs (avoid >> linkage and string generation). >> >> Paul. >> >>> On 18 Feb 2016, at 14:54, Uwe Schindler wrote: >>> >>> Hi, >>> >>> just a stupid question from somebody outside the OpenJDK developers: >>> You are already using ASM to generate the class files. Why not also use the >> Type class in ASM to generate the signatures of a class constant?: >>> >>> Instead of: >>> >>> static final String LF_HIDDEN_SIG = >> className("Ljava/lang/invoke/LambdaForm$Hidden;"); >>> >>> Use the following to define the constant: >>> >>> import jdk.internal.org.objectweb.asm.Type; >>> import java.lang.invoke.LambdaForm.Hidden; >>> static final String LF_HIDDEN_SIG = Type.getDescriptor(Hidden.class); >>> >>> This is compile-time checked, because of the .class notation. >>> >>> Thanks, >>> Uwe >>> >>> ----- >>> Uwe Schindler >>> uschindler at apache.org >>> ASF Member, Apache Lucene PMC / Committer >>> Bremen, Germany >>> http://lucene.apache.org/ >>> >>>> -----Original Message----- >>>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] On >>>> Behalf Of shilpi.rastogi at oracle.com >>>> Sent: Thursday, February 18, 2016 12:18 PM >>>> To: Vladimir Ivanov ; core-libs- >>>> dev at openjdk.java.net >>>> Subject: Re: RFR 8144931: Assert class signatures are correct and refer to >> valid >>>> classes >>>> >>>> Thank You Vladimir! >>>> >>>> I have done the changes. Please review the updated patch- >>>> >>>> http://cr.openjdk.java.net/~srastogi/8144931/webrev.02/ >>>> >>>> Regards, >>>> Shilpi >>>> >>>> On 2/18/2016 1:58 PM, Vladimir Ivanov wrote: >>>>> Shilpi, >>>>> >>>>> _CLASS suffix looks redundant and you can abbreviate LAMBDA_FORM >> to >>>> LF: >>>>> LF_HIDDEN_SIG >>>>> LF_COMPILED_SIG >>>>> FORCEINLINE_SIG >>>>> DONTINLINE_SIG >>>>> >>>>> Otherwise, looks fine. >>>>> >>>>> Best regards, >>>>> Vladimir Ivanov >>>>> >>>>> On 2/17/16 5:47 PM, shilpi rastogi wrote: >>>>>> Hi All, >>>>>> >>>>>> Please review fix for the following bug- >>>>>> >>>>>> https://bugs.openjdk.java.net/browse/JDK-8144931 >>>>>> http://cr.openjdk.java.net/~srastogi/8144931/webrev.01/ >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Shilpi >>> > From rachel.protacio at oracle.com Thu Feb 18 14:56:06 2016 From: rachel.protacio at oracle.com (Rachel Protacio) Date: Thu, 18 Feb 2016 09:56:06 -0500 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56C57763.4050907@oracle.com> References: <56A94C8C.9050306@oracle.com> <56C4E46A.9050702@oracle.com> <56C57763.4050907@oracle.com> Message-ID: <56C5DB86.6070703@oracle.com> Thanks for the reviews, David and Alan! Rachel On 2/18/2016 2:48 AM, Alan Bateman wrote: > > > On 17/02/2016 21:21, Rachel Protacio wrote: >> Hello, everyone, >> >> We are moving forward with "JNI_VERSION_9". If it later turns out >> that it should be "9_0", we will file a separate bug, but the plain 9 >> is most likely. There is currently a compatibility request in the >> approval process. >> >> Here is the updated code, which builds properly and still prints the >> correct version from my own private test. >> hotspot repo webrev: >> http://cr.openjdk.java.net/~rprotacio/JNI_hotspot.01/ >> jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk.01/ > This looks good to me. > > Iris - I assume you'll add a section to JEP 223 for this, it seems > like the right place to capture this. > > -Alan From paul.sandoz at oracle.com Thu Feb 18 15:03:40 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 18 Feb 2016 16:03:40 +0100 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C5DB53.1020504@oracle.com> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> <012401d16a57$93deb480$bb9c1d80$@apache.org> <56C5DB53.1020504@oracle.com> Message-ID: > On 18 Feb 2016, at 15:55, Vladimir Ivanov wrote: > >> P.S.: You could improve the assertion with the Type class, so it would also fail on otherwise broken descriptor strings (missing "L" or missing ";"): > I like how it shapes out with Type. Thanks, Uwe! > Me too! Paul. > Best regards, > Vladimir Ivanov > >> >> static boolean checkClassName(String cn) { >> Type tp = Type.getType(cn); >> // additional sanity so only valid "L;" descriptors work >> if (tp.getSort() != Type.OBJECT) { >> return false; >> } >> try { >> Class c = Class.forName(tp.getClassName(), false, null); >> return true; >> } catch (ClassNotFoundException e) { >> return false; >> } >> } >> >> ----- >> Uwe Schindler >> uschindler at apache.org >> ASF Member, Apache Lucene PMC / Committer >> Bremen, Germany >> http://lucene.apache.org/ From chris.hegarty at oracle.com Thu Feb 18 15:52:39 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 18 Feb 2016 15:52:39 +0000 Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package Message-ID: sun.misc.Version is the core libraries part of a private interface with the JVM to query and set specific JVM version and capabilities, as well as being responsible for setting the system properties for "java.version?, "java.runtime.version", and "java.runtime.name" ( which are generated during the build ). It is not a "Critical API", as defined by JEP 260, so should be moved out of sun.misc and placed into a more appropriate package where it can be encapsulated. Version is only used by j.l.System during initialization, so I?ve taken the liberty of moving it into java.lang as package-private. It could, however, be placed in jdk.internal.misc, if it is more generally useful. That said, most other usages should be able to use the JDK specific Version API. http://cr.openjdk.java.net/~chegar/8150162/ https://bugs.openjdk.java.net/browse/JDK-8150162 -Chris. P.S. 8150163 & 8150168 are in progress to replace unnecessary usages of sun.misc.Version with the JDK specific Version. From steve.drach at oracle.com Thu Feb 18 17:18:05 2016 From: steve.drach at oracle.com (Steve Drach) Date: Thu, 18 Feb 2016 09:18:05 -0800 Subject: RFR - 8132734: java.util.jar.* changes to support multi-release jar files In-Reply-To: <56C5AC22.7010609@oracle.com> References: <564B68A4.1060400@oracle.com> <229D6BCD-B955-47EE-BE5F-5FE92F0EAA3D@oracle.com> <56A1035A.9020805@oracle.com> <5BDE763F-EBBD-4215-8D7F-2668357D8D6D@oracle.com> <56A6532D.6050805@oracle.com> <27ECD893-2839-45FE-BCBA-38E17718F8DD@oracle.com> <56AB6B! ! F8.5010302@oracle.com> <0E368717-EBD7-4EDE-946C-AD9FF0F8B0DE@oracle.com> <56AC6D86.7050101@oracle.com> <60FB9B53-14AA-447F-BB71-976D7FB417C3@oracle.com> <56AFAF6E.4060806@oracle.com> <56B1C2CC.7080102@oracle.com> <636D6E48-3BA6-4CC5-A960-426DD68F10D2@oracle.com> <56C1C4F3.9000505@oracle.com> <56C5AC22.7010609@oracle.com> Message-ID: <73FEAC8B-97F3-4387-83DD-554FF690881C@oracle.com> >> Thank you Alan. I?ll address the issues you bring up before integration. > Thanks. Are you planning to update the webrev too as it would be nice to see the final javadoc? http://cr.openjdk.java.net/~sdrach/8132734/webrev.07/index.html From huizhe.wang at oracle.com Thu Feb 18 18:19:05 2016 From: huizhe.wang at oracle.com (huizhe wang) Date: Thu, 18 Feb 2016 10:19:05 -0800 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: <56c7676f64ad49bd9dc9cf8cf1ed617f@DEWDFE13DE11.global.corp.sap> References: <56C38060.2030703@oracle.com> <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> <56C3AF17.6050103@oracle.com> <56C4D342.9070705@oracle.com> <56c7676f64ad49bd9dc9cf8cf1ed617f@DEWDFE13DE11.global.corp.sap> Message-ID: <56C60B19.3040508@oracle.com> On 2/18/2016 5:15 AM, Langer, Christoph wrote: > Hi Joe, > > here is the next version: > http://cr.openjdk.java.net/~clanger/webrevs/8149915.2/ Looks good! Thanks. Copyright year in XSAttributeChecker.java is still incorrect. I meant to say the year "2016" needs to have a comma, so in this case, it needs to be "2015, 2016, " rather than "2015, 2016". Best, Joe > > Hopefully the final one :-) > >> Yes, it's good to have some cleanups. For the generic initialization, >> it's preferable to have no redundant type arguments, for example, >> instead of: >> >> protected Stack fEntityStack = new Stack(); >> >> do: >> protected Stack fEntityStack = new Stack<>(); >> >> Some of the "cleanup" in the webrev were reversing <> with added type >> argument(s), that would be unnecessary. > I think I did that on all the places that I touched now. > >> For the obsolete Vector, it would be good to replace it with ArrayList. > There are so many Vector instances in this code that it's out of scope for me now to touch them all... > >> Copyright year "2016" in XSAttributeChecker needs to be "2016," or else >> the script would complain. > For that one I thought that it would look like "2015,2016" if the initial copyright was done in 2015. But I changed it as you suggested. > >> In XSDHandler, note that there's another call to get SECURITY_MANAGER >> towards the end of the reset method (at 3572 in your new version) that >> may be removed. You may also consolidate most of those >> setFeature/Property statements in one try-catch block if you want. > OK, I did some further refacturing here. However, I've left the try/catch blocks as I think it's the intention to silently catch some exceptions but then try to go on with the next property. > >> For the test, the @bug tag is still desirable in the general comment >> section. For now, it's @bug 8149915. As we add more test to the class in >> the future, we'd then append more bug ids, @bug 8149915, xxxxxxx. > Done. I also left the @bug in the comments for the test method. > >>> I'm also wondering why some files have a copyright header like this, e.g. >> src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11D >> TDScannerImpl.java: >>> /* >>> * reserved comment block >>> * DO NOT REMOVE OR ALTER! >>> */ >>> >>> Shouldn't there be the standard copyright header or is there some reason >> behind it? >> >> The block is used by the licensing script to swap with license >> information required by licensees. For code that came from Xerces, we >> should keep the block as is with the Apache License. But when we make >> changes, we're required to put in the copyright header. Since the >> changes you made are cleanup / removing unused fields, it's okay to keep >> the block as is. > OK :) > > Thanks again for reviewing. > > All the Best > Christoph > > From christoph.langer at sap.com Thu Feb 18 19:33:39 2016 From: christoph.langer at sap.com (Langer, Christoph) Date: Thu, 18 Feb 2016 19:33:39 +0000 Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE In-Reply-To: <56C60B19.3040508@oracle.com> References: <56C38060.2030703@oracle.com> <80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap> <56C3AF17.6050103@oracle.com> <56C4D342.9070705@oracle.com> <56c7676f64ad49bd9dc9cf8cf1ed617f@DEWDFE13DE11.global.corp.sap> <56C60B19.3040508@oracle.com> Message-ID: <8f47645d4d074224bf30c08237df7cdf@DEWDFE13DE11.global.corp.sap> Hi Joe, I fixed the copyright in http://cr.openjdk.java.net/~clanger/webrevs/8149915.3/. So I think I'm done. Can you review and push the change then? Thanks Christoph -----Original Message----- From: huizhe wang [mailto:huizhe.wang at oracle.com] Sent: Donnerstag, 18. Februar 2016 19:19 To: Langer, Christoph Cc: core-libs-dev at openjdk.java.net Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE On 2/18/2016 5:15 AM, Langer, Christoph wrote: > Hi Joe, > > here is the next version: > http://cr.openjdk.java.net/~clanger/webrevs/8149915.2/ Looks good! Thanks. Copyright year in XSAttributeChecker.java is still incorrect. I meant to say the year "2016" needs to have a comma, so in this case, it needs to be "2015, 2016, " rather than "2015, 2016". Best, Joe > > Hopefully the final one :-) > >> Yes, it's good to have some cleanups. For the generic initialization, >> it's preferable to have no redundant type arguments, for example, >> instead of: >> >> protected Stack fEntityStack = new Stack(); >> >> do: >> protected Stack fEntityStack = new Stack<>(); >> >> Some of the "cleanup" in the webrev were reversing <> with added type >> argument(s), that would be unnecessary. > I think I did that on all the places that I touched now. > >> For the obsolete Vector, it would be good to replace it with ArrayList. > There are so many Vector instances in this code that it's out of scope for me now to touch them all... > >> Copyright year "2016" in XSAttributeChecker needs to be "2016," or else >> the script would complain. > For that one I thought that it would look like "2015,2016" if the initial copyright was done in 2015. But I changed it as you suggested. > >> In XSDHandler, note that there's another call to get SECURITY_MANAGER >> towards the end of the reset method (at 3572 in your new version) that >> may be removed. You may also consolidate most of those >> setFeature/Property statements in one try-catch block if you want. > OK, I did some further refacturing here. However, I've left the try/catch blocks as I think it's the intention to silently catch some exceptions but then try to go on with the next property. > >> For the test, the @bug tag is still desirable in the general comment >> section. For now, it's @bug 8149915. As we add more test to the class in >> the future, we'd then append more bug ids, @bug 8149915, xxxxxxx. > Done. I also left the @bug in the comments for the test method. > >>> I'm also wondering why some files have a copyright header like this, e.g. >> src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11D >> TDScannerImpl.java: >>> /* >>> * reserved comment block >>> * DO NOT REMOVE OR ALTER! >>> */ >>> >>> Shouldn't there be the standard copyright header or is there some reason >> behind it? >> >> The block is used by the licensing script to swap with license >> information required by licensees. For code that came from Xerces, we >> should keep the block as is with the Apache License. But when we make >> changes, we're required to put in the copyright header. Since the >> changes you made are cleanup / removing unused fields, it's okay to keep >> the block as is. > OK :) > > Thanks again for reviewing. > > All the Best > Christoph > > From Roger.Riggs at Oracle.com Thu Feb 18 21:13:53 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 18 Feb 2016 16:13:53 -0500 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <56C43817.7060805@gmail.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> <56C43817.7060805@gmail.com> Message-ID: <56C63411.9090206@Oracle.com> Hi Peter, Nice to see this improvement. I would support adding the drainQueue method to java.lang.ref.Cleaner; it provides a mechanism to share the cleanup load (as in the case of Direct buffers) that may be useful to other use cases. It is preferable to hacking in to the CleanerImpl. Making CleanerImpl implement Runnable is ok as long as CleanerImpl is limited to *only* be the implementation of Cleaner and is not expected to be used directly even internal to the base module. The conversions of lambdas to classes is necessary, unfortunately. The names of the Cleaner threads could use the threadId instead of introducing a counter, but probably not a big difference either way. The numbers are not going to be significant. Roger On 2/17/2016 4:06 AM, Peter Levart wrote: > Hi Kim, > > Thanks for looking into this. Answers inline... > > On 02/17/2016 01:20 AM, Kim Barrett wrote: >>> On Feb 16, 2016, at 11:15 AM, Peter Levart >>> wrote: >>> >>> Hello everybody, >>> >>> Thanks for looking into this and for all your comments. Now that it >>> seems we have a consensus that replacing internal Cleaner with >>> public Cleaner is not a wrong idea, I created an issue for it [1] so >>> that we may officially review the implementation. I also created an >>> updated webrev [2] which fixes some comments in usages that were not >>> correct any more after the change. I also took the liberty to modify >>> the CleanerImpl.InnocuousThreadFactory to give names to threads in >>> accordance to the style used in some other thread factories >>> (Timer-0, Timer-1, ..., Cleaner-0, Cleaner-1, ..., etc.). This gives >>> the thread of internal Cleaner instance, which is now constructed as >>> part of the boot-up sequence, a stable name: "Cleaner-0". >>> >>> If you feel that internal Cleaner instance should have a Thread with >>> MAX_PRIORITY, I can incorporate that too. >>> >>> [1] https://bugs.openjdk.java.net/browse/JDK-8149925 >>> [2] >>> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.03/ >>> >>> I re-ran the java/lang/ref and java/nio tests and all pass except >>> two ignored tests: >> I think some of the existing uses of sun.misc.Cleaner were really just >> a matter of convenience and the previous lack of >> java.lang.ref.Cleaner. An example of this that I have personal >> knowledge of the history for is the CallSite cleanup in >> MethodHandleNatives. It's good to see those converted. >> >> As for the others, if nobody wants to defend the special handling by >> the reference processing thread, I'm certainly happy to see it >> removed. However, I recall Roger saying there were existing tests that >> failed when certain uses of sun.misc.Cleaner were replaced with >> java.lang.ref.Cleaner. I don't remember the details, but maybe Roger >> does. > > If the failing test was the following: > > java/nio/Buffer/DirectBufferAllocTest.java > > ...then it has been taken care of (see Bits.java). All java/lang/ref > and java/nio tests pass on my PC. Are there any internal Oracle tests > that fail? > >> >> ------------------------------------------------------------------------------ >> >> src/java.base/share/classes/java/lang/ref/Reference.java >> >> After removal of the special handling of removing Cleaner objects from >> the pending list, do we still need to catch OOMEs from the pending >> list? If not sure, feel free to defer that in another RFE for cleanup. > > As you have already noticed it is still possible for a lock.wait() to > throw OOME because of not being able to allocate InterruptedException. > > The other place where OOME could still be thrown (and has not been > fixed yet) is from sun.misc.Cleaner.clean() special handling. I even > constructed a reproducer for it (see the last comment in JDK-8066859). > So removing special handling of sun.misc.Cleaner would finally put the > JDK-8066859 to the rest. > >> >> ------------------------------------------------------------------------------ >> >> src/java.base/share/classes/jdk/internal/ref/CleanerImpl.java >> >> I don't understand why CleanerImpl needs to be changed to public in >> order to provide access to the new drainQueue. Wouldn't it be better >> to add Cleaner.drainQueue? > > An interesting idea. But I don't know if such functionality is > generally useful enough to commit to it in a public API. > > I have another idea where java.lang.ref.Cleaner would use an Executor > instead of ThreadFactory. The default would be an instance of a > single-threaded executor per Cleaner instance, but if user passed in a > ForkJoinPool, he could use it's method ForkJoinPool.awaitQuiescence() > to help the executor's thread(s) do the cleaning. > >> >> Some of the other changes here don't seem related to the problem at >> hand. Are these proposed miscellaneous cleanups? I'm specifically >> looking at the CleanerCleanable class. And I'm not sure why the >> thread's constructor argument is being changed, requiring CleanerImpl >> to implement Runnable. > > One thing that this change unfortunately requires is to get rid of > lambdas and method references in the implementation and dependencies > of java.land.ref.Cleaner API, because it gets used early in the > boot-up sequence when java.lang.invoke infrastructure is not ready yet. > > The alternative to CleanerCleanable is a no-op Runnable implementation > passed to PhantomCleanableRef constructor. I opted for a subclass of > abstract PhantomCleanable class with an empty performCleanup() method. > Both approaches require a new class, but the CleanerCleanable is a > more direct way to express the intent. > > Making CleanerImpl implement Runnable and consequently making its > run() method public is also just a way to avoid introducing another > anonymous inner class as an adapter between Runnable.run() and > CleanerImpl.run(). CleanerImpl is in an internal concealed package, so > there's no danger of abuse. But I can revert it back to using an > anonymous inner adapter class (as was done in webrev.02) if desired or > I can just add a comment to public CleanerImpl.run() to warn against > it's direct use. > >> >> ------------------------------------------------------------------------------ >> >> src/java.base/share/classes/sun/nio/fs/NativeBuffer.java >> 76 Cleaner.Cleanable cleanable() { >> 77 return cleanable; >> 78 } >> >> [pre-existing, but if we're changing things anyway...] >> >> I'm kind of surprised by an accessor function (both here and in the >> original code) rather than a cleanup function. Is there a use case >> for anything other than buffer.cleanable().clean()? > > It can't be, since both old Cleaner and new Cleanable have only got a > single method - clean(). > >> >> Similarly for the DirectBuffer interface. > > This one is a critical method, used by various 3rd party softwares. In > order to make it less painful to support the change from > sun.misc.Cleaner to java.lang.ref.Cleaner.Cleanable, I think the > method should retain its name and semantics - it only changes it's > return type from a public type to another public type that both > contain a method with same signature - clean(). This way one can > construct a simple reflective adapter that is compatible with both JDK > 8- and JDK 9+. > > Regards, Peter > From iris.clark at oracle.com Thu Feb 18 21:37:08 2016 From: iris.clark at oracle.com (Iris Clark) Date: Thu, 18 Feb 2016 13:37:08 -0800 (PST) Subject: RFR [9] 8150163: JarFileSystem support for MRJARs should use the JDK specific Version API In-Reply-To: <3B35D44C-A88F-4725-8782-F5CF7CA84E35@oracle.com> References: <3B35D44C-A88F-4725-8782-F5CF7CA84E35@oracle.com> Message-ID: <23c91a97-f90f-44c3-bc20-62ad30551501@default> Hi, Chris. I think this change looks fine. I've added a note to JDK-8144062 (module for jdk.Version), indicating that parts of this changeset may need to be updated depending on where jdk.Version lands. Thanks, iris -----Original Message----- From: Chris Hegarty Sent: Thursday, February 18, 2016 4:56 AM To: core-libs-dev; Steve Drach Subject: RFR [9] 8150163: JarFileSystem support for MRJARs should use the JDK specific Version API To support MRJARs, currently JarFileSystem uses sun.misc.Version to retrieve the major version. It should be updated to use the new JDK specific Version API. http://cr.openjdk.java.net/~chegar/8150163/ https://bugs.openjdk.java.net/browse/JDK-8150163 -Chris. From joe.darcy at oracle.com Fri Feb 19 00:48:21 2016 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 18 Feb 2016 16:48:21 -0800 Subject: JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014 Message-ID: <56C66655.9000009@oracle.com> Hello, Right now, we are seeing some cross-platform intermittent failures of java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java as noted in bug JDK-8150014. Until that test instability is resolved, the test should be demoted from tier 1 to tier 2. Please see the patch below. (Circumstantial evidence points to the changes in JDK-8145485 being involved.) -Joe diff -r 3973fe856db2 test/TEST.groups --- a/test/TEST.groups Wed Feb 17 12:47:35 2016 -0800 +++ b/test/TEST.groups Thu Feb 18 16:44:40 2016 -0800 @@ -29,6 +29,7 @@ :jdk_lang \ -java/lang/ProcessHandle/TreeTest.java \ -java/util/zip/TestLocalTime.java \ + -java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ :jdk_util \ -java/util/WeakHashMap/GCDuringIteration.java \ -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ @@ -39,6 +40,7 @@ :jdk_math tier2 = \ + java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ java/lang/ProcessHandle/TreeTest.java \ java/util/zip/TestLocalTime.java \ java/util/WeakHashMap/GCDuringIteration.java \ From iris.clark at oracle.com Fri Feb 19 03:10:58 2016 From: iris.clark at oracle.com (Iris Clark) Date: Thu, 18 Feb 2016 19:10:58 -0800 (PST) Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: <56C57763.4050907@oracle.com> References: <56A94C8C.9050306@oracle.com> <56C4E46A.9050702@oracle.com> <56C57763.4050907@oracle.com> Message-ID: Hi, Alan and Rachel. Here's a proposal for the new text for JEP 223: --- [ insert between existing sections "@since..." and "Mercurial..."] JNI Version The [JNI Specification][JNISpec] defines a constant representing the JNI version number. The constant will drop the initial "1" and truncate trailing zeros. Thus, for JDK 9, the constant will be 'JNI_VERSION_9'. [JNISpec]: http://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#version_information --- What bugid should be used for this CCC request? What is the procedure for updating the JNI Spec itself? Thanks, iris -----Original Message----- From: Alan Bateman Sent: Wednesday, February 17, 2016 11:49 PM To: Rachel Protacio; hotspot-runtime-dev at openjdk.java.net; core-libs-dev at openjdk.java.net; Iris Clark Subject: Re: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 On 17/02/2016 21:21, Rachel Protacio wrote: > Hello, everyone, > > We are moving forward with "JNI_VERSION_9". If it later turns out that > it should be "9_0", we will file a separate bug, but the plain 9 is > most likely. There is currently a compatibility request in the > approval process. > > Here is the updated code, which builds properly and still prints the > correct version from my own private test. > hotspot repo webrev: > http://cr.openjdk.java.net/~rprotacio/JNI_hotspot.01/ > jdk repo webrev: http://cr.openjdk.java.net/~rprotacio/JNI_jdk.01/ This looks good to me. Iris - I assume you'll add a section to JEP 223 for this, it seems like the right place to capture this. -Alan From shilpi.rastogi at oracle.com Fri Feb 19 05:32:49 2016 From: shilpi.rastogi at oracle.com (shilpi.rastogi at oracle.com) Date: Fri, 19 Feb 2016 11:02:49 +0530 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> <012401d16a57$93deb480$bb9c1d80$@apache.org> <56C5DB53.1020504@oracle.com> Message-ID: <56C6A901.5090706@oracle.com> Hi All, Please see the updated webrev http://cr.openjdk.java.net/~srastogi/8144931/webrev.03/ Regards, Shilpi On 2/18/2016 8:33 PM, Paul Sandoz wrote: >> On 18 Feb 2016, at 15:55, Vladimir Ivanov wrote: >> >>> P.S.: You could improve the assertion with the Type class, so it would also fail on otherwise broken descriptor strings (missing "L" or missing ";"): >> I like how it shapes out with Type. Thanks, Uwe! >> > Me too! > > Paul. > >> Best regards, >> Vladimir Ivanov >> >>> static boolean checkClassName(String cn) { >>> Type tp = Type.getType(cn); >>> // additional sanity so only valid "L;" descriptors work >>> if (tp.getSort() != Type.OBJECT) { >>> return false; >>> } >>> try { >>> Class c = Class.forName(tp.getClassName(), false, null); >>> return true; >>> } catch (ClassNotFoundException e) { >>> return false; >>> } >>> } >>> >>> ----- >>> Uwe Schindler >>> uschindler at apache.org >>> ASF Member, Apache Lucene PMC / Committer >>> Bremen, Germany >>> http://lucene.apache.org/ From martinrb at google.com Fri Feb 19 06:18:35 2016 From: martinrb at google.com (Martin Buchholz) Date: Thu, 18 Feb 2016 22:18:35 -0800 Subject: JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014 In-Reply-To: <56C66655.9000009@oracle.com> References: <56C66655.9000009@oracle.com> Message-ID: It's easy enough to get the test to pass by delambdafication. But we may not want to fix the test because it's not clear whether it's a jdk lambda bug or not, and we don't sweep bugs under the rug. Maybe instead of a "tier2" bucket we really want a "test found a bug that's not yet fixed" bucket? On Thu, Feb 18, 2016 at 4:48 PM, joe darcy wrote: > Hello, > > Right now, we are seeing some cross-platform intermittent failures of > > java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java > > as noted in bug JDK-8150014. Until that test instability is resolved, the > test should be demoted from tier 1 to tier 2. > > Please see the patch below. > > (Circumstantial evidence points to the changes in JDK-8145485 being > involved.) > > -Joe > > diff -r 3973fe856db2 test/TEST.groups > --- a/test/TEST.groups Wed Feb 17 12:47:35 2016 -0800 > +++ b/test/TEST.groups Thu Feb 18 16:44:40 2016 -0800 > @@ -29,6 +29,7 @@ > :jdk_lang \ > -java/lang/ProcessHandle/TreeTest.java \ > -java/util/zip/TestLocalTime.java \ > + -java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ > :jdk_util \ > -java/util/WeakHashMap/GCDuringIteration.java \ > -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ > @@ -39,6 +40,7 @@ > :jdk_math > > tier2 = \ > + java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ > java/lang/ProcessHandle/TreeTest.java \ > java/util/zip/TestLocalTime.java \ > java/util/WeakHashMap/GCDuringIteration.java \ > From joe.darcy at oracle.com Fri Feb 19 06:37:03 2016 From: joe.darcy at oracle.com (joe darcy) Date: Thu, 18 Feb 2016 22:37:03 -0800 Subject: JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014 In-Reply-To: References: <56C66655.9000009@oracle.com> Message-ID: <56C6B80F.2090206@oracle.com> Hi Martin, On 2/18/2016 10:18 PM, Martin Buchholz wrote: > It's easy enough to get the test to pass by delambdafication. > > But we may not want to fix the test because it's not clear whether > it's a jdk lambda bug or not, and we don't sweep bugs under the rug. > > Maybe instead of a "tier2" bucket we really want a "test found a bug > that's not yet fixed" bucket? Well, we have a bug for the test failure in JDK-8150014 and now this bug to demote the test from tier 1 to tier 2. The planned snapshot for the next promoted build is next Monday. I don't think we should knowingly let an intermittent tier 1 test failure escape to a promoted build. Alternatives to achieve that goal include: 1) Demoting the test to tier 2 temporarily (If the cause of the problem is known, this is less helpful than if more data needed to be gathered.) 2) Putting the test on the problem list so it doesn't get run at all. 3) Changing the test so that is passes (delambdafication as you've suggested). 4) Changing / reverting the other libs code which triggered this failure. The closest approach to "test found a bug that's not yet fixed" is putting the test on the problem list, option 2). I'm happy to do either 1) or 2) under JDK-8150206 as an investigation into 3) or 4) proceeds over a longer period of time. I just don't think we should leave the test in its current flaky state for the next promotion. Thanks, -Joe > > On Thu, Feb 18, 2016 at 4:48 PM, joe darcy wrote: >> Hello, >> >> Right now, we are seeing some cross-platform intermittent failures of >> >> java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java >> >> as noted in bug JDK-8150014. Until that test instability is resolved, the >> test should be demoted from tier 1 to tier 2. >> >> Please see the patch below. >> >> (Circumstantial evidence points to the changes in JDK-8145485 being >> involved.) >> >> -Joe >> >> diff -r 3973fe856db2 test/TEST.groups >> --- a/test/TEST.groups Wed Feb 17 12:47:35 2016 -0800 >> +++ b/test/TEST.groups Thu Feb 18 16:44:40 2016 -0800 >> @@ -29,6 +29,7 @@ >> :jdk_lang \ >> -java/lang/ProcessHandle/TreeTest.java \ >> -java/util/zip/TestLocalTime.java \ >> + -java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >> :jdk_util \ >> -java/util/WeakHashMap/GCDuringIteration.java \ >> -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ >> @@ -39,6 +40,7 @@ >> :jdk_math >> >> tier2 = \ >> + java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >> java/lang/ProcessHandle/TreeTest.java \ >> java/util/zip/TestLocalTime.java \ >> java/util/WeakHashMap/GCDuringIteration.java \ >> From david.holmes at oracle.com Fri Feb 19 06:53:10 2016 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Feb 2016 16:53:10 +1000 Subject: RFR 9: 8149750 Decouple sun.misc.Signal from the base module In-Reply-To: <56C52451.6040108@oracle.com> References: <56BE100F.9060604@Oracle.com> <56BE1AC4.2040405@Oracle.com> <56C15A52.1050403@oracle.com> <56C396B7.8000803@Oracle.com> <56C4306C.6060206@oracle.com> <56C49671.3060107@Oracle.com> <56C52451.6040108@oracle.com> Message-ID: <56C6BBD6.5030308@oracle.com> On 18/02/2016 11:54 AM, David Holmes wrote: > On 18/02/2016 1:49 AM, Roger Riggs wrote: >> Hi David, >> >> On 2/17/2016 3:33 AM, David Holmes wrote: >>> Hi Roger, >>> >>> On 17/02/2016 7:37 AM, Roger Riggs wrote: >>>> Hi David, >>>> >>>> Webrev updated in place: >>>> http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ >>> >>> Thanks for those tweaks. >>> >>> I'm still perplexed by the test. We established that for USR2, PIPE >>> and XFSZ the handler is not actually invoked - yet you are testing >>> that case. ?? >> I'll have to dig a bit more. >> When that test is run, the SignalHandler does get called with the >> expected signal. >> The test succeeds on Linux with jdk 8 and jdk 9. > > In my local testing (with the original java.util.Signal changes) raise() > throws IllegalStateException for USR2, PIPE and XFSZ (even though > register succeeded). > > If I can get the time I'll try applying these exact changes and testing > them. And everything worked as you said. I'm now extremely perplexed. David ----- > Thanks, > David > >> Roger >> >>> >>> David >>> >>>> On 2/14/2016 11:55 PM, David Holmes wrote: >>>>> Hi Roger, >>>>> >>>>> A few mostly minor comments ... >>>>> >>>>> On 13/02/2016 3:47 AM, Roger Riggs wrote: >>>>>> Please review moving the functionality of sun.misc.Signal to >>>>>> jdk.internal.misc and >>>>>> creating a wrapper sun.misc.Signal for existing external uses. >>>>>> >>>>>> +++ This time including the hotspot changes to update the target of >>>>>> the >>>>>> upcalls. >>>>>> >>>>>> A new replacement API will be considered separately. >>>>>> >>>>>> The update includes a test that passes with or without the changes. >>>>>> (Except for an NPE instead of SEGV if null is passed). >>>>>> >>>>>> Webrev: >>>>>> jdk: http://cr.openjdk.java.net/~rriggs/webrev-signal-9149750/ >>>>> >>>>> I take it we don't care about reorder files any more? >>>> There does not seem to be any particular guidance about how/when to use >>>> them on Solaris. >>>> So it seemed as reasonable to remove the entries as to update them. >>>>> >>>>> --- >>>>> >>>>> src/java.base/share/classes/sun/misc/Signal.java >>>>> >>>>> The @since should not be changed to 9. You only changed the >>>>> implementation not the API. Conversely the renamed class should >>>>> arguably be @since 9 and not @since 1.2. But it is wrong to say >>>>> sun.misc.Signal is @since 9. >>>> The new sun.misc.Signal is a wrapper around the original implementation >>>> now in jdk.internal.misc. >>>> I had modified the mercurial commands to rename sun.misc.Signal to >>>> jdk.internal.misc to preserve the history >>>> of the implementation. >>>> >>>> I will correct sun.misc.Signal to retain the original @since and use >>>> @since 9 for the jdk.internal.misc >>>>> >>>>> The *Handler.of method name reads strangely to me - "for" would be >>>>> better but I presume that is not allowed. >>>> The xx.of () pattern has been used recently. >>>>> >>>>> --- >>>>> >>>>> src/java.base/share/classes/jdk/internal/misc/Signal.java >>>>> >>>>> Not sure I understand the role of NativeSignalHandler any more - given >>>>> it can't actually be used ?? >>>> They are used to retain the previous handler and can be used to restore >>>> that handler. >>>> The ability to invoke the handler directly was removed as a >>>> simplification. >>>>> >>>>> --- >>>>> >>>>> test/sun/misc/SunMiscSignalTest.java >>>>> >>>>> Based on our email discussion this test can not be testing that the >>>>> handler actually gets invoked - as we know it does not in some cases. >>>>> I have doubts that sun.misc.Signal ever intended to support all the >>>>> signals you are testing. >>>> There were no tests for sun.misc.Signal. I created the tests based on >>>> the current implementations. >>>> They can be updated to reflect current support for handle, raise, and >>>> whether a signal handler is called >>>> and of course -Xrs. It should allow detection of unintended changes in >>>> behavior. >>>> >>>>> >>>>>> hotspot: http://cr.openjdk.java.net/~rriggs/webrev-hs-signal-9149750/ >>>>> >>>>> Hotspot changes are fine. >>>> >>>> Thanks, Roger >>>>> >>>>> Thanks, >>>>> David >>>>> ----- >>>>> >>>>>> Issue: >>>>>> https://bugs.openjdk.java.net/browse/JDK-8149750 >>>>>> >>>>>> Thanks, Roger >>>>>> >>>>>> JEP 260: https://bugs.openjdk.java.net/browse/JDK-8132928 >>>>>> >>>>>> >>>> >> From Alan.Bateman at oracle.com Fri Feb 19 08:00:25 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Feb 2016 08:00:25 +0000 Subject: JNI VERSION CHANGE: RFR: 8145098: JNI GetVersion should return JNI_VERSION_9 In-Reply-To: References: <56A94C8C.9050306@oracle.com> <56C4E46A.9050702@oracle.com> <56C57763.4050907@oracle.com> Message-ID: <56C6CB99.6080609@oracle.com> On 19/02/2016 03:10, Iris Clark wrote: > Hi, Alan and Rachel. > > Here's a proposal for the new text for JEP 223: > > --- > > [ insert between existing sections "@since..." and "Mercurial..."] > > JNI Version > > The [JNI Specification][JNISpec] defines a constant representing the JNI version number. The constant will drop the initial "1" and truncate trailing zeros. Thus, for JDK 9, the constant will be 'JNI_VERSION_9'. > > [JNISpec]: > http://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#version_information > > --- > > What bugid should be used for this CCC request? What is the procedure for updating the JNI Spec itself? > > This wording looks okay to me and I assume it could go on to say that the constant will also be proposed for Java SE 9. JDK-8145098 is the issue that is being used to update the JNI version. While we're at it, could I interest you in a 2-for-1 deal where JEP 223 also captures the update to the JVM TI version? We've added JVMTI_VERSION_9 in the jake forest and it would be nice to get agreement on aligning the JVM TI version number (and also get it into JDK 9 in advance of the module system). -Alan. From paul.sandoz at oracle.com Fri Feb 19 09:40:42 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 19 Feb 2016 10:40:42 +0100 Subject: JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014 In-Reply-To: References: <56C66655.9000009@oracle.com> Message-ID: <997ABE4C-F6D5-4F91-A32C-1D60B7913158@oracle.com> Hi Martin, Your patch to TLR and SR looks good: http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/unlambdafyBootstrap/jsr166-jdk9-integration- Sorry, i should of caught this in review, did notice the change to use lambdas but i forgot that this might cause issues at VM startup. ConcurrentHashMap is used by infrastructure of j.l.invoke. CHM cannot itself leverage or depend on stuff that leverages lambdas otherwise it may result in a circular dependency at VM startup. In this case the test tickles the code paths that results in the initialization of ThreadLocalRandom by CHM during startup which explains why it was yet to be observed in other cases. Paul. > On 19 Feb 2016, at 07:18, Martin Buchholz wrote: > > It's easy enough to get the test to pass by delambdafication. > > But we may not want to fix the test because it's not clear whether > it's a jdk lambda bug or not, and we don't sweep bugs under the rug. > > Maybe instead of a "tier2" bucket we really want a "test found a bug > that's not yet fixed" bucket? > > On Thu, Feb 18, 2016 at 4:48 PM, joe darcy wrote: >> Hello, >> >> Right now, we are seeing some cross-platform intermittent failures of >> >> java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java >> >> as noted in bug JDK-8150014. Until that test instability is resolved, the >> test should be demoted from tier 1 to tier 2. >> >> Please see the patch below. >> >> (Circumstantial evidence points to the changes in JDK-8145485 being >> involved.) >> >> -Joe >> >> diff -r 3973fe856db2 test/TEST.groups >> --- a/test/TEST.groups Wed Feb 17 12:47:35 2016 -0800 >> +++ b/test/TEST.groups Thu Feb 18 16:44:40 2016 -0800 >> @@ -29,6 +29,7 @@ >> :jdk_lang \ >> -java/lang/ProcessHandle/TreeTest.java \ >> -java/util/zip/TestLocalTime.java \ >> + -java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >> :jdk_util \ >> -java/util/WeakHashMap/GCDuringIteration.java \ >> -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ >> @@ -39,6 +40,7 @@ >> :jdk_math >> >> tier2 = \ >> + java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >> java/lang/ProcessHandle/TreeTest.java \ >> java/util/zip/TestLocalTime.java \ >> java/util/WeakHashMap/GCDuringIteration.java \ >> From vladimir.x.ivanov at oracle.com Fri Feb 19 10:27:20 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 19 Feb 2016 13:27:20 +0300 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C6A901.5090706@oracle.com> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> <012401d16a57$93deb480$bb9c1d80$@apache.org> <56C5DB53.1020504@oracle.com> <56C6A901.5090706@oracle.com> Message-ID: <56C6EE08.5030201@oracle.com> Looks good. Best regards, Vladimir Ivanov On 2/19/16 8:32 AM, shilpi.rastogi at oracle.com wrote: > Please see the updated webrev > http://cr.openjdk.java.net/~srastogi/8144931/webrev.03/ From paul.sandoz at oracle.com Fri Feb 19 11:37:29 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 19 Feb 2016 12:37:29 +0100 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: <56C6A901.5090706@oracle.com> References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> <012401d16a57$93deb480$bb9c1d80$@apache.org> <56C5DB53.1020504@oracle.com> <56C6A901.5090706@oracle.com> Message-ID: > On 19 Feb 2016, at 06:32, shilpi.rastogi at oracle.com wrote: > > Hi All, > > Please see the updated webrev > http://cr.openjdk.java.net/~srastogi/8144931/webrev.03/ > +1 Paul. From aleksey.shipilev at oracle.com Fri Feb 19 11:55:34 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 19 Feb 2016 14:55:34 +0300 Subject: RFR (S) 8150180: String.value contents should be trusted Message-ID: <56C702B6.5010903@oracle.com> Hi, Please review a simple performance improvement in Strings: https://bugs.openjdk.java.net/browse/JDK-8150180 In short, we want VM to trust constant String contents, so that "Foo".charAt(0) is folded. As far as current Hotspot goes, this is only achievable with @Stable -- we need to trust the array contents: http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/ This, however, steps into the compiler bug caused by StringUTF16.getChar intrinsic producing a mismatched load, that is then folded incorrectly. So we instead rely on Java level folding in cases like these: http://cr.openjdk.java.net/~shade/8150180/webrev.hs.01/ ...and it works: http://cr.openjdk.java.net/~shade/8150180/notes.txt While VM change looks like a workaround, Vladimir I. and me concluded that @Stable folding code should just reject folding mismatched loads to begin with. So, getChar intrinsic change is the actual fix. Vladimir I. ought to add more assertions in @Stable folding code separately: https://bugs.openjdk.java.net/browse/JDK-8150186 Since this issue might trigger compiler bugs, I would like to push through hs-comp to pass compiler nightlies. Cheers, -Aleksey From vladimir.x.ivanov at oracle.com Fri Feb 19 12:42:18 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 19 Feb 2016 15:42:18 +0300 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C702B6.5010903@oracle.com> References: <56C702B6.5010903@oracle.com> Message-ID: <56C70DAA.6000806@oracle.com> > http://cr.openjdk.java.net/~shade/8150180/webrev.hs.01/ Why don't you check that index is also a constant? Otherwise, C2 can't constant fold the loads. + if (value->is_Con()) { + return false; + } Best regards, Vladimir Ivanov From peter.levart at gmail.com Fri Feb 19 14:05:12 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 Feb 2016 15:05:12 +0100 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C702B6.5010903@oracle.com> References: <56C702B6.5010903@oracle.com> Message-ID: <56C72118.6030304@gmail.com> Hi, Just a stupid question. Your comment in String: 140 * @implNote Note this field is not {@link Stable}, because we want 141 * LATIN1 (0) coder to fold too. {@link Stable} would not allow that, 142 * as it reserves the default value as "not initialized" value. 143 * Constant-folding this field is handled internally in VM. 144 */ 145 private final byte coder; Couldn't @Stable final instance fields constant-fold the default value too in general? I mean can't it be assumed that the final field has been set before JIT kicks in? Regards, Peter On 02/19/2016 12:55 PM, Aleksey Shipilev wrote: > Hi, > > Please review a simple performance improvement in Strings: > https://bugs.openjdk.java.net/browse/JDK-8150180 > > In short, we want VM to trust constant String contents, so that > "Foo".charAt(0) is folded. As far as current Hotspot goes, this is only > achievable with @Stable -- we need to trust the array contents: > http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/ > > This, however, steps into the compiler bug caused by StringUTF16.getChar > intrinsic producing a mismatched load, that is then folded incorrectly. > So we instead rely on Java level folding in cases like these: > http://cr.openjdk.java.net/~shade/8150180/webrev.hs.01/ > > ...and it works: > http://cr.openjdk.java.net/~shade/8150180/notes.txt > > While VM change looks like a workaround, Vladimir I. and me concluded > that @Stable folding code should just reject folding mismatched loads to > begin with. So, getChar intrinsic change is the actual fix. Vladimir I. > ought to add more assertions in @Stable folding code separately: > https://bugs.openjdk.java.net/browse/JDK-8150186 > > Since this issue might trigger compiler bugs, I would like to push > through hs-comp to pass compiler nightlies. > > Cheers, > -Aleksey > From aleksey.shipilev at oracle.com Fri Feb 19 14:07:57 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 19 Feb 2016 17:07:57 +0300 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C72118.6030304@gmail.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> Message-ID: <56C721BD.9010505@oracle.com> Hi Peter, On 02/19/2016 05:05 PM, Peter Levart wrote: > Your comment in String: > > 140 * @implNote Note this field is not {@link Stable}, because we > want > 141 * LATIN1 (0) coder to fold too. {@link Stable} would not allow > that, > 142 * as it reserves the default value as "not initialized" value. > 143 * Constant-folding this field is handled internally in VM. > 144 */ > 145 private final byte coder; > > > Couldn't @Stable final instance fields constant-fold the default value > too in general? I mean can't it be assumed that the final field has been > set before JIT kicks in? We've been over this multiple times, actually. @Stable is not designed to fold default value -- it is a part of its design to ignore final fields with default values. Because @Stable is used largely for lazy initialization, and default value is the important marker in field's lifecycles. Cheers, -Aleksey From aleksey.shipilev at oracle.com Fri Feb 19 14:17:43 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 19 Feb 2016 17:17:43 +0300 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C70DAA.6000806@oracle.com> References: <56C702B6.5010903@oracle.com> <56C70DAA.6000806@oracle.com> Message-ID: <56C72407.2070301@oracle.com> On 02/19/2016 03:42 PM, Vladimir Ivanov wrote: >> http://cr.openjdk.java.net/~shade/8150180/webrev.hs.01/ > > Why don't you check that index is also a constant? Otherwise, C2 can't > constant fold the loads. > > + if (value->is_Con()) { > + return false; > + } Right. Without that, constant-value + non-constant-index also bail unnecessarily. Actually, we should also bail only for getChar, but not putChar: http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/ http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/ Cheers, -Aleksey From peter.levart at gmail.com Fri Feb 19 14:36:36 2016 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 19 Feb 2016 15:36:36 +0100 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C721BD.9010505@oracle.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> <56C721BD.9010505@oracle.com> Message-ID: <56C72874.2070606@gmail.com> On 02/19/2016 03:07 PM, Aleksey Shipilev wrote: > Hi Peter, > > On 02/19/2016 05:05 PM, Peter Levart wrote: >> Your comment in String: >> >> 140 * @implNote Note this field is not {@link Stable}, because we >> want >> 141 * LATIN1 (0) coder to fold too. {@link Stable} would not allow >> that, >> 142 * as it reserves the default value as "not initialized" value. >> 143 * Constant-folding this field is handled internally in VM. >> 144 */ >> 145 private final byte coder; >> >> >> Couldn't @Stable final instance fields constant-fold the default value >> too in general? I mean can't it be assumed that the final field has been >> set before JIT kicks in? > We've been over this multiple times, actually. @Stable is not designed > to fold default value -- it is a part of its design to ignore final > fields with default values. Because @Stable is used largely for lazy > initialization, and default value is the important marker in field's > lifecycles. > > Cheers, > -Aleksey > Yes, I understand the general principle of @Stable design and the role of default value in @Stable fields. But given that: - instance final fields must be set in the constructor - the reference to the constructed object can not be assigned to a static final field until the constructor is finished ...the compiler could assume that even the default value is the final field's stable value, couldn't it? I understand that these are Java language constraints and not necessarily bytecode constraints. Also, while a static final field can only be assigned to reference an instance after the instance is fully constructed, this does not hold for @Stable fields in general, for example: class Foo { @Stable static Foo foo; @Stable final int coder; Foo() { foo = this; // outside code could already access Foo.foo.coder and constant fold it to 0 coder = 1; } } ...given that @Stable is an internal annotation, such usages could be advised against. It is generally bad practice to publish an object from within its constructor anyway. But never mind. This is just me thinking loud. Regards, Peter From vladimir.x.ivanov at oracle.com Fri Feb 19 14:42:27 2016 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 19 Feb 2016 17:42:27 +0300 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C72407.2070301@oracle.com> References: <56C702B6.5010903@oracle.com> <56C70DAA.6000806@oracle.com> <56C72407.2070301@oracle.com> Message-ID: <56C729D3.80404@oracle.com> > http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/ > http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/ Looks good. Best regards, Vladimir Ivanov From aleksey.shipilev at oracle.com Fri Feb 19 15:50:16 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Fri, 19 Feb 2016 18:50:16 +0300 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C72874.2070606@gmail.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> <56C721BD.9010505@oracle.com> <56C72874.2070606@gmail.com> Message-ID: <56C739B8.3020601@oracle.com> On 02/19/2016 05:36 PM, Peter Levart wrote: > Yes, I understand the general principle of @Stable design and the role > of default value in @Stable fields. But given that: > > But never mind. This is just me thinking loud. Sure. The answer to those questions about @Stable is, as always: it works like that; it was never supposed to do more than it does right now; the time spent dealing with @Stable is better spent dealing with generic problem of trusting final fields, without VM internal annotations. @Stable paves the way in VM internals for that, but it does not mean it should be catch-all feature. -Aleksey From iris.clark at oracle.com Fri Feb 19 17:40:24 2016 From: iris.clark at oracle.com (Iris Clark) Date: Fri, 19 Feb 2016 09:40:24 -0800 (PST) Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: References: Message-ID: <5b1a5288-3c75-4437-aa78-b63169615db7@default> Hi, Chris. > http://cr.openjdk.java.net/~chegar/8150162/ >From a versioning perspective, I think your changes are fine. Regarding the choice of package, I think a package-private class in java.lang is reasonable. Beyond initialization, I don't see any reason to use this class. The new jdk.Version API should be used in preference to these *{Major,Minor,Security,...}Version() methods. I'm not the code maintainer, but I'd consider deprecating them. Thanks, iris (not a Reviewer) -----Original Message----- From: Chris Hegarty Sent: Thursday, February 18, 2016 7:53 AM To: core-libs-dev; Iris Clark Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package sun.misc.Version is the core libraries part of a private interface with the JVM to query and set specific JVM version and capabilities, as well as being responsible for setting the system properties for "java.version", "java.runtime.version", and "java.runtime.name" ( which are generated during the build ). It is not a "Critical API", as defined by JEP 260, so should be moved out of sun.misc and placed into a more appropriate package where it can be encapsulated. Version is only used by j.l.System during initialization, so I've taken the liberty of moving it into java.lang as package-private. It could, however, be placed in jdk.internal.misc, if it is more generally useful. That said, most other usages should be able to use the JDK specific Version API. http://cr.openjdk.java.net/~chegar/8150162/ https://bugs.openjdk.java.net/browse/JDK-8150162 -Chris. P.S. 8150163 & 8150168 are in progress to replace unnecessary usages of sun.misc.Version with the JDK specific Version. From chris.hegarty at oracle.com Fri Feb 19 17:43:23 2016 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 19 Feb 2016 17:43:23 +0000 Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: <5b1a5288-3c75-4437-aa78-b63169615db7@default> References: <5b1a5288-3c75-4437-aa78-b63169615db7@default> Message-ID: On 19 Feb 2016, at 17:40, Iris Clark wrote: > Hi, Chris. > >> http://cr.openjdk.java.net/~chegar/8150162/ > > From a versioning perspective, I think your changes are fine. Thanks Iris. > Regarding the choice of package, I think a package-private class in java.lang is reasonable. Beyond initialization, I don't see any reason to use this class. The new jdk.Version API should be used in preference to these *{Major,Minor,Security,...}Version() methods. Yes, exactly. That was my thought too. > I'm not the code maintainer, but I'd consider deprecating them. Or just removing them. Thanks, -Chris > Thanks, > iris > (not a Reviewer) > > -----Original Message----- > From: Chris Hegarty > Sent: Thursday, February 18, 2016 7:53 AM > To: core-libs-dev; Iris Clark > Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package > > sun.misc.Version is the core libraries part of a private interface with the JVM to query and set specific JVM version and capabilities, as well as being responsible for setting the system properties for "java.version", "java.runtime.version", and "java.runtime.name" ( which are generated during the build ). > > It is not a "Critical API", as defined by JEP 260, so should be moved out of sun.misc and placed into a more appropriate package where it can be encapsulated. > > Version is only used by j.l.System during initialization, so I've taken the liberty of moving it into java.lang as package-private. It could, however, be placed in jdk.internal.misc, if it is more generally useful. That said, most other usages should be able to use the JDK specific Version API. > > http://cr.openjdk.java.net/~chegar/8150162/ > https://bugs.openjdk.java.net/browse/JDK-8150162 > > -Chris. > > P.S. 8150163 & 8150168 are in progress to replace unnecessary usages of sun.misc.Version with the JDK specific Version. From iris.clark at oracle.com Fri Feb 19 17:52:17 2016 From: iris.clark at oracle.com (Iris Clark) Date: Fri, 19 Feb 2016 09:52:17 -0800 (PST) Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: References: <5b1a5288-3c75-4437-aa78-b63169615db7@default> Message-ID: <66f6308b-60e1-4874-ae21-25c19f822f26@default> Hi, Chris. >> I'm not the code maintainer, but I'd consider deprecating them. > Or just removing them. Agreed. iris -----Original Message----- From: Chris Hegarty Sent: Friday, February 19, 2016 9:43 AM To: Iris Clark Cc: core-libs-dev Subject: Re: RFR [9] 8150162: Move sun.misc.Version to a truly internal package On 19 Feb 2016, at 17:40, Iris Clark wrote: > Hi, Chris. > >> http://cr.openjdk.java.net/~chegar/8150162/ > > From a versioning perspective, I think your changes are fine. Thanks Iris. > Regarding the choice of package, I think a package-private class in java.lang is reasonable. Beyond initialization, I don't see any reason to use this class. The new jdk.Version API should be used in preference to these *{Major,Minor,Security,...}Version() methods. Yes, exactly. That was my thought too. > I'm not the code maintainer, but I'd consider deprecating them. Or just removing them. Thanks, -Chris From christian.thalinger at oracle.com Fri Feb 19 17:57:07 2016 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 19 Feb 2016 07:57:07 -1000 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C72118.6030304@gmail.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> Message-ID: <7AB17C7E-B33A-4EA4-A04F-C77E8E0EA21B@oracle.com> > On Feb 19, 2016, at 4:05 AM, Peter Levart wrote: > > Hi, > > Just a stupid question. > > Your comment in String: > > 140 * @implNote Note this field is not {@link Stable}, because we want > 141 * LATIN1 (0) coder to fold too. {@link Stable} would not allow that, > 142 * as it reserves the default value as "not initialized" value. > 143 * Constant-folding this field is handled internally in VM. > 144 */ > 145 private final byte coder; Why don?t you change the values to: static final byte LATIN1 = 1; static final byte UTF16 = 2; ? > > > Couldn't @Stable final instance fields constant-fold the default value too in general? I mean can't it be assumed that the final field has been set before JIT kicks in? > > Regards, Peter > > On 02/19/2016 12:55 PM, Aleksey Shipilev wrote: >> Hi, >> >> Please review a simple performance improvement in Strings: >> https://bugs.openjdk.java.net/browse/JDK-8150180 >> >> In short, we want VM to trust constant String contents, so that >> "Foo".charAt(0) is folded. As far as current Hotspot goes, this is only >> achievable with @Stable -- we need to trust the array contents: >> http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/ >> >> This, however, steps into the compiler bug caused by StringUTF16.getChar >> intrinsic producing a mismatched load, that is then folded incorrectly. >> So we instead rely on Java level folding in cases like these: >> http://cr.openjdk.java.net/~shade/8150180/webrev.hs.01/ >> >> ...and it works: >> http://cr.openjdk.java.net/~shade/8150180/notes.txt >> >> While VM change looks like a workaround, Vladimir I. and me concluded >> that @Stable folding code should just reject folding mismatched loads to >> begin with. So, getChar intrinsic change is the actual fix. Vladimir I. >> ought to add more assertions in @Stable folding code separately: >> https://bugs.openjdk.java.net/browse/JDK-8150186 >> >> Since this issue might trigger compiler bugs, I would like to push >> through hs-comp to pass compiler nightlies. >> >> Cheers, >> -Aleksey >> > From Roger.Riggs at Oracle.com Fri Feb 19 18:01:58 2016 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 19 Feb 2016 13:01:58 -0500 Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: References: Message-ID: <56C75896.4050205@Oracle.com> Hi Chris, Looks fine. (As a package private class in java.lang) Roger On 2/18/2016 10:52 AM, Chris Hegarty wrote: > sun.misc.Version is the core libraries part of a private interface with the > JVM to query and set specific JVM version and capabilities, as well as > being responsible for setting the system properties for "java.version?, > "java.runtime.version", and "java.runtime.name" ( which are generated > during the build ). > > It is not a "Critical API", as defined by JEP 260, so should be moved > out of sun.misc and placed into a more appropriate package where it > can be encapsulated. > > Version is only used by j.l.System during initialization, so I?ve taken the > liberty of moving it into java.lang as package-private. It could, however, > be placed in jdk.internal.misc, if it is more generally useful. That said, > most other usages should be able to use the JDK specific Version API. > > http://cr.openjdk.java.net/~chegar/8150162/ > https://bugs.openjdk.java.net/browse/JDK-8150162 > > -Chris. > > P.S. 8150163 & 8150168 are in progress to replace unnecessary usages > of sun.misc.Version with the JDK specific Version. From Alan.Bateman at oracle.com Fri Feb 19 18:08:51 2016 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 19 Feb 2016 18:08:51 +0000 Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: References: Message-ID: <56C75A33.4060907@oracle.com> On 18/02/2016 15:52, Chris Hegarty wrote: > sun.misc.Version is the core libraries part of a private interface with the > JVM to query and set specific JVM version and capabilities, as well as > being responsible for setting the system properties for "java.version?, > "java.runtime.version", and "java.runtime.name" ( which are generated > during the build ). > > It is not a "Critical API", as defined by JEP 260, so should be moved > out of sun.misc and placed into a more appropriate package where it > can be encapsulated. > > Version is only used by j.l.System during initialization, so I?ve taken the > liberty of moving it into java.lang as package-private. It could, however, > be placed in jdk.internal.misc, if it is more generally useful. That said, > most other usages should be able to use the JDK specific Version API. > > http://cr.openjdk.java.net/~chegar/8150162/ > https://bugs.openjdk.java.net/browse/JDK-8150162 > This looks okay although I expect we will need to change VersionTest again soon to not use @run main/bootclasspath. -Alan From joe.darcy at oracle.com Fri Feb 19 18:44:59 2016 From: joe.darcy at oracle.com (joe darcy) Date: Fri, 19 Feb 2016 10:44:59 -0800 Subject: JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014 In-Reply-To: <997ABE4C-F6D5-4F91-A32C-1D60B7913158@oracle.com> References: <56C66655.9000009@oracle.com> <997ABE4C-F6D5-4F91-A32C-1D60B7913158@oracle.com> Message-ID: <56C762AB.1000108@oracle.com> Hello, I'm happy if Martin's delambdafication is the right solution. However, if that is not pushed by Monday Feb. 22, I'll want to go ahead and problem list the test. Thanks, -Joe On 2/19/2016 1:40 AM, Paul Sandoz wrote: > Hi Martin, > > Your patch to TLR and SR looks good: > > http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/unlambdafyBootstrap/jsr166-jdk9-integration- > > Sorry, i should of caught this in review, did notice the change to use lambdas but i forgot that this might cause issues at VM startup. > > ConcurrentHashMap is used by infrastructure of j.l.invoke. CHM cannot itself leverage or depend on stuff that leverages lambdas otherwise it may result in a circular dependency at VM startup. > > In this case the test tickles the code paths that results in the initialization of ThreadLocalRandom by CHM during startup which explains why it was yet to be observed in other cases. > > Paul. > >> On 19 Feb 2016, at 07:18, Martin Buchholz wrote: >> >> It's easy enough to get the test to pass by delambdafication. >> >> But we may not want to fix the test because it's not clear whether >> it's a jdk lambda bug or not, and we don't sweep bugs under the rug. >> >> Maybe instead of a "tier2" bucket we really want a "test found a bug >> that's not yet fixed" bucket? >> >> On Thu, Feb 18, 2016 at 4:48 PM, joe darcy wrote: >>> Hello, >>> >>> Right now, we are seeing some cross-platform intermittent failures of >>> >>> java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java >>> >>> as noted in bug JDK-8150014. Until that test instability is resolved, the >>> test should be demoted from tier 1 to tier 2. >>> >>> Please see the patch below. >>> >>> (Circumstantial evidence points to the changes in JDK-8145485 being >>> involved.) >>> >>> -Joe >>> >>> diff -r 3973fe856db2 test/TEST.groups >>> --- a/test/TEST.groups Wed Feb 17 12:47:35 2016 -0800 >>> +++ b/test/TEST.groups Thu Feb 18 16:44:40 2016 -0800 >>> @@ -29,6 +29,7 @@ >>> :jdk_lang \ >>> -java/lang/ProcessHandle/TreeTest.java \ >>> -java/util/zip/TestLocalTime.java \ >>> + -java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >>> :jdk_util \ >>> -java/util/WeakHashMap/GCDuringIteration.java \ >>> -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ >>> @@ -39,6 +40,7 @@ >>> :jdk_math >>> >>> tier2 = \ >>> + java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >>> java/lang/ProcessHandle/TreeTest.java \ >>> java/util/zip/TestLocalTime.java \ >>> java/util/WeakHashMap/GCDuringIteration.java \ >>> From mandy.chung at oracle.com Fri Feb 19 18:48:29 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 19 Feb 2016 10:48:29 -0800 Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: <66f6308b-60e1-4874-ae21-25c19f822f26@default> References: <5b1a5288-3c75-4437-aa78-b63169615db7@default> <66f6308b-60e1-4874-ae21-25c19f822f26@default> Message-ID: <90AE881D-5940-4BFF-91E6-7637AEB7D4EE@oracle.com> > On Feb 19, 2016, at 9:52 AM, Iris Clark wrote: > > Hi, Chris. > >>> I'm not the code maintainer, but I'd consider deprecating them. > >> Or just removing them. > > Agreed. Before Verona, sun.misc.Version was generated at build time and was intended to set up the system properties about ?java.version?, ?java.runtime.version?, ?java.runtime.name?. Did Verona change the logic such that this class is no longer needed? On the other hand, I agree that the static methods getting the version numbers should be removed as they are replaced by the new Version API. As for the webrev, moving to java.lang as a package-private type is fine. I suggest to rename it to avoid any confusion with the new Version API such as VersionProps (or something better)? Mandy From iris.clark at oracle.com Fri Feb 19 18:59:12 2016 From: iris.clark at oracle.com (Iris Clark) Date: Fri, 19 Feb 2016 10:59:12 -0800 (PST) Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: <90AE881D-5940-4BFF-91E6-7637AEB7D4EE@oracle.com> References: <5b1a5288-3c75-4437-aa78-b63169615db7@default> <66f6308b-60e1-4874-ae21-25c19f822f26@default> <90AE881D-5940-4BFF-91E6-7637AEB7D4EE@oracle.com> Message-ID: <431be29a-f350-4790-81bd-13365b505b0e@default> Hi, Mandy. > Before Verona, sun.misc.Version was generated at build time and was intended to set up the system properties about ?java.version?, ?java.runtime.version?, ?java.runtime.name?. Did Verona change the logic such that this class is no longer needed? Verona did not change logic for VM initialization, we just updated it to reflect differences in naming. The class is still needed to handle VM initialization and to print out version information (see lines 67-124 [0]). Recommendations for deprecation/removal were in reference to the static methods, *{Major,Minor,...}Version() beginning at new line 127 which appear to simply retrieve portions of the system properties. Thanks, Iris [0] http://cr.openjdk.java.net/~chegar/8150162/jdk/src/java.base/share/classes/java/lang/Version.java.template.html -----Original Message----- From: Mandy Chung Sent: Friday, February 19, 2016 10:48 AM To: Iris Clark Cc: Mandy Chung; Chris Hegarty; core-libs-dev Subject: Re: RFR [9] 8150162: Move sun.misc.Version to a truly internal package > On Feb 19, 2016, at 9:52 AM, Iris Clark wrote: > > Hi, Chris. > >>> I'm not the code maintainer, but I'd consider deprecating them. > >> Or just removing them. > > Agreed. Before Verona, sun.misc.Version was generated at build time and was intended to set up the system properties about ?java.version?, ?java.runtime.version?, ?java.runtime.name?. Did Verona change the logic such that this class is no longer needed? On the other hand, I agree that the static methods getting the version numbers should be removed as they are replaced by the new Version API. As for the webrev, moving to java.lang as a package-private type is fine. I suggest to rename it to avoid any confusion with the new Version API such as VersionProps (or something better)? Mandy From john.r.rose at oracle.com Fri Feb 19 19:03:53 2016 From: john.r.rose at oracle.com (John Rose) Date: Fri, 19 Feb 2016 11:03:53 -0800 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <7AB17C7E-B33A-4EA4-A04F-C77E8E0EA21B@oracle.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> <7AB17C7E-B33A-4EA4-A04F-C77E8E0EA21B@oracle.com> Message-ID: On Feb 19, 2016, at 9:57 AM, Christian Thalinger wrote: > > Why don?t you change the values to: > > static final byte LATIN1 = 1; > static final byte UTF16 = 2; Not sure what you are asking, but here's my take on why 0,1 is the (slight) winner. The values 0,1 are the log2 of the array element sizes 1,2, so they can be used with shift instructions. With 1,2 they would have to be used with multiply instructions, or else tweaked back to shift inputs. Most loops will speculate on the scale factor, but those loops which must work with a non-constant scale will be slightly cleaner with 0,1. ? John From mandy.chung at oracle.com Fri Feb 19 19:05:09 2016 From: mandy.chung at oracle.com (Mandy Chung) Date: Fri, 19 Feb 2016 11:05:09 -0800 Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: <431be29a-f350-4790-81bd-13365b505b0e@default> References: <5b1a5288-3c75-4437-aa78-b63169615db7@default> <66f6308b-60e1-4874-ae21-25c19f822f26@default> <90AE881D-5940-4BFF-91E6-7637AEB7D4EE@oracle.com> <431be29a-f350-4790-81bd-13365b505b0e@default> Message-ID: > On Feb 19, 2016, at 10:59 AM, Iris Clark wrote: > > Hi, Mandy. > >> Before Verona, sun.misc.Version was generated at build time and was intended to set up the system properties about ?java.version?, ?java.runtime.version?, ?java.runtime.name?. Did Verona change the logic such that this class is no longer needed? > > Verona did not change logic for VM initialization, we just updated it to reflect differences in naming. The class is still needed to handle VM initialization and to print out version information (see lines 67-124 [0]). Thanks for confirming that. > > Recommendations for deprecation/removal were in reference to the static methods, *{Major,Minor,...}Version() beginning at new line 127 which appear to simply retrieve portions of the system properties. > Good. Those methods were added for internal use for HS express era (supporting VM of different versions that is gone for long). It?s time to remove them. Mandy > Thanks, > Iris > > [0] http://cr.openjdk.java.net/~chegar/8150162/jdk/src/java.base/share/classes/java/lang/Version.java.template.html > > -----Original Message----- > From: Mandy Chung > Sent: Friday, February 19, 2016 10:48 AM > To: Iris Clark > Cc: Mandy Chung; Chris Hegarty; core-libs-dev > Subject: Re: RFR [9] 8150162: Move sun.misc.Version to a truly internal package > > >> On Feb 19, 2016, at 9:52 AM, Iris Clark wrote: >> >> Hi, Chris. >> >>>> I'm not the code maintainer, but I'd consider deprecating them. >> >>> Or just removing them. >> >> Agreed. > > Before Verona, sun.misc.Version was generated at build time and was intended to set up the system properties about ?java.version?, ?java.runtime.version?, ?java.runtime.name?. Did Verona change the logic such that this class is no longer needed? > > On the other hand, I agree that the static methods getting the version numbers should be removed as they are replaced by the new Version API. > > As for the webrev, moving to java.lang as a package-private type is fine. I suggest to rename it to avoid any confusion with the new Version API such as VersionProps (or something better)? > > Mandy > From martinrb at google.com Fri Feb 19 21:37:33 2016 From: martinrb at google.com (Martin Buchholz) Date: Fri, 19 Feb 2016 13:37:33 -0800 Subject: JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014 In-Reply-To: <56C762AB.1000108@oracle.com> References: <56C66655.9000009@oracle.com> <997ABE4C-F6D5-4F91-A32C-1D60B7913158@oracle.com> <56C762AB.1000108@oracle.com> Message-ID: I will commit by Monday. The only other change I'm thinking of would be to move the Unsafe initialization to the top of file (i.e. beginning of ) instead of bottom of file where we've traditionally placed it. Here a static method call that used U found it to be null - we must still be inside , eh? On Fri, Feb 19, 2016 at 10:44 AM, joe darcy wrote: > Hello, > > I'm happy if Martin's delambdafication is the right solution. > > However, if that is not pushed by Monday Feb. 22, I'll want to go ahead and > problem list the test. > > Thanks, > > -Joe > > > On 2/19/2016 1:40 AM, Paul Sandoz wrote: >> >> Hi Martin, >> >> Your patch to TLR and SR looks good: >> >> >> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/unlambdafyBootstrap/jsr166-jdk9-integration- >> >> Sorry, i should of caught this in review, did notice the change to use >> lambdas but i forgot that this might cause issues at VM startup. >> >> ConcurrentHashMap is used by infrastructure of j.l.invoke. CHM cannot >> itself leverage or depend on stuff that leverages lambdas otherwise it may >> result in a circular dependency at VM startup. >> >> In this case the test tickles the code paths that results in the >> initialization of ThreadLocalRandom by CHM during startup which explains why >> it was yet to be observed in other cases. >> >> Paul. >> >>> On 19 Feb 2016, at 07:18, Martin Buchholz wrote: >>> >>> It's easy enough to get the test to pass by delambdafication. >>> >>> But we may not want to fix the test because it's not clear whether >>> it's a jdk lambda bug or not, and we don't sweep bugs under the rug. >>> >>> Maybe instead of a "tier2" bucket we really want a "test found a bug >>> that's not yet fixed" bucket? >>> >>> On Thu, Feb 18, 2016 at 4:48 PM, joe darcy wrote: >>>> >>>> Hello, >>>> >>>> Right now, we are seeing some cross-platform intermittent failures of >>>> >>>> java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java >>>> >>>> as noted in bug JDK-8150014. Until that test instability is resolved, >>>> the >>>> test should be demoted from tier 1 to tier 2. >>>> >>>> Please see the patch below. >>>> >>>> (Circumstantial evidence points to the changes in JDK-8145485 being >>>> involved.) >>>> >>>> -Joe >>>> >>>> diff -r 3973fe856db2 test/TEST.groups >>>> --- a/test/TEST.groups Wed Feb 17 12:47:35 2016 -0800 >>>> +++ b/test/TEST.groups Thu Feb 18 16:44:40 2016 -0800 >>>> @@ -29,6 +29,7 @@ >>>> :jdk_lang \ >>>> -java/lang/ProcessHandle/TreeTest.java \ >>>> -java/util/zip/TestLocalTime.java \ >>>> + -java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >>>> :jdk_util \ >>>> -java/util/WeakHashMap/GCDuringIteration.java \ >>>> -java/util/concurrent/ThreadPoolExecutor/ConfigChanges.java \ >>>> @@ -39,6 +40,7 @@ >>>> :jdk_math >>>> >>>> tier2 = \ >>>> + java/lang/invoke/LFCaching/LFMultiThreadCachingTest.java \ >>>> java/lang/ProcessHandle/TreeTest.java \ >>>> java/util/zip/TestLocalTime.java \ >>>> java/util/WeakHashMap/GCDuringIteration.java \ >>>> > From christian.thalinger at oracle.com Fri Feb 19 22:40:40 2016 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 19 Feb 2016 12:40:40 -1000 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> <7AB17C7E-B33A-4EA4-A04F-C77E8E0EA21B@oracle.com> Message-ID: <14FEA851-0CA5-4565-8B7B-22AD26EFB204@oracle.com> > On Feb 19, 2016, at 9:03 AM, John Rose wrote: > > On Feb 19, 2016, at 9:57 AM, Christian Thalinger > wrote: >> >> Why don?t you change the values to: >> >> static final byte LATIN1 = 1; >> static final byte UTF16 = 2; > > > Not sure what you are asking, but here's my take on why 0,1 is the (slight) winner. > The values 0,1 are the log2 of the array element sizes 1,2, so they can be used with shift instructions. > With 1,2 they would have to be used with multiply instructions, or else tweaked back to shift inputs. > Most loops will speculate on the scale factor, but those loops which must work with a non-constant scale will be slightly cleaner with 0,1. I see what you are saying: int len = val.length >> coder; // assume LATIN1=0/UTF16=1; But if coder is stable for both values the compiler can constant fold the if-statement for the shift value: int len = val.length >> (coder == LATIN1 ? 0 : 1); That should produce the same code and we would avoid: 143 * Constant-folding this field is handled internally in VM. From john.r.rose at oracle.com Fri Feb 19 22:54:05 2016 From: john.r.rose at oracle.com (John Rose) Date: Fri, 19 Feb 2016 14:54:05 -0800 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <14FEA851-0CA5-4565-8B7B-22AD26EFB204@oracle.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> <7AB17C7E-B33A-4EA4-A04F-C77E8E0EA21B@oracle.com> <14FEA851-0CA5-4565-8B7B-22AD26EFB204@oracle.com> Message-ID: <195285F1-8037-46F6-952E-64DC3AC3FE0E@oracle.com> On Feb 19, 2016, at 2:40 PM, Christian Thalinger wrote: > >> >> On Feb 19, 2016, at 9:03 AM, John Rose > wrote: >> >> On Feb 19, 2016, at 9:57 AM, Christian Thalinger > wrote: >>> >>> Why don?t you change the values to: >>> >>> static final byte LATIN1 = 1; >>> static final byte UTF16 = 2; >> >> >> Not sure what you are asking, but here's my take on why 0,1 is the (slight) winner. >> The values 0,1 are the log2 of the array element sizes 1,2, so they can be used with shift instructions. >> With 1,2 they would have to be used with multiply instructions, or else tweaked back to shift inputs. >> Most loops will speculate on the scale factor, but those loops which must work with a non-constant scale will be slightly cleaner with 0,1. > > I see what you are saying: > > int len = val.length >> coder; // assume LATIN1=0/UTF16=1; > > But if coder is stable for both values the compiler can constant fold the if-statement for the shift value: > > int len = val.length >> (coder == LATIN1 ? 0 : 1); > > That should produce the same code and we would avoid: > > 143 * Constant-folding this field is handled internally in VM. Yes, the coder value can usually be speculated, especially if we encourage the JIT to pay special attention. I'm only talking about the parts where the coder cannot be speculated upon. ? John From aleksey.shipilev at oracle.com Fri Feb 19 23:22:28 2016 From: aleksey.shipilev at oracle.com (Aleksey Shipilev) Date: Sat, 20 Feb 2016 02:22:28 +0300 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <14FEA851-0CA5-4565-8B7B-22AD26EFB204@oracle.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> <7AB17C7E-B33A-4EA4-A04F-C77E8E0EA21B@oracle.com> <14FEA851-0CA5-4565-8B7B-22AD26EFB204@oracle.com> Message-ID: <56C7A3B4.7010904@oracle.com> On 02/20/2016 01:40 AM, Christian Thalinger wrote: >> On Feb 19, 2016, at 9:03 AM, John Rose wrote: >> On Feb 19, 2016, at 9:57 AM, Christian Thalinger > wrote: >>> Why don?t you change the values to: >>> >>> static final byte LATIN1 = 1; >>> static final byte UTF16 = 2; We've been over this during Compact Strings development. The idea that John has below is related to our actual insights leading to 0/1 instead of 1/2. The best thing you can do with 1/2 is, apparently: int length() { return value.length >> (coder - 1); } char charAt(int idx) { return getCharAt(idx * coder); // variant 1 return getCharAt(idx << (coder - 1)); // variant 2 } ...and you are better off not doing excess "-1" or non-strength-reducible multiplications in a very hot paths in String. Anyhow, that ship had sailed, and any change in coder definitions would require to respin an awful lot of Compact String testing, and probably revalidating lots of premises in the code. This is good as a thought experiment, but not practical at this point in JDK 9 development. > But if coder is stable for both values the compiler can constant fold the if-statement for the shift value: > > int len = val.length >> (coder == LATIN1 ? 0 : 1); > > That should produce the same code and we would avoid: > > 143 * Constant-folding this field is handled internally in VM. The constant-folding story is not the only story you should care about. For most Strings, we do not know either String.value or String.coder statically. This particular constant-folding bit affects String concatenation with constants, but the solution to it cannot possibly regress an overwhelming case of non-constant Strings. Changing the coder designations *would* affect non-constant Strings. I would guess that the comment on "coder" field throws a reader into thinking that @Stable is the answer to constant folding woes. But VM already trusts final fields in String (e.g. String.value.arraylength is folded, see JDK-8149813) -- as the part of TrustStaticNonFinalFields machinery, which hopefully some day would get exposed to other fields. Frozen arrays would hit the final (pun intended) nail into String.value folding. @Stable hack is doing that today; but that's a hack, for a very performance-sensitive corner in JDK. Hopefully the rewritten comments spell it better: http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.02/ http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/ Cheers, -Aleksey From christian.thalinger at oracle.com Fri Feb 19 23:31:06 2016 From: christian.thalinger at oracle.com (Christian Thalinger) Date: Fri, 19 Feb 2016 13:31:06 -1000 Subject: RFR (S) 8150180: String.value contents should be trusted In-Reply-To: <56C7A3B4.7010904@oracle.com> References: <56C702B6.5010903@oracle.com> <56C72118.6030304@gmail.com> <7AB17C7E-B33A-4EA4-A04F-C77E8E0EA21B@oracle.com> <14FEA851-0CA5-4565-8B7B-22AD26EFB204@oracle.com> <56C7A3B4.7010904@oracle.com> Message-ID: <60B3151D-7EDB-43A1-87BA-C2459501F0AD@oracle.com> > On Feb 19, 2016, at 1:22 PM, Aleksey Shipilev wrote: > > On 02/20/2016 01:40 AM, Christian Thalinger wrote: >>> On Feb 19, 2016, at 9:03 AM, John Rose wrote: >>> On Feb 19, 2016, at 9:57 AM, Christian Thalinger > wrote: >>>> Why don?t you change the values to: >>>> >>>> static final byte LATIN1 = 1; >>>> static final byte UTF16 = 2; > > We've been over this during Compact Strings development. The idea that > John has below is related to our actual insights leading to 0/1 instead > of 1/2. The best thing you can do with 1/2 is, apparently: > > int length() { > return value.length >> (coder - 1); > } > > char charAt(int idx) { > return getCharAt(idx * coder); // variant 1 > return getCharAt(idx << (coder - 1)); // variant 2 > } > > ...and you are better off not doing excess "-1" or > non-strength-reducible multiplications in a very hot paths in String. > > Anyhow, that ship had sailed, and any change in coder definitions would > require to respin an awful lot of Compact String testing, and probably > revalidating lots of premises in the code. This is good as a thought > experiment, but not practical at this point in JDK 9 development. > > >> But if coder is stable for both values the compiler can constant fold the if-statement for the shift value: >> >> int len = val.length >> (coder == LATIN1 ? 0 : 1); >> >> That should produce the same code and we would avoid: >> >> 143 * Constant-folding this field is handled internally in VM. > > The constant-folding story is not the only story you should care about. > > For most Strings, we do not know either String.value or String.coder > statically. This particular constant-folding bit affects String > concatenation with constants, but the solution to it cannot possibly > regress an overwhelming case of non-constant Strings. Changing the coder > designations *would* affect non-constant Strings. > > I would guess that the comment on "coder" field throws a reader into > thinking that @Stable is the answer to constant folding woes. But VM > already trusts final fields in String (e.g. String.value.arraylength is > folded, see JDK-8149813) -- as the part of TrustStaticNonFinalFields > machinery, which hopefully some day would get exposed to other fields. > Frozen arrays would hit the final (pun intended) nail into String.value > folding. @Stable hack is doing that today; but that's a hack, for a very > performance-sensitive corner in JDK. > > Hopefully the rewritten comments spell it better: That comment is better. > http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.02/ > http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/ > > Cheers, > -Aleksey > > From martinrb at google.com Sat Feb 20 20:41:01 2016 From: martinrb at google.com (Martin Buchholz) Date: Sat, 20 Feb 2016 12:41:01 -0800 Subject: JDK 9 RFR of 8150206: Demote LFMultiThreadCachingTest.java to tier 2 until resolution of JDK-8150014 In-Reply-To: References: <56C66655.9000009@oracle.com> <997ABE4C-F6D5-4F91-A32C-1D60B7913158@oracle.com> <56C762AB.1000108@oracle.com> Message-ID: On Fri, Feb 19, 2016 at 1:37 PM, Martin Buchholz wrote: > I will commit by Monday. Pushed. > The only other change I'm thinking of would be to move the Unsafe > initialization to the top of file (i.e. beginning of ) instead > of bottom of file where we've traditionally placed it. Here a static > method call that used U found it to be null - we must still be inside > , eh? Left for a future change. From amaembo at gmail.com Sun Feb 21 12:15:29 2016 From: amaembo at gmail.com (Tagir Valeev) Date: Sun, 21 Feb 2016 18:15:29 +0600 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> References: <6728448.20160214205343@gmail.com> <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> Message-ID: Thank you for valueable coments, here's updated webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r2/ Changes: - Spec updated according to Brian Goetz suggestion - Predicate wildcard - two-arg versions of iterate() are rewritten with Spliterator and started flag - IterateTest rewritten: now withData().stream().expectedResult().exercize() used; expected results added. With best regards, Tagir Valeev On Tue, Feb 16, 2016 at 7:32 PM, Paul Sandoz wrote: > Hi Tagir, > > The implementation approach looks good. I agree with Brian on the > documentation, esp. about the connection to a for loop. > > Testing-wise adding to the spliterator data sources is ok. Because you > have done that there is no need to explicitly use SpliteratorTestHelper in > IterateTest, as that is reasonably covered by SpliteratorTest. Since you > have the data providers for streams just run it through exerciseStream > instead. > > Regarding wild cards we could clean this up later and include other > methods [1]. It?s easy to go overboard on this, recommend sticking to ? > extends/? super where appropriate. The existing iterate methods use > UnaryOperator, thus there is no wiggle room there, am i ok with reusing > that for the 3-arg method. > > Regarding the existing two-args method, my inclination is to keep the > implementations separate from the 3-args, but feel free to change to using > abstract spliterator (reduce layering) using a similar code pattern (no > need for "finished" or overriding forEachRemaining). Then you can remove > Streams.NONE and it?s sneaky erased usage :-) > > Paul. > > [1] https://bugs.openjdk.java.net/browse/JDK-8132097 > > > On 14 Feb 2016, at 15:53, Tagir F. Valeev wrote: > > > > Hello! > > > > I wanted to work on foldLeft, but Brian asked me to take this issue > > instead. So here's webrev: > > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/ > > > > I don't like iterator-based Stream source implementations, so I made > > them AbstractSpliterator-based. I also implemented manually > > forEachRemaining as, I believe, this improves the performance in > > non-short-circuiting cases. > > > > I also decided to keep two flags (started and finished) to track the > > state. Currently existing implementation of infinite iterate() does > > not use started flag, but instead reads one element ahead for > > primitive streams. This seems wrong to me and may even lead to > > unexpected exceptions (*). I could get rid of "started" flag for > > Stream.iterate() using Streams.NONE, but this would make object > > implementation different from primitive implementations. It would also > > be possible to keep single three-state variable (byte or int, > > NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve > > the performance or footprint. Having two flags looks more readable to > > me. > > > > Currently existing two-arg iterate methods can now be expressed as a > > partial case of the new method: > > > > public static Stream iterate(final T seed, final UnaryOperator > f) { > > return iterate(seed, x -> true, f); > > } > > (same for primitive streams). I may do this if you think it's > > reasonable. > > > > I created new test class and added new iterate sources to existing > > data providers. > > > > Please review and sponsor! > > > > With best regards, > > Tagir Valeev. > > > > (*) Consider the following code: > > > > int[] data = {1,2,3,4,-1}; > > IntStream.iterate(0, x -> data[x]) > > .takeWhile(x -> x >= 0) > > .forEach(System.out::println); > > > > Currently this unexpectedly throws an AIOOBE, because > > IntStream.iterate unnecessarily tries to read one element ahead. > > > > From peter.levart at gmail.com Sun Feb 21 18:55:11 2016 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 21 Feb 2016 19:55:11 +0100 Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more In-Reply-To: <7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com> References: <56B72242.7050102@gmail.com> <56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com> <56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com> <015201d16813$333650c0$99a2f240$@apache.org> <56C34B1B.8050001@gmail.com> <56C43817.7060805@gmail.com> <7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com> Message-ID: <56CA080F.6010308@gmail.com> Hi Kim, Roger, I have prepared another webrev which addresses your outstanding concerns: http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.04/ I added new method to Cleaner: public boolean helpClean() { ... } I think this form of method that just does one quantum of work and returns a boolean indicating whether there's more work waiting is a better fit for some clients that might want to do just a limited amount of work at once (like for example sun.java2d.Disposer.pollRemove that you mentioned). This method also deals with helping the ReferenceHandler thread, which is necessary to be able to "squeeze" out all outstanding work. As Cleaner is in the same package as Reference and helping ReferenceHandler thread is implicitly included in Cleaner.helpClean(), the JavaLangRefAccess interface and a field in SharedSecrets can be removed. I also simplified the API in sun.nio.fs.NativeBuffer and replaced the accessor of Cleanable with a simple void free() method (called free because it deallocates memory). I think this will have to be submitted to CCC for approval, right? Can you help me with it? Regards, Peter On 02/17/2016 11:34 PM, Kim Barrett wrote: >> On Feb 17, 2016, at 4:06 AM, Peter Levart wrote: >> >> Hi Kim, >> >> Thanks for looking into this. Answers inline? > Peter, > > Thanks for the explanations. > >> On 02/17/2016 01:20 AM, Kim Barrett wrote: >>>> However, I recall Roger saying there were existing tests that >>> failed when certain uses of sun.misc.Cleaner were replaced with >>> java.lang.ref.Cleaner. I don't remember the details, but maybe Roger >>> does. >> If the failing test was the following: >> >> java/nio/Buffer/DirectBufferAllocTest.java >> >> ...then it has been taken care of (see Bits.java). > That looks familiar. And yes, I see what you did there, and I don't > think Roger's initial prototype testing did anything similar, so > indeed this is likely the failure he encountered. > > Though I'm still inclined to object to that form of drainQueue (see > below). > >>> I don't understand why CleanerImpl needs to be changed to public in >>> order to provide access to the new drainQueue. Wouldn't it be better >>> to add Cleaner.drainQueue? >> An interesting idea. But I don't know if such functionality is generally useful enough to commit to it in a public API. > java.desktop:sun.java2d.Disposer.pollRemove seems to me to be > addressing an essentially similar requirement, with > java.desktop:sun.font.StrikeCache being the user of that API. > > Of course, that's already got all the scaffolding for using phantom > references, and there's no need to rewrite it to use > java.lang.ref.Cleaner. But maybe there are others like this? > > In any case, I really dislike the approach of exposing the CleanerImpl > object to get at this functionality. > >>> Some of the other changes here don't seem related to the problem at >>> hand. ... >> One thing that this change unfortunately requires is to get rid of lambdas and method references in the implementation and dependencies of java.land.ref.Cleaner API, because it gets used early in the boot-up sequence when java.lang.invoke infrastructure is not ready yet. > Oh, right! Bootstrapping issues! > >> The alternative to CleanerCleanable is a no-op Runnable implementation passed to PhantomCleanableRef constructor. ? > OK. Now I understand. > >> Making CleanerImpl implement Runnable ? > I'd be fine with this if the CleanerImpl wasn't exposed as part of the > drainQueue functionality. > >>> ------------------------------------------------------------------------------ >>> src/java.base/share/classes/sun/nio/fs/NativeBuffer.java >>> 76 Cleaner.Cleanable cleanable() { >>> 77 return cleanable; >>> 78 } >>> >>> [pre-existing, but if we're changing things anyway...] >>> >>> I'm kind of surprised by an accessor function (both here and in the >>> original code) rather than a cleanup function. Is there a use case >>> for anything other than buffer.cleanable().clean()? >> It can't be, since both old Cleaner and new Cleanable have only got a single method - clean(). > So this could be replaced with > > void clean() { > cleanable.clean(); > } > > To me, that seems better. > >>> Similarly for the DirectBuffer interface. >> This one is a critical method, used by various 3rd party softwares... > I want to cover my ears when people start talking about some of the > things that have done? OK. > > > > From david.holmes at oracle.com Mon Feb 22 00:45:09 2016 From: david.holmes at oracle.com (David Holmes) Date: Mon, 22 Feb 2016 10:45:09 +1000 Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package In-Reply-To: References: Message-ID: <56CA5A15.2070901@oracle.com> Hi Chris, Hotspot and JDK changes seem fine to me. Thanks, David On 19/02/2016 1:52 AM, Chris Hegarty wrote: > sun.misc.Version is the core libraries part of a private interface with the > JVM to query and set specific JVM version and capabilities, as well as > being responsible for setting the system properties for "java.version?, > "java.runtime.version", and "java.runtime.name" ( which are generated > during the build ). > > It is not a "Critical API", as defined by JEP 260, so should be moved > out of sun.misc and placed into a more appropriate package where it > can be encapsulated. > > Version is only used by j.l.System during initialization, so I?ve taken the > liberty of moving it into java.lang as package-private. It could, however, > be placed in jdk.internal.misc, if it is more generally useful. That said, > most other usages should be able to use the JDK specific Version API. > > http://cr.openjdk.java.net/~chegar/8150162/ > https://bugs.openjdk.java.net/browse/JDK-8150162 > > -Chris. > > P.S. 8150163 & 8150168 are in progress to replace unnecessary usages > of sun.misc.Version with the JDK specific Version. > From michael.haupt at oracle.com Mon Feb 22 08:02:45 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Mon, 22 Feb 2016 09:02:45 +0100 Subject: RFR 8144931: Assert class signatures are correct and refer to valid classes In-Reply-To: References: <56C487F1.3030609@oracle.com> <56C5809F.3080506@oracle.com> <56C5A874.90606@oracle.com> <011201d16a53$e1e6e390$a5b4aab0$@apache.org> <012401d16a57$93deb480$bb9c1d80$@apache.org> <56C5DB53.1020504@oracle.com> <56C6A901.5090706@oracle.com> Message-ID: <1BC4DD06-7DD6-4837-9D4B-B6E57AE56008@oracle.com> All, I'll sponsor this. Best, Michael > Am 19.02.2016 um 12:37 schrieb Paul Sandoz : > > >> On 19 Feb 2016, at 06:32, shilpi.rastogi at oracle.com wrote: >> >> Hi All, >> >> Please see the updated webrev >> http://cr.openjdk.java.net/~srastogi/8144931/webrev.03/ >> > > +1 > > Paul. -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From nadeesh.tv at oracle.com Mon Feb 22 08:50:20 2016 From: nadeesh.tv at oracle.com (nadeesh tv) Date: Mon, 22 Feb 2016 14:20:20 +0530 Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with short time zone offset ("+01") In-Reply-To: <56BCED89.7040007@oracle.com> References: <56BCED89.7040007@oracle.com> Message-ID: <56CACBCC.6070609@oracle.com> Gentle Reminder On 2/12/2016 1:52 AM, nadeesh tv wrote: > Hi all, > > Please review a fix for > > Bug Id https://bugs.openjdk.java.net/browse/JDK-8032051 > > webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/ > -- Thanks and Regards, Nadeesh TV From paul.sandoz at oracle.com Mon Feb 22 08:50:39 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 22 Feb 2016 09:50:39 +0100 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: References: <6728448.20160214205343@gmail.com> <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> Message-ID: <617F3D61-3D50-4F41-B4A0-7E774084F3CC@oracle.com> Hi Tagir, > On 21 Feb 2016, at 13:15, Tagir Valeev wrote: > > Thank you for valueable coments, here's updated webrev: > > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r2/ > > Changes: > - Spec updated according to Brian Goetz suggestion > - Predicate wildcard > - two-arg versions of iterate() are rewritten with Spliterator and started flag > - IterateTest rewritten: now withData().stream().expectedResult().exercize() used; expected results added. > Looking good. Just minor comments. For the primitive streams ? + Spliterator.OfDouble spliterator = new Spliterators.AbstractDoubleSpliterator(Long.MAX_VALUE, + Spliterator.ORDERED | Spliterator.IMMUTABLE) { + double prev; ... - return StreamSupport.doubleStream(Spliterators.spliteratorUnknownSize( - iterator, - Spliterator.ORDERED | Spliterator.IMMUTABLE | Spliterator.NONNULL), false); We should retain the NONNULL characteristic. Thinking a little more that is something that should perhaps be added by default by the primitive abstract spliterators. Something to consider later maybe. Stream ? public static Stream iterate(final T seed, final UnaryOperator f) { Objects.requireNonNull(f); - final Iterator iterator = new Iterator() { - @SuppressWarnings("unchecked") - T t = (T) Streams.NONE; I believe you can now remove the field Streams.NONE? 1186 public static Stream iterate(final T seed, final UnaryOperator f) { 1187 Objects.requireNonNull(f); 1188 Spliterator spliterator = new Spliterators.AbstractSpliterator(Long.MAX_VALUE, 1189 Spliterator.ORDERED | Spliterator.IMMUTABLE) { I forgot to mention before you can now use the ?<>" on the RHS, although IDEs might not like it javac will :-) 1270 @Override 1271 public void forEachRemaining(Consumer action) { 1272 Objects.requireNonNull(action); 1273 if (finished) 1274 return; 1275 T t = started ? f.apply(prev) : seed; 1276 finished = true; A really minor thing but if you move ?finished = true? above one line, it means there is one less place where unintended recursive traversal can occur. IterateTest ? 57 {Arrays.asList(1, 2, 4, 8, 16, 32, 64, 128, 256, 512), 58 Factory.ofSupplier("ref.ten", () -> Stream.iterate(1, x -> x < 1000, x -> x * 2))}, We now have the method ?List.of", up to you if you wanna update to use that or not. Thanks, Paul. From amaembo at gmail.com Mon Feb 22 10:06:01 2016 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 22 Feb 2016 16:06:01 +0600 Subject: Collectors.minBy/maxBy behavior for null value Message-ID: Hello! Consider the following code: Comparator cmp = Comparator.nullsFirst(Comparator.naturalOrder()); System.out.println(Stream.of("a", "b", null).collect(Collectors.minBy(cmp))); It prints Optional.empty, so the result is indistinguishable from empty Stream (siimlar for maxBy). This behavior is not consistent with Stream.findFirst() (which throws NPE if the resulting element is null). Currently minBy spec says: This produces a result equivalent to: reducing(BinaryOperator.minBy(comparator)) However reducing spec says nothing what will occur if the result of reduction is null. To my opinion at very least the spec of reducing should be updated to specify the behavior when reducing result is null. It probably would also be nice to add a note into minBy/maxBy spec about this. What do you think? With best regards, Tagir Valeev. From amaembo at gmail.com Mon Feb 22 10:08:42 2016 From: amaembo at gmail.com (Tagir Valeev) Date: Mon, 22 Feb 2016 16:08:42 +0600 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: <617F3D61-3D50-4F41-B4A0-7E774084F3CC@oracle.com> References: <6728448.20160214205343@gmail.com> <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> <617F3D61-3D50-4F41-B4A0-7E774084F3CC@oracle.com> Message-ID: Hello! Here's updated webrev: http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r3/ All mentioned issues were fixed. I agree that using new features like List.of is better, because this way it's easier to find problems with them prior to release. With best regards, Tagir Valeev. On Mon, Feb 22, 2016 at 2:50 PM, Paul Sandoz wrote: > Hi Tagir, > > > On 21 Feb 2016, at 13:15, Tagir Valeev wrote: > > > > Thank you for valueable coments, here's updated webrev: > > > > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r2/ > > > > Changes: > > - Spec updated according to Brian Goetz suggestion > > - Predicate wildcard > > - two-arg versions of iterate() are rewritten with Spliterator and > started flag > > - IterateTest rewritten: now > withData().stream().expectedResult().exercize() used; expected results > added. > > > > Looking good. Just minor comments. > > For the primitive streams > ? > > + Spliterator.OfDouble spliterator = new > Spliterators.AbstractDoubleSpliterator(Long.MAX_VALUE, > + Spliterator.ORDERED | Spliterator.IMMUTABLE) { > + double prev; > ... > - return > StreamSupport.doubleStream(Spliterators.spliteratorUnknownSize( > - iterator, > - Spliterator.ORDERED | Spliterator.IMMUTABLE | > Spliterator.NONNULL), false); > > We should retain the NONNULL characteristic. > > Thinking a little more that is something that should perhaps be added by > default by the primitive abstract spliterators. Something to consider later > maybe. > > > Stream > ? > > public static Stream iterate(final T seed, final > UnaryOperator f) { > Objects.requireNonNull(f); > > - final Iterator iterator = new Iterator() { > - @SuppressWarnings("unchecked") > - T t = (T) Streams.NONE; > > I believe you can now remove the field Streams.NONE? > > > 1186 public static Stream iterate(final T seed, final > UnaryOperator f) { > 1187 Objects.requireNonNull(f); > 1188 Spliterator spliterator = new > Spliterators.AbstractSpliterator(Long.MAX_VALUE, > 1189 Spliterator.ORDERED | Spliterator.IMMUTABLE) { > > I forgot to mention before you can now use the ?<>" on the RHS, although > IDEs might not like it javac will :-) > > > 1270 @Override > 1271 public void forEachRemaining(Consumer action) { > 1272 Objects.requireNonNull(action); > 1273 if (finished) > 1274 return; > 1275 T t = started ? f.apply(prev) : seed; > 1276 finished = true; > > A really minor thing but if you move ?finished = true? above one line, it > means there is one less place where unintended recursive traversal can > occur. > > > IterateTest > ? > > 57 {Arrays.asList(1, 2, 4, 8, 16, 32, 64, 128, 256, 512), > 58 Factory.ofSupplier("ref.ten", () -> Stream.iterate(1, > x -> x < 1000, x -> x * 2))}, > > We now have the method ?List.of", up to you if you wanna update to use > that or not. > > > Thanks, > Paul. > > > From michael.haupt at oracle.com Mon Feb 22 11:40:05 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Mon, 22 Feb 2016 12:40:05 +0100 Subject: RFR(S): 8150360: augment/correct MethodHandle API documentation Message-ID: <4966FDEE-26F9-4343-9FC1-07CCEF5C410E@oracle.com> Dear all, please review this (doc-only) fix. Bug: https://bugs.openjdk.java.net/browse/JDK-8150360 Webrev: http://cr.openjdk.java.net/~mhaupt/8150360/webrev.00 The fix addresses one contradiction in the API documentation of MethodHandles.tryFinally(), and adds the following note to the API documentation of the method handle combinators given below: + *

    + * Note: The resulting adapter is never a {@linkplain MethodHandle#asVarargsCollector + * variable-arity method handle}, even if the original target method handle was. * MethodHandle.asCollector(), .bindTo() * MethodHandles.collectArguments(), .dropArguments(), .filterArguments(), .filterReturnValue(), .foldArguments(), .permuteArguments() Thanks, Michael -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From amy.lu at oracle.com Mon Feb 22 12:41:36 2016 From: amy.lu at oracle.com (Amy Lu) Date: Mon, 22 Feb 2016 20:41:36 +0800 Subject: JDK 9 RFR of JDK-8149154: tools/pack200/Pack200Test.java failed with NullPointerException Message-ID: <56CB0200.4060402@oracle.com> Please help to review this minor fix to post-test clean-up where NullPointerException should be avoided. bug: https://bugs.openjdk.java.net/browse/JDK-8149154 webrev: http://cr.openjdk.java.net/~amlu/8149154/webrev.00/ Thanks, Amy --- old/test/tools/pack200/Pack200Test.java 2016-02-20 19:12:36.000000000 +0800 +++ new/test/tools/pack200/Pack200Test.java 2016-02-20 19:12:35.000000000 +0800 @@ -81,7 +81,9 @@ System.out.println("Packing [" + in.toString() + "]"); // Call the packer Utils.pack(jarFile, packFile); + System.out.println("Done Packing [" + in.toString() + "]"); jarFile.close(); + System.out.println("Start leak check"); leakCheck(); System.out.println(" Unpacking using java unpacker"); --- old/test/tools/pack200/Utils.java 2016-02-20 19:12:38.000000000 +0800 +++ new/test/tools/pack200/Utils.java 2016-02-20 19:12:38.000000000 +0800 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2007, 2014, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2007, 2016, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -324,6 +324,9 @@ private static void findFiles0(File startDir, List list, FileFilter filter) throws IOException { File[] foundFiles = startDir.listFiles(filter); + if (foundFiles == null) { + return; + } list.addAll(Arrays.asList(foundFiles)); File[] dirs = startDir.listFiles(DIR_FILTER); for (File dir : dirs) { From michael.haupt at oracle.com Mon Feb 22 14:08:37 2016 From: michael.haupt at oracle.com (Michael Haupt) Date: Mon, 22 Feb 2016 15:08:37 +0100 Subject: RFR(S): 8143410: augment pseudo-code descriptions in MethodHandles API Message-ID: <4C5FD37F-D976-414F-97FA-4EB9731A7675@oracle.com> Dear all, please review this (doc-only) fix. Bug: https://bugs.openjdk.java.net/browse/JDK-8143410 Webrev: http://cr.openjdk.java.net/~mhaupt/8143410/webrev.00 The change documents the type variables used in the pseudo-code in the API documentation of the following methods in MethodHandles: catchException(), collectArguments(), filterArguments(), filterReturnValue(), foldArguments(), and guardWithTest(). Thanks, Michael -- Dr. Michael Haupt | Principal Member of Technical Staff Phone: +49 331 200 7277 | Fax: +49 331 200 7561 Oracle Java Platform Group | LangTools Team | Nashorn Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen Registergericht: Amtsgericht M?nchen, HRA 95603 Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697 Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher Oracle is committed to developing practices and products that help protect the environment From peter.levart at gmail.com Mon Feb 22 15:09:45 2016 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Feb 2016 16:09:45 +0100 Subject: Collectors.minBy/maxBy behavior for null value In-Reply-To: References: Message-ID: <56CB24B9.4010907@gmail.com> On 02/22/2016 11:06 AM, Tagir Valeev wrote: > Hello! > > Consider the following code: > > Comparator cmp = Comparator.nullsFirst(Comparator.naturalOrder()); > System.out.println(Stream.of("a", "b", > null).collect(Collectors.minBy(cmp))); > > It prints Optional.empty, so the result is indistinguishable from empty > Stream (siimlar for maxBy). This behavior is not consistent with > Stream.findFirst() (which throws NPE if the resulting element is null). > Currently minBy spec says: > > This produces a result equivalent to: > reducing(BinaryOperator.minBy(comparator)) > > However reducing spec says nothing what will occur if the result of > reduction is null. To my opinion at very least the spec of reducing should > be updated to specify the behavior when reducing result is null. It > probably would also be nice to add a note into minBy/maxBy spec about this. > > What do you think? My personal opinion is that it would be better to throw NPE in this case (like with findFirst()). Silently converting null element to Optional.empty() is probably not the desired behavior and I doubt anyone is exploiting this implementation detail in such way. Regards, Peter > > With best regards, > Tagir Valeev. From paul.sandoz at oracle.com Mon Feb 22 15:19:32 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 22 Feb 2016 16:19:32 +0100 Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite In-Reply-To: References: <6728448.20160214205343@gmail.com> <009FEBB7-FF29-4C08-AEE0-BCA5562040AD@oracle.com> <617F3D61-3D50-4F41-B4A0-7E774084F3CC@oracle.com> Message-ID: <332DAC2A-AD11-4881-B370-C49EB2D6BC42@oracle.com> > On 22 Feb 2016, at 11:08, Tagir Valeev wrote: > > Hello! > > Here's updated webrev: > http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r3/ > All mentioned issues were fixed. Many thanks, i will start an internal compatibility review (CCC) and i will also do one for onClose issue at the same time, once those complete i will push ?em. > I agree that using new features like List.of is better, because this way it's easier to find problems with them prior to release. > Yes, plus reviews can be an opportunity to communicate new stuff, even if not used. In general I like would to see List.of become the replacement for Arrays.asList (when in-place modification is not required). Paul. > With best regards, > Tagir Valeev. > From kumar.x.srinivasan at oracle.com Mon Feb 22 15:24:14 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Mon, 22 Feb 2016 07:24:14 -0800 Subject: JDK 9 RFR of JDK-8149154: tools/pack200/Pack200Test.java failed with NullPointerException In-Reply-To: <56CB0200.4060402@oracle.com> References: <56CB0200.4060402@oracle.com> Message-ID: <56CB281E.5020708@oracle.com> Hi Amy, Thanks for doing this. Looks good. Kumar > Please help to review this minor fix to post-test clean-up where > NullPointerException should be avoided. > > bug: https://bugs.openjdk.java.net/browse/JDK-8149154 > webrev: http://cr.openjdk.java.net/~amlu/8149154/webrev.00/ > > Thanks, > Amy > > --- old/test/tools/pack200/Pack200Test.java 2016-02-20 19:12:36.000000000 +0800 > +++ new/test/tools/pack200/Pack200Test.java 2016-02-20 19:12:35.000000000 +0800 > @@ -81,7 +81,9 @@ > System.out.println("Packing [" + in.toString() + "]"); > // Call the packer > Utils.pack(jarFile, packFile); > + System.out.println("Done Packing [" + in.toString() + "]"); > jarFile.close(); > + System.out.println("Start leak check"); > leakCheck(); > > System.out.println(" Unpacking using java unpacker"); > --- old/test/tools/pack200/Utils.java 2016-02-20 19:12:38.000000000 +0800 > +++ new/test/tools/pack200/Utils.java 2016-02-20 19:12:38.000000000 +0800 > @@ -1,5 +1,5 @@ > /* > - * Copyright (c) 2007, 2014, Oracle and/or its affiliates. All rights reserved. > + * Copyright (c) 2007, 2016, Oracle and/or its affiliates. All rights reserved. > * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. > * > * This code is free software; you can redistribute it and/or modify it > @@ -324,6 +324,9 @@ > private static void findFiles0(File startDir, List list, > FileFilter filter) throws IOException { > File[] foundFiles = startDir.listFiles(filter); > + if (foundFiles == null) { > + return; > + } > list.addAll(Arrays.asList(foundFiles)); > File[] dirs = startDir.listFiles(DIR_FILTER); > for (File dir : dirs) { > From kumar.x.srinivasan at oracle.com Mon Feb 22 16:00:25 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Mon, 22 Feb 2016 08:00:25 -0800 Subject: RFR 8124977 cmdline encoding challenges on Windows In-Reply-To: References: <558C017A.2020708@oracle.com> <5651E9AA.9060908@oracle.com> <56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com> Message-ID: <56CB3099.4030108@oracle.com> Hi Naoto, Sherman, can you please take a look. I tested with the jprt build and test all tests pass. Hi Vladimir, et. al., It appears that there has been more simplifications from the previous webrev.04. :-) It would've helped if you highlight the changes you have made from the previous revision, unfortunately this is one of the deficiencies of webrev. There are some inconsistencies in the coding conventions: parse_manifest.c + if (q == 0) return -1; we expect the return to be on the next line. similarly main.c if (0 == q) { I can fix those up. If I were to push this change, who should I attribute the changes to ? ie. in the Contributed-by: line of the commit info ? Please note these have to be email addresses of the contributors. Thanks Kumar > Hi Kumar, > > We posted another web review here: http://cr.openjdk.java.net/~kshoop/8124977/webrev.05/ > > The patch was successfully tested. > > Test details: > * Regression tests folder: jdk/test/tools/launcher/ > * Builds were used: windows-x86_64-normal-server-fastdebug, windows-x86_64-normal-server-release, windows-x86-normal-server-release; > * Platforms were used: Windows 7(64 bit), Windows 8.1, Windows Server 2012 R2 DC, Windows 10 ; > * System locales were used: English (United States), Persian, Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia); > > Thanks, > Vladimir. > > -----Original Message----- > From: Martin Sawicki > Sent: Thursday, January 14, 2016 11:34 AM > To: Kumar Srinivasan ; Vladimir Shcherbakov > Cc: core-libs-dev Libs ; Naoto Sato > Subject: RE: RFR 8124977 cmdline encoding challenges on Windows > > Thanks for the feedback. > Investigating the regression failure. > We'll get back as soon as we figure this out. (and yes, we'll run this through some localized Windows VMs) > > Cheers > > -----Original Message----- > From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] > Sent: Tuesday, January 12, 2016 2:35 PM > To: Martin Sawicki ; Vladimir Shcherbakov > Cc: core-libs-dev Libs ; Naoto Sato > Subject: Re: RFR 8124977 cmdline encoding challenges on Windows > > Hi Martin, Vladimir, > > It was suggested that this patch be tested on localized Windows machines and/or trying with the various Windows native encodings, appreciate if you can verify this as well. > > Thanks > Kumar > > On 1/11/2016 1:10 PM, Kumar Srinivasan wrote: >> Hi, >> >> Was on vacation, I started to prepare the patch from webrev.04 for >> integration. Please note: made some adjustments to your patch to pass >> jcheck, ie. usage of tabs and space at line endings, and modifications >> to Copyright dates. >> >> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE. >> I have attached a patch to for your reference. >> >> However, there is a regression test failure on Windows, >> jdk/test/tools/launcher/I18NTest.java >> >> ---Test info---- >> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java >> >> ++++Test Output++++ >> javac: file not found: i18nH?lloWorld.java ----End test info----- >> >> Have you run all the launcher regression tests with this changeset ? >> >> Thanks >> Kumar >> >>> Hi Kumar, just wondering if there are any updates on processing this >>> submission. >>> Thanks! >>> >>> -----Original Message----- >>> From: Vladimir Shcherbakov >>> Sent: Wednesday, November 25, 2015 2:38 PM >>> To: Kumar Srinivasan ; Martin Sawicki >>> >>> Cc: Kirk Shoop ; core-libs-dev Libs >>> >>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows >>> >>> Hi Kumar, >>> >>> Please find updated webreview here: >>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open >>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarcin >>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f141 >>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNfwn >>> bIw%3d >>> >>> Thanks, >>> Vladimir. >>> >>> -----Original Message----- >>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] >>> Sent: Sunday, November 22, 2015 8:14 AM >>> To: Martin Sawicki >>> Cc: Kirk Shoop ; Vladimir Shcherbakov >>> ; core-libs-dev Libs >>> >>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >>> >>> >>> Hi Martin, et. al., >>> >>> Sorry for not getting back earlier, I am very busy right now with my >>> other large commitments for JDK9. >>> >>> I will sponsor this "enhancement/bug fix" sometime in the new year, >>> meanwhile, there is the changeset [1] which is likely to cause merge >>> conflicts, and perhaps logic issues. >>> >>> Thanks >>> Kumar >>> >>> [1] >>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.op >>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7 >>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988b >>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7M >>> g4lt2P0sdzLS%2fT9Q%3d >>>> Hi all >>>> Here's an updated webrev attempting to take into account the various >>>> pieces of feedback we have received: >>>> >>>> Issue: >>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. >>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic >>>> ro >>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d >>>> 7c >>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E >>>> %3 >>>> d >>>> Webrev: >>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.ope >>>> nj >>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashc >>>> h% >>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141a >>>> f9 >>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJn >>>> WY >>>> 0%3d >>>> >>>> (Vladimir Shcherbakov is now working on this from our side) >>>> >>>> Looking forward to any other feedback. >>>> Thanks >>>> >>>> -----Original Message----- >>>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] >>>> On Behalf Of Kumar Srinivasan >>>> Sent: Thursday, June 25, 2015 6:26 AM >>>> To: Kirk Shoop (MS OPEN TECH) >>>> Cc: Valery Kopylov (Akvelon) ; core-libs-dev >>>> Libs >>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >>>> >>>> Hi Kirk, >>>> >>>> Thanks for proposing this change. >>>> >>>> If you notice all the posix calls are wrapped in JLI_* this gives us >>>> the ability to use "W" functions. I almost got it done, several >>>> years ago, but we upgraded to VS2010 and my work based on VS2003 >>>> keeled over, meanwhile my focus was "shifted" to something else. >>>> >>>> main.c: is really envisioned to be a stub compiled by the tool >>>> launchers, like java, javac, javah, jar etc. I prefer to see all the >>>> heavy logic in this file moved to the platform specific file >>>> windows/java_md.* >>>> >>>> For the reason specified above we need to move fprintf or any naked >>>> posix calls to JLI_* indirections. >>>> >>>> I don't see any tests ? The tests must be written in java and placed >>>> in jdk/test/tools/launcher, there is a helper framework >>>> TestHelper.java. >>>> >>>> There are other changes in nio, charsets etc, this will be reviewed >>>> by my colleague specializing in that area (Sherman) cc'ed. >>>> >>>> >>>> Thanks >>>> >>>> Kumar >>>> >>>> >>>> >>>> >>>> >>>> >>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote: >>>>> Hi, >>>>> >>>>> Issue: >>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbu >>>>> gs >>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m >>>>> ic >>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91a >>>>> b2 >>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsU >>>>> Iq >>>>> 9E%3d >>>>> >>>>> Webrev: >>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op >>>>> en >>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micros >>>>> of >>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7c >>>>> d0 >>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M% >>>>> 3d >>>>> >>>>> This webrev intends to address interaction between Windows console >>>>> and java apps. >>>>> >>>>> Two switches were added that change the behavior of the launcher. >>>>> The defaults do not change the launcher behavior. >>>>> >>>>> -Dwindows.UnicodeConsole=true - switches on Unicode support in >>>>> the Windows console. This optional switch causes the launcher to >>>>> call GetCommandLineW() and parse the arguments in unicode. It also >>>>> modifies how the codepage for console output is selected. >>>>> >>>>> -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset >>>>> to use; If not specified, UTF-8 is used by default. Ignored when >>>>> windows.UnicodeConsole is not set to true. When the first switch is >>>>> used, this optional switch allows the codepage for console output >>>>> to be controlled. >>>>> >>>>> I would like to get feedback on the approach here and any >>>>> additional work that is required solve these particular Unicode >>>>> issues on Windows. >>>>> >>>>> Kirk From kumar.x.srinivasan at oracle.com Mon Feb 22 16:18:05 2016 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Mon, 22 Feb 2016 08:18:05 -0800 Subject: RFR 8124977 cmdline encoding challenges on Windows In-Reply-To: <56CB3099.4030108@oracle.com> References: <558C017A.2020708@oracle.com> <5651E9AA.9060908@oracle.com> <56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com> <56CB3099.4030108@oracle.com> Message-ID: <56CB34BD.4010707@oracle.com> Few more comments on the tests: UnicodeCmdTestRun.java: The summary is fine, but it would be good to add a comment explaining what this test really does, random folks look at the tests when a test failure occurs, such a comment should help, similar to what you have in UnicodeCmdTest.java UnicodeCmdTest.java: it would be good add an error at line 33, + System.out.println("Error: 0 length argument"); System.exit(1); Thanks Kumar > > Hi Naoto, Sherman, can you please take a look. > I tested with the jprt build and test all tests pass. > > Hi Vladimir, et. al., > > It appears that there has been more simplifications from the previous > webrev.04. :-) > > It would've helped if you highlight the changes you have made from the > previous > revision, unfortunately this is one of the deficiencies of webrev. > > There are some inconsistencies in the coding conventions: > > parse_manifest.c > + if (q == 0) return -1; > > we expect the return to be on the next line. > > similarly main.c > > if (0 == q) > { > > I can fix those up. If I were to push this change, who should I > attribute the > changes to ? ie. in the Contributed-by: line of the commit info ? > Please note these have to be email addresses of the contributors. > > Thanks > Kumar > >> Hi Kumar, >> >> We posted another web review here: >> http://cr.openjdk.java.net/~kshoop/8124977/webrev.05/ >> >> The patch was successfully tested. >> >> Test details: >> * Regression tests folder: jdk/test/tools/launcher/ >> * Builds were used: windows-x86_64-normal-server-fastdebug, >> windows-x86_64-normal-server-release, windows-x86-normal-server-release; >> * Platforms were used: Windows 7(64 bit), Windows 8.1, Windows >> Server 2012 R2 DC, Windows 10 ; >> * System locales were used: English (United States), Persian, >> Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia); >> >> Thanks, >> Vladimir. >> >> -----Original Message----- >> From: Martin Sawicki >> Sent: Thursday, January 14, 2016 11:34 AM >> To: Kumar Srinivasan ; Vladimir >> Shcherbakov >> Cc: core-libs-dev Libs ; Naoto Sato >> >> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows >> >> Thanks for the feedback. >> Investigating the regression failure. >> We'll get back as soon as we figure this out. (and yes, we'll run >> this through some localized Windows VMs) >> >> Cheers >> >> -----Original Message----- >> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] >> Sent: Tuesday, January 12, 2016 2:35 PM >> To: Martin Sawicki ; Vladimir Shcherbakov >> >> Cc: core-libs-dev Libs ; Naoto Sato >> >> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >> >> Hi Martin, Vladimir, >> >> It was suggested that this patch be tested on localized Windows >> machines and/or trying with the various Windows native encodings, >> appreciate if you can verify this as well. >> >> Thanks >> Kumar >> >> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote: >>> Hi, >>> >>> Was on vacation, I started to prepare the patch from webrev.04 for >>> integration. Please note: made some adjustments to your patch to pass >>> jcheck, ie. usage of tabs and space at line endings, and modifications >>> to Copyright dates. >>> >>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE. >>> I have attached a patch to for your reference. >>> >>> However, there is a regression test failure on Windows, >>> jdk/test/tools/launcher/I18NTest.java >>> >>> ---Test info---- >>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java >>> >>> ++++Test Output++++ >>> javac: file not found: i18nH?lloWorld.java ----End test info----- >>> >>> Have you run all the launcher regression tests with this changeset ? >>> >>> Thanks >>> Kumar >>> >>>> Hi Kumar, just wondering if there are any updates on processing this >>>> submission. >>>> Thanks! >>>> >>>> -----Original Message----- >>>> From: Vladimir Shcherbakov >>>> Sent: Wednesday, November 25, 2015 2:38 PM >>>> To: Kumar Srinivasan ; Martin Sawicki >>>> >>>> Cc: Kirk Shoop ; core-libs-dev Libs >>>> >>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows >>>> >>>> Hi Kumar, >>>> >>>> Please find updated webreview here: >>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open >>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarcin >>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f141 >>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNfwn >>>> bIw%3d >>>> >>>> Thanks, >>>> Vladimir. >>>> >>>> -----Original Message----- >>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] >>>> Sent: Sunday, November 22, 2015 8:14 AM >>>> To: Martin Sawicki >>>> Cc: Kirk Shoop ; Vladimir Shcherbakov >>>> ; core-libs-dev Libs >>>> >>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >>>> >>>> >>>> Hi Martin, et. al., >>>> >>>> Sorry for not getting back earlier, I am very busy right now with my >>>> other large commitments for JDK9. >>>> >>>> I will sponsor this "enhancement/bug fix" sometime in the new year, >>>> meanwhile, there is the changeset [1] which is likely to cause merge >>>> conflicts, and perhaps logic issues. >>>> >>>> Thanks >>>> Kumar >>>> >>>> [1] >>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.op >>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7 >>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988b >>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7M >>>> g4lt2P0sdzLS%2fT9Q%3d >>>>> Hi all >>>>> Here's an updated webrev attempting to take into account the various >>>>> pieces of feedback we have received: >>>>> >>>>> Issue: >>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. >>>>> >>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic >>>>> ro >>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d >>>>> 7c >>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E >>>>> %3 >>>>> d >>>>> Webrev: >>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.ope >>>>> nj >>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashc >>>>> h% >>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141a >>>>> f9 >>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJn >>>>> WY >>>>> 0%3d >>>>> >>>>> (Vladimir Shcherbakov is now working on this from our side) >>>>> >>>>> Looking forward to any other feedback. >>>>> Thanks >>>>> >>>>> -----Original Message----- >>>>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] >>>>> On Behalf Of Kumar Srinivasan >>>>> Sent: Thursday, June 25, 2015 6:26 AM >>>>> To: Kirk Shoop (MS OPEN TECH) >>>>> Cc: Valery Kopylov (Akvelon) ; core-libs-dev >>>>> Libs >>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >>>>> >>>>> Hi Kirk, >>>>> >>>>> Thanks for proposing this change. >>>>> >>>>> If you notice all the posix calls are wrapped in JLI_* this gives us >>>>> the ability to use "W" functions. I almost got it done, several >>>>> years ago, but we upgraded to VS2010 and my work based on VS2003 >>>>> keeled over, meanwhile my focus was "shifted" to something else. >>>>> >>>>> main.c: is really envisioned to be a stub compiled by the tool >>>>> launchers, like java, javac, javah, jar etc. I prefer to see all the >>>>> heavy logic in this file moved to the platform specific file >>>>> windows/java_md.* >>>>> >>>>> For the reason specified above we need to move fprintf or any naked >>>>> posix calls to JLI_* indirections. >>>>> >>>>> I don't see any tests ? The tests must be written in java and placed >>>>> in jdk/test/tools/launcher, there is a helper framework >>>>> TestHelper.java. >>>>> >>>>> There are other changes in nio, charsets etc, this will be reviewed >>>>> by my colleague specializing in that area (Sherman) cc'ed. >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Kumar >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote: >>>>>> Hi, >>>>>> >>>>>> Issue: >>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbu >>>>>> gs >>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m >>>>>> ic >>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91a >>>>>> b2 >>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsU >>>>>> Iq >>>>>> 9E%3d >>>>>> >>>>>> Webrev: >>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op >>>>>> en >>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micros >>>>>> of >>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7c >>>>>> d0 >>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M% >>>>>> 3d >>>>>> >>>>>> This webrev intends to address interaction between Windows console >>>>>> and java apps. >>>>>> >>>>>> Two switches were added that change the behavior of the launcher. >>>>>> The defaults do not change the launcher behavior. >>>>>> >>>>>> -Dwindows.UnicodeConsole=true - switches on Unicode support in >>>>>> the Windows console. This optional switch causes the launcher to >>>>>> call GetCommandLineW() and parse the arguments in unicode. It also >>>>>> modifies how the codepage for console output is selected. >>>>>> >>>>>> -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset >>>>>> to use; If not specified, UTF-8 is used by default. Ignored when >>>>>> windows.UnicodeConsole is not set to true. When the first switch is >>>>>> used, this optional switch allows the codepage for console output >>>>>> to be controlled. >>>>>> >>>>>> I would like to get feedback on the approach here and any >>>>>> additional work that is required solve these particular Unicode >>>>>> issues on Windows. >>>>>> >>>>>> Kirk > From vlashch at microsoft.com Wed Feb 17 21:53:13 2016 From: vlashch at microsoft.com (Vladimir Shcherbakov) Date: Wed, 17 Feb 2016 21:53:13 +0000 Subject: RFR 8124977 cmdline encoding challenges on Windows In-Reply-To: References: <558C017A.2020708@oracle.com> <5651E9AA.9060908@oracle.com> <56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com> Message-ID: Hi Kumar, We posted another web review here: http://cr.openjdk.java.net/~kshoop/8124977/webrev.05/ The patch was successfully tested. Test details: * Regression tests folder: jdk/test/tools/launcher/ * Builds were used: windows-x86_64-normal-server-fastdebug, windows-x86_64-normal-server-release, windows-x86-normal-server-release; * Platforms were used: Windows 7(64 bit), Windows 8.1, Windows Server 2012 R2 DC, Windows 10 ; * System locales were used: English (United States), Persian, Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia); Thanks, Vladimir. -----Original Message----- From: Martin Sawicki Sent: Thursday, January 14, 2016 11:34 AM To: Kumar Srinivasan ; Vladimir Shcherbakov Cc: core-libs-dev Libs ; Naoto Sato Subject: RE: RFR 8124977 cmdline encoding challenges on Windows Thanks for the feedback. Investigating the regression failure. We'll get back as soon as we figure this out. (and yes, we'll run this through some localized Windows VMs) Cheers -----Original Message----- From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] Sent: Tuesday, January 12, 2016 2:35 PM To: Martin Sawicki ; Vladimir Shcherbakov Cc: core-libs-dev Libs ; Naoto Sato Subject: Re: RFR 8124977 cmdline encoding challenges on Windows Hi Martin, Vladimir, It was suggested that this patch be tested on localized Windows machines and/or trying with the various Windows native encodings, appreciate if you can verify this as well. Thanks Kumar On 1/11/2016 1:10 PM, Kumar Srinivasan wrote: > Hi, > > Was on vacation, I started to prepare the patch from webrev.04 for > integration. Please note: made some adjustments to your patch to pass > jcheck, ie. usage of tabs and space at line endings, and modifications > to Copyright dates. > > Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE. > I have attached a patch to for your reference. > > However, there is a regression test failure on Windows, > jdk/test/tools/launcher/I18NTest.java > > ---Test info---- > Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java > > ++++Test Output++++ > javac: file not found: i18nH?lloWorld.java ----End test info----- > > Have you run all the launcher regression tests with this changeset ? > > Thanks > Kumar > >> Hi Kumar, just wondering if there are any updates on processing this >> submission. >> Thanks! >> >> -----Original Message----- >> From: Vladimir Shcherbakov >> Sent: Wednesday, November 25, 2015 2:38 PM >> To: Kumar Srinivasan ; Martin Sawicki >> >> Cc: Kirk Shoop ; core-libs-dev Libs >> >> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows >> >> Hi Kumar, >> >> Please find updated webreview here: >> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open >> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarcin >> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f141 >> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNfwn >> bIw%3d >> >> Thanks, >> Vladimir. >> >> -----Original Message----- >> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] >> Sent: Sunday, November 22, 2015 8:14 AM >> To: Martin Sawicki >> Cc: Kirk Shoop ; Vladimir Shcherbakov >> ; core-libs-dev Libs >> >> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >> >> >> Hi Martin, et. al., >> >> Sorry for not getting back earlier, I am very busy right now with my >> other large commitments for JDK9. >> >> I will sponsor this "enhancement/bug fix" sometime in the new year, >> meanwhile, there is the changeset [1] which is likely to cause merge >> conflicts, and perhaps logic issues. >> >> Thanks >> Kumar >> >> [1] >> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.op >> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7 >> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988b >> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7M >> g4lt2P0sdzLS%2fT9Q%3d >>> Hi all >>> Here's an updated webrev attempting to take into account the various >>> pieces of feedback we have received: >>> >>> Issue: >>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. >>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic >>> ro >>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d >>> 7c >>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E >>> %3 >>> d >>> Webrev: >>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.ope >>> nj >>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashc >>> h% >>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141a >>> f9 >>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJn >>> WY >>> 0%3d >>> >>> (Vladimir Shcherbakov is now working on this from our side) >>> >>> Looking forward to any other feedback. >>> Thanks >>> >>> -----Original Message----- >>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net] >>> On Behalf Of Kumar Srinivasan >>> Sent: Thursday, June 25, 2015 6:26 AM >>> To: Kirk Shoop (MS OPEN TECH) >>> Cc: Valery Kopylov (Akvelon) ; core-libs-dev >>> Libs >>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows >>> >>> Hi Kirk, >>> >>> Thanks for proposing this change. >>> >>> If you notice all the posix calls are wrapped in JLI_* this gives us >>> the ability to use "W" functions. I almost got it done, several >>> years ago, but we upgraded to VS2010 and my work based on VS2003 >>> keeled over, meanwhile my focus was "shifted" to something else. >>> >>> main.c: is really envisioned to be a stub compiled by the tool >>> launchers, like java, javac, javah, jar etc. I prefer to see all the >>> heavy logic in this file moved to the platform specific file >>> windows/java_md.* >>> >>> For the reason specified above we need to move fprintf or any naked >>> posix calls to JLI_* indirections. >>> >>> I don't see any tests ? The tests must be written in java and placed >>> in jdk/test/tools/launcher, there is a helper framework >>> TestHelper.java. >>> >>> There are other changes in nio, charsets etc, this will be reviewed >>> by my colleague specializing in that area (Sherman) cc'ed. >>> >>> >>> Thanks >>> >>> Kumar >>> >>> >>> >>> >>> >>> >>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote: >>>> Hi, >>>> >>>> Issue: >>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbu >>>> gs >>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m >>>> ic >>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91a >>>> b2 >>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsU >>>> Iq >>>> 9E%3d >>>> >>>> Webrev: >>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op >>>> en >>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micros >>>> of >>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7c >>>> d0 >>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M% >>>> 3d >>>> >>>> This webrev intends to address interaction between Windows console >>>> and java apps. >>>> >>>> Two switches were added that change the behavior of the launcher. >>>> The defaults do not change the launcher behavior. >>>> >>>> -Dwindows.UnicodeConsole=true - switches on Unicode support in >>>> the Windows console. This optional switch causes the launcher to >>>> call GetCommandLineW() and parse the arguments in unicode. It also >>>> modifies how the codepage for console output is selected. >>>> >>>> -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset >>>> to use; If not specified, UTF-8 is used by default. Ignored when >>>> windows.UnicodeConsole is not set to true. When the first switch is >>>> used, this optional switch allows the codepage for console output >>>> to be controlled. >>>> >>>> I would like to get feedback on the approach here and any >>>> additional work that is required solve these particular Unicode >>>> issues on Windows. >>>> >>>> Kirk > From mark.reinhold at oracle.com Mon Feb 22 18:11:33 2016 From: mark.reinhold at oracle.com (mark.reinhold at oracle.com) Date: Mon, 22 Feb 2016 10:11:33 -0800 Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal In-Reply-To: References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com> <561428BC.3060806@cs.oswego.edu> <015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com> <56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu> <54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com> <561FC6F9.1080701@cs.oswego.edu> <25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com> <562BC968.70603@cs.oswego.edu> <8917A25C-4A16-4291-B145-FA7180E09F78@azul.com> <4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com> <20151207093440.607909@eggemoggin.niobe.net> Message-ID: <20160222101133.946427992eggemoggin.niobe.net> 2016/1/28 9:25 -0800, gil at azul.com: > This thread seems to have "hopped away" to the concurrency-interest > list in mid-Dec-2015. This posting is intended to capture a summary of > reasoning and some of the discussion there so that we have it in the > record in core-libs-dev. Mostly by including the contents of several > posts in the continuations of the original thread. > > See thread continuations here: > http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 > and here: > http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580 > > Summary: > > ... Thanks for the summary. I still don't buy the argument that this method belongs in j.l.Runtime. To say that this method should go there because it's an instruction to the run-time system is pretty weak. I agree with Vitaly [1] that if that's the threshold for adding methods to the Runtime class then lots of other stuff belongs there as well, including much of what's now in java.lang.Thread and java.util.concurrent and, arguably, anything else related to interacting with the environment in which the application runs (file and network I/O, process manipulation, etc.). This thread-related method really belongs in either java.lang.Thread or java.util.concurrent.LockSupport. j.l.Thread already has plenty of expert-level static methods related to the current thread, one of which (Thread::yield) is even a hint, just like this one. j.u.c.LockSupport is even more obviously intended for expert users and hence may be the best choice, but I could live with either one. - Mark [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-January/038400.html From david.lloyd at redhat.com Mon Feb 22 18:24:38 2016 From: david.lloyd at redhat.com (David M. Lloyd) Date: Mon, 22 Feb 2016 12:24:38 -0600 Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal In-Reply-To: <20160222101133.946427992eggemoggin.niobe.net> References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com> <561428BC.3060806@cs.oswego.edu> <015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com> <56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu> <54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com> <561FC6F9.1080701@cs.oswego.edu> <25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com> <562BC968.70603@cs.oswego.edu> <8917A25C-4A16-4291-B145-FA7180E09F78@azul.com> <4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com> <20151207093440.607909@eggemoggin.niobe.net> <20160222101133.946427992eggemoggin.niobe.net> Message-ID: <56CB5266.70903@redhat.com> On 02/22/2016 12:11 PM, mark.reinhold at oracle.com wrote: > 2016/1/28 9:25 -0800, gil at azul.com: >> This thread seems to have "hopped away" to the concurrency-interest >> list in mid-Dec-2015. This posting is intended to capture a summary of >> reasoning and some of the discussion there so that we have it in the >> record in core-libs-dev. Mostly by including the contents of several >> posts in the continuations of the original thread. >> >> See thread continuations here: >> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576 >> and here: >> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580 >> >> Summary: >> >> ... > > Thanks for the summary. > > I still don't buy the argument that this method belongs in j.l.Runtime. > > To say that this method should go there because it's an instruction to > the run-time system is pretty weak. I agree with Vitaly [1] that if > that's the threshold for adding methods to the Runtime class then lots > of other stuff belongs there as well, including much of what's now in > java.lang.Thread and java.util.concurrent and, arguably, anything else > related to interacting with the environment in which the application > runs (file and network I/O, process manipulation, etc.). > > This thread-related method really belongs in either java.lang.Thread or > java.util.concurrent.LockSupport. j.l.Thread already has plenty of > expert-level static methods related to the current thread, one of which > (Thread::yield) is even a hint, just like this one. j.u.c.LockSupport > is even more obviously intended for expert users and hence may be the > best choice, but I could live with either one. I agree, and I would also point out (perhaps redundantly?) that there was talk on concurrency-interest of possibly relocating or replicating the park/unpark statics into j.l.Thread, which further strengthens the idea that Thread is the best place for things like this. -- - DML From ivan.gerasimov at oracle.com Mon Feb 22 19:20:22 2016 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Mon, 22 Feb 2016 22:20:22 +0300 Subject: RFR: 8149330: Capacity of StringBuilder should not get close to Integer.MAX_VALUE unless necessary Message-ID: <56CB5F76.3030102@oracle.com> Hello! When the capacity of a StringBuilder needs to be increased (either due to append() or due to explicit call to ensureCapacity()), the new capacity is calculated as "twice the old capacity, plus 2", rounded down to Integer.MAX_VALUE: http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int- Because of that, StringBuilder throws OOM early, even though there may be room to grow. The proposed solution is to reserve a few bytes at the top of the range and only try to allocate them if absolutely required. The regression test is @ignored by default, as it is too greedy for memory. Would you please help review the fix? BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330 WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/ Sincerely yours, Ivan From martinrb at google.com Mon Feb 22 19:49:41 2016 From: martinrb at google.com (Martin Buchholz) Date: Mon, 22 Feb 2016 11:49:41 -0800 Subject: RFR: 8149330: Capacity of StringBuilder should not get close to Integer.MAX_VALUE unless necessary In-Reply-To: <56CB5F76.3030102@oracle.com> References: <56CB5F76.3030102@oracle.com> Message-ID: Can you make the code look more like the recently optimized code in ArrayList and Vector? On Mon, Feb 22, 2016 at 11:20 AM, Ivan Gerasimov wrote: > Hello! > > When the capacity of a StringBuilder needs to be increased (either due to > append() or due to explicit call to ensureCapacity()), the new capacity is > calculated as "twice the old capacity, plus 2", rounded down to > Integer.MAX_VALUE: > http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int- > Because of that, StringBuilder throws OOM early, even though there may be > room to grow. > > The proposed solution is to reserve a few bytes at the top of the range and > only try to allocate them if absolutely required. > > The regression test is @ignored by default, as it is too greedy for memory. > > Would you please help review the fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330 > WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/ > > Sincerely yours, > Ivan From paul.sandoz at oracle.com Mon Feb 22 20:22:43 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 22 Feb 2016 21:22:43 +0100 Subject: RFR(S): 8150360: augment/correct MethodHandle API documentation In-Reply-To: <4966FDEE-26F9-4343-9FC1-07CCEF5C410E@oracle.com> References: <4966FDEE-26F9-4343-9FC1-07CCEF5C410E@oracle.com> Message-ID: <20DEA5F7-F8C7-44A0-9CF1-67664274C5D8@oracle.com> > On 22 Feb 2016, at 12:40, Michael Haupt wrote: > > Dear all, > > please review this (doc-only) fix. > Bug: https://bugs.openjdk.java.net/browse/JDK-8150360 > Webrev: http://cr.openjdk.java.net/~mhaupt/8150360/webrev.00 > > The fix addresses one contradiction in the API documentation of MethodHandles.tryFinally(), and adds the following note to the API documentation of the method handle combinators given below: > > + *

    > + * Note: The resulting adapter is never a {@linkplain MethodHandle#asVarargsCollector > + * variable-arity method handle}, even if the original target method handle was. > > * MethodHandle.asCollector(), .bindTo() > * MethodHandles.collectArguments(), .dropArguments(), .filterArguments(), .filterReturnValue(), .foldArguments(), .permuteArguments() > Looks good. Minor thing, no need for another webrev round. MethodHandles ? 3849 * The {@code target} and {@code cleanup} handles must have the same corresponding argument and return types, except 3850 * that the {@code cleanup} handle may omit trailing arguments. Also, the {@code cleanup} must have one or two extra 3851 * leading parameters:

      Either ?Also, the {@code cleanup} handle must...? or ?Also, {@code cleanup} must?? ? Paul. From vitalyd at gmail.com Mon Feb 22 20:43:17 2016 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Mon, 22 Feb 2016 15:43:17 -0500 Subject: RFR: 8149330: Capacity of StringBuilder should not get close to Integer.MAX_VALUE unless necessary In-Reply-To: <56CB5F76.3030102@oracle.com> References: <56CB5F76.3030102@oracle.com> Message-ID: 165 final int SAFE_BOUND = (MAX_ARRAY_SIZE >> coder); 166 if (((SAFE_BOUND - newCapacity) | newCapacity) < 0) { Do the hotspot compiler engineers know you guys are doing manual branch elimination like this? :) On Mon, Feb 22, 2016 at 2:20 PM, Ivan Gerasimov wrote: > Hello! > > When the capacity of a StringBuilder needs to be increased (either due to > append() or due to explicit call to ensureCapacity()), the new capacity is > calculated as "twice the old capacity, plus 2", rounded down to > Integer.MAX_VALUE: > > http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int- > Because of that, StringBuilder throws OOM early, even though there may be > room to grow. > > The proposed solution is to reserve a few bytes at the top of the range > and only try to allocate them if absolutely required. > > The regression test is @ignored by default, as it is too greedy for memory. > > Would you please help review the fix? > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330 > WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/ > > Sincerely yours, > Ivan > From paul.sandoz at oracle.com Mon Feb 22 20:45:49 2016 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 22 Feb 2016 21:45:49 +0100 Subject: RFR(S): 8143410: augment pseudo-code descriptions in MethodHandles API In-Reply-To: <4C5FD37F-D976-414F-97FA-4EB9731A7675@oracle.com> References: <4C5FD37F-D976-414F-97FA-4EB9731A7675@oracle.com> Message-ID: > On 22 Feb 2016, at 15:08, Michael Haupt wrote: > > Dear all, > > please review this (doc-only) fix. > Bug: https://bugs.openjdk.java.net/browse/JDK-8143410 > Webrev: http://cr.openjdk.java.net/~mhaupt/8143410/webrev.00 > > The change documents the type variables used in the pseudo-code in the API documentation of the following methods in MethodHandles: catchException(), collectArguments(), filterArguments(), filterReturnValue(), foldArguments(), and guardWithTest(). > Just minor stuff: 2642 *

      Here is pseudocode for the resulting adapter. In the code, {@code T} 2643 * denotes the return type of both the {@code target} and resulting adapter. 2644 * {@code P}/{@code p} and {@code B}/{@code b} represent the types / values 2645 * of the parameters / arguments that precede / follow the filter position 2646 * {@code pos}. {@code A[i]}/{@code a[i]} stand for the types / values of "precede / follow ..." -> "proceed and follow ?, respectively? ? like that for the last modified method doc. 2884 * } 2885 *

      Here is pseudocode for the resulting adapter. In the code, {@code V} 2886 * represents the result type of the {@code target}; {@code T}, that of the 2887 * {@code filter}; and {@code A}/{@code a}, the types / values of the 2888 * parameters / arguments of the {@code target} as well as the resulting 2889 * adapter. 2890 * 

      {@code
2891      * V target(A...);
2892      * T filter(V);


Should target return a type of T, and filter V for consistency with the other pseudocode examples? That might be more changes than you wanna apply in this case :-)


3123      * Here is pseudocode for the resulting adapter. In the code, {@code V}

"{@code V}? -> "{@code T}?


3124      * represents the return type of the {@code target} and {@code handler},
3125      * and correspondingly that of the resulting adapter; {@code A}/{@code a},
3126      * the types and values of arguments to the resulting handle consumed by
3127      * {@code handler}; and {@code B}/{@code b}, those of arguments to the
3128      * resulting handle discarded by {@code handler}.
3129      * 
      {@code
3130      * T target(A..., B...);
3131      * T handler(ExType, A...);
3132      * T adapter(A... a, B... b) {
3133      *   try {
3134      *     return target(a..., b...);
3135      *   } catch (ExType ex) {
3136      *     return handler(ex, a...);
3137      *   }
3138      * }
3139      * }
      


Paul.

> Thanks,
> 
> Michael
> 
> --
> 
> 
> Dr. Michael Haupt | Principal Member of Technical Staff
> Phone: +49 331 200 7277 | Fax: +49 331 200 7561
> Oracle Java Platform Group | LangTools Team | Nashorn
> Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany
> 
> ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
> Registergericht: Amtsgericht M?nchen, HRA 95603
> 
> Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
> Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
> Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
> 	Oracle is committed to developing practices and products that help protect the environment
> 



From Roger.Riggs at Oracle.com  Mon Feb 22 21:22:25 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 22 Feb 2016 16:22:25 -0500
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: <56CACBCC.6070609@oracle.com>
References: <56BCED89.7040007@oracle.com> <56CACBCC.6070609@oracle.com>
Message-ID: <56CB7C11.1040904@Oracle.com>

Hi Nadeesh,

Sorry for the delay.

DateTimeFrmatterBuilder.java: 3387:  avoid testing isStrict twice.
   - refactor the if's so there is an outer if for context.isStrict == 
false and
     inner if's  (or tertiary ?: assignment) for the replacement parseType.
    if (context.isStrict() == false) {
       if ( type == 6) {
         parseType = 10;
       } else if (type == 5) {
         parseType = 9;
       }
    }


Otherwise looks fine.

Thanks Roger



On 2/22/2016 3:50 AM, nadeesh tv wrote:
> Gentle Reminder
> On 2/12/2016 1:52 AM, nadeesh tv wrote:
>> Hi all,
>>
>> Please review a fix for
>>
>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>
>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>
>



From mandy.chung at oracle.com  Mon Feb 22 21:41:45 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 22 Feb 2016 13:41:45 -0800
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CA080F.6010308@gmail.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com>
	<56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
Message-ID: 


> On Feb 21, 2016, at 10:55 AM, Peter Levart  wrote:
> 
> Hi Kim, Roger,
> 
> I have prepared another webrev which addresses your outstanding concerns:
> 
> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.04/
> 

MethodHandleNatives.java
   line 71-74 Nit: can you break the long lines 

> I added new method to Cleaner:
> 
>    public boolean helpClean() { }
> 

I?m in two minds of making this a public method. An explicit way to enqueue pending references as well as invoke Cleanable::clean.  If it goes in, the method name needs to be renamed.

The existing way to do that is to register phantom references in your own ReferenceQueue and then drain the queue at appropriate point.  Would you consider having a method to return ReferenceQueue maintained by the cleaner instead?

Other than this new method, the change looks good to me.

Mandy

> I think this form of method that just does one quantum of work and returns a boolean indicating whether there's more work waiting is a better fit for some clients that might want to do just a limited amount of work at once (like for example sun.java2d.Disposer.pollRemove that you mentioned). This method also deals with helping the ReferenceHandler thread, which is necessary to be able to "squeeze" out all outstanding work. As Cleaner is in the same package as Reference and helping ReferenceHandler thread is implicitly included in Cleaner.helpClean(), the JavaLangRefAccess interface and a field in SharedSecrets can be removed.
> 
> I also simplified the API in sun.nio.fs.NativeBuffer and replaced the accessor of Cleanable with a simple void free() method (called free because it deallocates memory).
> 
> I think this will have to be submitted to CCC for approval, right? Can you help me with it?
> 
> 
> Regards, Peter
> 
> On 02/17/2016 11:34 PM, Kim Barrett wrote:
>>> On Feb 17, 2016, at 4:06 AM, Peter Levart  wrote:
>>> 
>>> Hi Kim,
>>> 
>>> Thanks for looking into this. Answers inline?
>> Peter,
>> 
>> Thanks for the explanations.
>> 
>>> On 02/17/2016 01:20 AM, Kim Barrett wrote:
>>>>> However, I recall Roger saying there were existing tests that
>>>> failed when certain uses of sun.misc.Cleaner were replaced with
>>>> java.lang.ref.Cleaner. I don't remember the details, but maybe Roger
>>>> does.
>>> If the failing test was the following:
>>> 
>>> java/nio/Buffer/DirectBufferAllocTest.java
>>> 
>>> ...then it has been taken care of (see Bits.java).
>> That looks familiar. And yes, I see what you did there, and I don't
>> think Roger's initial prototype testing did anything similar, so
>> indeed this is likely the failure he encountered.
>> 
>> Though I'm still inclined to object to that form of drainQueue (see
>> below).
>> 
>>>> I don't understand why CleanerImpl needs to be changed to public in
>>>> order to provide access to the new drainQueue. Wouldn't it be better
>>>> to add Cleaner.drainQueue?
>>> An interesting idea. But I don't know if such functionality is generally useful enough to commit to it in a public API.
>> java.desktop:sun.java2d.Disposer.pollRemove seems to me to be
>> addressing an essentially similar requirement, with
>> java.desktop:sun.font.StrikeCache being the user of that API.
>> 
>> Of course, that's already got all the scaffolding for using phantom
>> references, and there's no need to rewrite it to use
>> java.lang.ref.Cleaner.  But maybe there are others like this?
>> 
>> In any case, I really dislike the approach of exposing the CleanerImpl
>> object to get at this functionality.
>> 
>>>> Some of the other changes here don't seem related to the problem at
>>>> hand. ...
>>> One thing that this change unfortunately requires is to get rid of lambdas and method references in the implementation and dependencies of java.land.ref.Cleaner API, because it gets used early in the boot-up sequence when java.lang.invoke infrastructure is not ready yet.
>> Oh, right!  Bootstrapping issues!
>> 
>>> The alternative to CleanerCleanable is a no-op Runnable implementation passed to PhantomCleanableRef constructor. ?
>> OK.  Now I understand.
>> 
>>> Making CleanerImpl implement Runnable ?
>> I'd be fine with this if the CleanerImpl wasn't exposed as part of the
>> drainQueue functionality.
>> 
>>>> ------------------------------------------------------------------------------
>>>> src/java.base/share/classes/sun/nio/fs/NativeBuffer.java
>>>>   76     Cleaner.Cleanable cleanable() {
>>>>   77         return cleanable;
>>>>   78     }
>>>> 
>>>> [pre-existing, but if we're changing things anyway...]
>>>> 
>>>> I'm kind of surprised by an accessor function (both here and in the
>>>> original code) rather than a cleanup function.  Is there a use case
>>>> for anything other than buffer.cleanable().clean()?
>>> It can't be, since both old Cleaner and new Cleanable have only got a single method - clean().
>> So this could be replaced with
>> 
>>   void clean() {
>>     cleanable.clean();
>>   }
>> 
>> To me, that seems better.
>> 
>>>> Similarly for the DirectBuffer interface.
>>> This one is a critical method, used by various 3rd party softwares...
>> I want to cover my ears when people start talking about some of the
>> things that have done?  OK.
>> 
>> 
>> 
>> 
> 



From Roger.Riggs at Oracle.com  Mon Feb 22 21:56:15 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 22 Feb 2016 16:56:15 -0500
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: 
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
Message-ID: <56CB83FF.4010808@Oracle.com>

Hi Mandy,

On 2/22/2016 4:41 PM, Mandy Chung wrote:
>> On Feb 21, 2016, at 10:55 AM, Peter Levart  wrote:
>>

>> I added new method to Cleaner:
>>
>>     public boolean helpClean() { }
>>
> I?m in two minds of making this a public method. An explicit way to enqueue pending references as well as invoke Cleanable::clean.  If it goes in, the method name needs to be renamed.
I would suggest a name related to cleaning a single Cleanable.
>
> The existing way to do that is to register phantom references in your own ReferenceQueue and then drain the queue at appropriate point.  Would you consider having a method to return ReferenceQueue maintained by the cleaner instead?
If the queue is exposed, there is no assurance that the cleanable 
function would be called.
Any caller could introduce a bug by not doing the proper cleaning.

I was more concerned with the crossing of Reference.tryHandlePending 
with the cleaning thread.
The method description does not mention anything related to the 
Reference processing thread
though that is all implementation.  The @implNote might be a bit more 
concise and less informal.

Roger

>
> Other than this new method, the change looks good to me.
>
> Mandy
>
>> I think this form of method that just does one quantum of work and returns a boolean indicating whether there's more work waiting is a better fit for some clients that might want to do just a limited amount of work at once (like for example sun.java2d.Disposer.pollRemove that you mentioned). This method also deals with helping the ReferenceHandler thread, which is necessary to be able to "squeeze" out all outstanding work. As Cleaner is in the same package as Reference and helping ReferenceHandler thread is implicitly included in Cleaner.helpClean(), the JavaLangRefAccess interface and a field in SharedSecrets can be removed.
>>
>> I also simplified the API in sun.nio.fs.NativeBuffer and replaced the accessor of Cleanable with a simple void free() method (called free because it deallocates memory).
>>
>> I think this will have to be submitted to CCC for approval, right? Can you help me with it?
>>
>>
>> Regards, Peter
>>
>> On 02/17/2016 11:34 PM, Kim Barrett wrote:
>>>> On Feb 17, 2016, at 4:06 AM, Peter Levart  wrote:
>>>>
>>>> Hi Kim,
>>>>
>>>> Thanks for looking into this. Answers inline?
>>> Peter,
>>>
>>> Thanks for the explanations.
>>>
>>>> On 02/17/2016 01:20 AM, Kim Barrett wrote:
>>>>>> However, I recall Roger saying there were existing tests that
>>>>> failed when certain uses of sun.misc.Cleaner were replaced with
>>>>> java.lang.ref.Cleaner. I don't remember the details, but maybe Roger
>>>>> does.
>>>> If the failing test was the following:
>>>>
>>>> java/nio/Buffer/DirectBufferAllocTest.java
>>>>
>>>> ...then it has been taken care of (see Bits.java).
>>> That looks familiar. And yes, I see what you did there, and I don't
>>> think Roger's initial prototype testing did anything similar, so
>>> indeed this is likely the failure he encountered.
>>>
>>> Though I'm still inclined to object to that form of drainQueue (see
>>> below).
>>>
>>>>> I don't understand why CleanerImpl needs to be changed to public in
>>>>> order to provide access to the new drainQueue. Wouldn't it be better
>>>>> to add Cleaner.drainQueue?
>>>> An interesting idea. But I don't know if such functionality is generally useful enough to commit to it in a public API.
>>> java.desktop:sun.java2d.Disposer.pollRemove seems to me to be
>>> addressing an essentially similar requirement, with
>>> java.desktop:sun.font.StrikeCache being the user of that API.
>>>
>>> Of course, that's already got all the scaffolding for using phantom
>>> references, and there's no need to rewrite it to use
>>> java.lang.ref.Cleaner.  But maybe there are others like this?
>>>
>>> In any case, I really dislike the approach of exposing the CleanerImpl
>>> object to get at this functionality.
>>>
>>>>> Some of the other changes here don't seem related to the problem at
>>>>> hand. ...
>>>> One thing that this change unfortunately requires is to get rid of lambdas and method references in the implementation and dependencies of java.land.ref.Cleaner API, because it gets used early in the boot-up sequence when java.lang.invoke infrastructure is not ready yet.
>>> Oh, right!  Bootstrapping issues!
>>>
>>>> The alternative to CleanerCleanable is a no-op Runnable implementation passed to PhantomCleanableRef constructor. ?
>>> OK.  Now I understand.
>>>
>>>> Making CleanerImpl implement Runnable ?
>>> I'd be fine with this if the CleanerImpl wasn't exposed as part of the
>>> drainQueue functionality.
>>>
>>>>> ------------------------------------------------------------------------------
>>>>> src/java.base/share/classes/sun/nio/fs/NativeBuffer.java
>>>>>    76     Cleaner.Cleanable cleanable() {
>>>>>    77         return cleanable;
>>>>>    78     }
>>>>>
>>>>> [pre-existing, but if we're changing things anyway...]
>>>>>
>>>>> I'm kind of surprised by an accessor function (both here and in the
>>>>> original code) rather than a cleanup function.  Is there a use case
>>>>> for anything other than buffer.cleanable().clean()?
>>>> It can't be, since both old Cleaner and new Cleanable have only got a single method - clean().
>>> So this could be replaced with
>>>
>>>    void clean() {
>>>      cleanable.clean();
>>>    }
>>>
>>> To me, that seems better.
>>>
>>>>> Similarly for the DirectBuffer interface.
>>>> This one is a critical method, used by various 3rd party softwares...
>>> I want to cover my ears when people start talking about some of the
>>> things that have done?  OK.
>>>
>>>
>>>
>>>



From mandy.chung at oracle.com  Mon Feb 22 22:01:08 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Mon, 22 Feb 2016 14:01:08 -0800
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CB83FF.4010808@Oracle.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com>
	<56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com>
Message-ID: 


> On Feb 22, 2016, at 1:56 PM, Roger Riggs  wrote:
> 
> Hi Mandy,
> 
> On 2/22/2016 4:41 PM, Mandy Chung wrote:
>>> On Feb 21, 2016, at 10:55 AM, Peter Levart  wrote:
>>> 
> 
>>> I added new method to Cleaner:
>>> 
>>>    public boolean helpClean() { }
>>> 
>> I?m in two minds of making this a public method. An explicit way to enqueue pending references as well as invoke Cleanable::clean.  If it goes in, the method name needs to be renamed.
> I would suggest a name related to cleaning a single Cleanable.
>> 
>> The existing way to do that is to register phantom references in your own ReferenceQueue and then drain the queue at appropriate point.  Would you consider having a method to return ReferenceQueue maintained by the cleaner instead?
> If the queue is exposed, there is no assurance that the cleanable function would be called.
> Any caller could introduce a bug by not doing the proper cleaning.
> 
> I was more concerned with the crossing of Reference.tryHandlePending with the cleaning thread.
> The method description does not mention anything related to the Reference processing thread
> though that is all implementation.  The @implNote might be a bit more concise and less informal.
> 

I agree the spec needs some work if it?s a public method.  This seems better to file a RFE to follow up.

Mandy

> Roger
> 
>> 
>> Other than this new method, the change looks good to me.
>> 
>> Mandy
>> 
>>> I think this form of method that just does one quantum of work and returns a boolean indicating whether there's more work waiting is a better fit for some clients that might want to do just a limited amount of work at once (like for example sun.java2d.Disposer.pollRemove that you mentioned). This method also deals with helping the ReferenceHandler thread, which is necessary to be able to "squeeze" out all outstanding work. As Cleaner is in the same package as Reference and helping ReferenceHandler thread is implicitly included in Cleaner.helpClean(), the JavaLangRefAccess interface and a field in SharedSecrets can be removed.
>>> 
>>> I also simplified the API in sun.nio.fs.NativeBuffer and replaced the accessor of Cleanable with a simple void free() method (called free because it deallocates memory).
>>> 
>>> I think this will have to be submitted to CCC for approval, right? Can you help me with it?
>>> 
>>> 
>>> Regards, Peter
>>> 
>>> On 02/17/2016 11:34 PM, Kim Barrett wrote:
>>>>> On Feb 17, 2016, at 4:06 AM, Peter Levart  wrote:
>>>>> 
>>>>> Hi Kim,
>>>>> 
>>>>> Thanks for looking into this. Answers inline?
>>>> Peter,
>>>> 
>>>> Thanks for the explanations.
>>>> 
>>>>> On 02/17/2016 01:20 AM, Kim Barrett wrote:
>>>>>>> However, I recall Roger saying there were existing tests that
>>>>>> failed when certain uses of sun.misc.Cleaner were replaced with
>>>>>> java.lang.ref.Cleaner. I don't remember the details, but maybe Roger
>>>>>> does.
>>>>> If the failing test was the following:
>>>>> 
>>>>> java/nio/Buffer/DirectBufferAllocTest.java
>>>>> 
>>>>> ...then it has been taken care of (see Bits.java).
>>>> That looks familiar. And yes, I see what you did there, and I don't
>>>> think Roger's initial prototype testing did anything similar, so
>>>> indeed this is likely the failure he encountered.
>>>> 
>>>> Though I'm still inclined to object to that form of drainQueue (see
>>>> below).
>>>> 
>>>>>> I don't understand why CleanerImpl needs to be changed to public in
>>>>>> order to provide access to the new drainQueue. Wouldn't it be better
>>>>>> to add Cleaner.drainQueue?
>>>>> An interesting idea. But I don't know if such functionality is generally useful enough to commit to it in a public API.
>>>> java.desktop:sun.java2d.Disposer.pollRemove seems to me to be
>>>> addressing an essentially similar requirement, with
>>>> java.desktop:sun.font.StrikeCache being the user of that API.
>>>> 
>>>> Of course, that's already got all the scaffolding for using phantom
>>>> references, and there's no need to rewrite it to use
>>>> java.lang.ref.Cleaner.  But maybe there are others like this?
>>>> 
>>>> In any case, I really dislike the approach of exposing the CleanerImpl
>>>> object to get at this functionality.
>>>> 
>>>>>> Some of the other changes here don't seem related to the problem at
>>>>>> hand. ...
>>>>> One thing that this change unfortunately requires is to get rid of lambdas and method references in the implementation and dependencies of java.land.ref.Cleaner API, because it gets used early in the boot-up sequence when java.lang.invoke infrastructure is not ready yet.
>>>> Oh, right!  Bootstrapping issues!
>>>> 
>>>>> The alternative to CleanerCleanable is a no-op Runnable implementation passed to PhantomCleanableRef constructor. ?
>>>> OK.  Now I understand.
>>>> 
>>>>> Making CleanerImpl implement Runnable ?
>>>> I'd be fine with this if the CleanerImpl wasn't exposed as part of the
>>>> drainQueue functionality.
>>>> 
>>>>>> ------------------------------------------------------------------------------
>>>>>> src/java.base/share/classes/sun/nio/fs/NativeBuffer.java
>>>>>>   76     Cleaner.Cleanable cleanable() {
>>>>>>   77         return cleanable;
>>>>>>   78     }
>>>>>> 
>>>>>> [pre-existing, but if we're changing things anyway...]
>>>>>> 
>>>>>> I'm kind of surprised by an accessor function (both here and in the
>>>>>> original code) rather than a cleanup function.  Is there a use case
>>>>>> for anything other than buffer.cleanable().clean()?
>>>>> It can't be, since both old Cleaner and new Cleanable have only got a single method - clean().
>>>> So this could be replaced with
>>>> 
>>>>   void clean() {
>>>>     cleanable.clean();
>>>>   }
>>>> 
>>>> To me, that seems better.
>>>> 
>>>>>> Similarly for the DirectBuffer interface.
>>>>> This one is a critical method, used by various 3rd party softwares...
>>>> I want to cover my ears when people start talking about some of the
>>>> things that have done?  OK.
>>>> 
>>>> 
>>>> 
>>>> 
> 



From ivan.gerasimov at oracle.com  Mon Feb 22 23:36:59 2016
From: ivan.gerasimov at oracle.com (Ivan Gerasimov)
Date: Tue, 23 Feb 2016 02:36:59 +0300
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: 
References: <56CB5F76.3030102@oracle.com>
	
Message-ID: <56CB9B9B.8070509@oracle.com>



On 22.02.2016 22:49, Martin Buchholz wrote:
> Can you make the code look more like the recently optimized code in
> ArrayList and Vector?

Sure.  Please find the updated webrev below.
Of course, there are nuances, as we need to deal with the compact 
strings, so the code isn't quite the same as in ArrayList.

http://cr.openjdk.java.net/~igerasim/8149330/01/webrev/

Sincerely yours,
Ivan

> On Mon, Feb 22, 2016 at 11:20 AM, Ivan Gerasimov
>  wrote:
>> Hello!
>>
>> When the capacity of a StringBuilder needs to be increased (either due to
>> append() or due to explicit call to ensureCapacity()), the new capacity is
>> calculated as "twice the old capacity, plus 2", rounded down to
>> Integer.MAX_VALUE:
>> http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int-
>> Because of that, StringBuilder throws OOM early, even though there may be
>> room to grow.
>>
>> The proposed solution is to reserve a few bytes at the top of the range and
>> only try to allocate them if absolutely required.
>>
>> The regression test is @ignored by default, as it is too greedy for memory.
>>
>> Would you please help review the fix?
>>
>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330
>> WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/
>>
>> Sincerely yours,
>> Ivan



From ivan.gerasimov at oracle.com  Mon Feb 22 23:40:55 2016
From: ivan.gerasimov at oracle.com (Ivan Gerasimov)
Date: Tue, 23 Feb 2016 02:40:55 +0300
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: 
References: <56CB5F76.3030102@oracle.com>
	
Message-ID: <56CB9C87.7070605@oracle.com>



On 22.02.2016 23:43, Vitaly Davidovich wrote:
> 165 final int SAFE_BOUND = (MAX_ARRAY_SIZE >> coder);
> 166 if (((SAFE_BOUND - newCapacity) | newCapacity) < 0) {
> Do the hotspot compiler engineers know you guys are doing manual 
> branch elimination like this? :)
>
>
Well, both these checks will be performed in the common case (and we 
know it in advance), so combining them together seems to be worthwhile.

Sincerely yours,
Ivan

> On Mon, Feb 22, 2016 at 2:20 PM, Ivan Gerasimov 
> > wrote:
>
>     Hello!
>
>     When the capacity of a StringBuilder needs to be increased (either
>     due to append() or due to explicit call to ensureCapacity()), the
>     new capacity is calculated as "twice the old capacity, plus 2",
>     rounded down to Integer.MAX_VALUE:
>     http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int-
>     Because of that, StringBuilder throws OOM early, even though there
>     may be room to grow.
>
>     The proposed solution is to reserve a few bytes at the top of the
>     range and only try to allocate them if absolutely required.
>
>     The regression test is @ignored by default, as it is too greedy
>     for memory.
>
>     Would you please help review the fix?
>
>     BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330
>     WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/
>     
>
>     Sincerely yours,
>     Ivan
>
>



From vitalyd at gmail.com  Tue Feb 23 00:00:15 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Mon, 22 Feb 2016 19:00:15 -0500
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: <56CB9C87.7070605@oracle.com>
References: <56CB5F76.3030102@oracle.com>
	
	<56CB9C87.7070605@oracle.com>
Message-ID: 

Yes, but my hope would be that code could be written in the
natural/"canonical" manner and the JIT could then optimize accordingly
(possibly based on the target).  I looked at 8u60 C2 generated asm and it
does not do a transformation similar to the manual elimination in your
code; I played around with similar looking C++ code using gcc 5.3 and clang
3.7, and they actually sometimes (depends on what's inside the if/else
blocks themselves) generate better code with the || present.

But really, this is the JIT's domain -- it knows the target arch, it knows
whether it's an OoO cpu or not, it knows whether (highly predictable)
branch elimination can be worthwhile, it can estimate whether logical OR
version (as written above) that carries a dependency on both sides of the
expression (granted dependencies are in registers/stack slots here) is
better than letting cpu run both sides in parallel (no data dependency),
and so on.  I'm not even entirely certain the above transform yields any
benefit on modern cpus, yet it contorts the code somewhat (this particular
example isn't too bad though); that's not even taking into account that
what follows this nano optimization is an Arrays.copyOf operation, which
will almost certainly dominate the cost here.

I recall seeing Martin doing similar gymnastics in the ArrayList/Vector
changes he alluded to, but my hope would be that code can be written in a
"canonical" manner and let the JIT optimize it as it sees fit.

Just my $.02 :)


On Mon, Feb 22, 2016 at 6:40 PM, Ivan Gerasimov 
wrote:

>
>
> On 22.02.2016 23:43, Vitaly Davidovich wrote:
>
> 165         final int SAFE_BOUND = (MAX_ARRAY_SIZE >> coder); 166         if (((SAFE_BOUND - newCapacity) | newCapacity) < 0) {
>
> Do the hotspot compiler engineers know you guys are doing manual branch
> elimination like this? :)
>
>
> Well, both these checks will be performed in the common case (and we know
> it in advance), so combining them together seems to be worthwhile.
>
> Sincerely yours,
> Ivan
>
> On Mon, Feb 22, 2016 at 2:20 PM, Ivan Gerasimov  > wrote:
>
>> Hello!
>>
>> When the capacity of a StringBuilder needs to be increased (either due to
>> append() or due to explicit call to ensureCapacity()), the new capacity is
>> calculated as "twice the old capacity, plus 2", rounded down to
>> Integer.MAX_VALUE:
>>
>> http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int-
>> Because of that, StringBuilder throws OOM early, even though there may be
>> room to grow.
>>
>> The proposed solution is to reserve a few bytes at the top of the range
>> and only try to allocate them if absolutely required.
>>
>> The regression test is @ignored by default, as it is too greedy for
>> memory.
>>
>> Would you please help review the fix?
>>
>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330
>> WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/
>>
>> Sincerely yours,
>> Ivan
>>
>
>
>


From michael.haupt at oracle.com  Tue Feb 23 06:18:39 2016
From: michael.haupt at oracle.com (Michael Haupt)
Date: Tue, 23 Feb 2016 07:18:39 +0100
Subject: RFR(S): 8150360: augment/correct MethodHandle API documentation
In-Reply-To: <20DEA5F7-F8C7-44A0-9CF1-67664274C5D8@oracle.com>
References: <4966FDEE-26F9-4343-9FC1-07CCEF5C410E@oracle.com>
	<20DEA5F7-F8C7-44A0-9CF1-67664274C5D8@oracle.com>
Message-ID: <773B50A1-FC00-4432-894C-DF5B87AC8E42@oracle.com>

Hi Paul,

> Am 22.02.2016 um 21:22 schrieb Paul Sandoz :
> Looks good. Minor thing, no need for another webrev round.
> 
> MethodHandles
> ?
> 
> 3849      * The {@code target} and {@code cleanup} handles must have the same corresponding argument and return types, except
> 3850      * that the {@code cleanup} handle may omit trailing arguments. Also, the {@code cleanup} must have one or two extra
> 3851      * leading parameters:
        
> 
> Either ?Also, the {@code cleanup} handle must...? or ?Also, {@code cleanup} must?? ?


thank you. Fixed and pushed.

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603

Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
 	Oracle is committed to developing practices and products that help protect the environment



From martinrb at google.com  Tue Feb 23 06:23:31 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 22 Feb 2016 22:23:31 -0800
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: <56CB9B9B.8070509@oracle.com>
References: <56CB5F76.3030102@oracle.com>
	
	<56CB9B9B.8070509@oracle.com>
Message-ID: 

Thanks, Ivan.

I see that the capacity growth x -> 2*x + 2 is mandated by the spec,
so we can't change that.  Growing by more than a factor of two may
mean that the "overflow-conscious" code I originally wrote may need
more checks than ArrayList, where growing by 3/2 provides some sanity.

---

Looking at the compact string code for the first time, I wonder that
replacing a char[] with a byte[] has effectively reduced the capacity
by 2, when non-LATIN1 is stored.  If you wanted to preserve the
maximum capacity, you would need to use a char[] for storage as
before?

----

Anyways, AbstractStringBuilder looks even trickier to get right than
ArrayList or Vector.

On Mon, Feb 22, 2016 at 3:36 PM, Ivan Gerasimov
 wrote:
>
>
> On 22.02.2016 22:49, Martin Buchholz wrote:
>>
>> Can you make the code look more like the recently optimized code in
>> ArrayList and Vector?
>
>
> Sure.  Please find the updated webrev below.
> Of course, there are nuances, as we need to deal with the compact strings,
> so the code isn't quite the same as in ArrayList.
>
> http://cr.openjdk.java.net/~igerasim/8149330/01/webrev/
>
> Sincerely yours,
> Ivan
>
>
>> On Mon, Feb 22, 2016 at 11:20 AM, Ivan Gerasimov
>>  wrote:
>>>
>>> Hello!
>>>
>>> When the capacity of a StringBuilder needs to be increased (either due to
>>> append() or due to explicit call to ensureCapacity()), the new capacity
>>> is
>>> calculated as "twice the old capacity, plus 2", rounded down to
>>> Integer.MAX_VALUE:
>>>
>>> http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int-
>>> Because of that, StringBuilder throws OOM early, even though there may be
>>> room to grow.
>>>
>>> The proposed solution is to reserve a few bytes at the top of the range
>>> and
>>> only try to allocate them if absolutely required.
>>>
>>> The regression test is @ignored by default, as it is too greedy for
>>> memory.
>>>
>>> Would you please help review the fix?
>>>
>>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330
>>> WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/
>>>
>>> Sincerely yours,
>>> Ivan
>
>


From martinrb at google.com  Tue Feb 23 06:38:40 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 22 Feb 2016 22:38:40 -0800
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: 
References: <56CB5F76.3030102@oracle.com>
	
	<56CB9B9B.8070509@oracle.com>
	
Message-ID: 

In your test you refer to "Utf8"  but there's no UTF-8 in use here.
You probably mean UTF-16?

You could port Vector/ArrayManagement that I wrote when fixing
ArrayList/Vector, but I see there are more existing tests for capacity
of StringB*er


From michael.haupt at oracle.com  Tue Feb 23 06:39:18 2016
From: michael.haupt at oracle.com (Michael Haupt)
Date: Tue, 23 Feb 2016 07:39:18 +0100
Subject: RFR(S): 8143410: augment pseudo-code descriptions in
	MethodHandles API
In-Reply-To: 
References: <4C5FD37F-D976-414F-97FA-4EB9731A7675@oracle.com>
	
Message-ID: <85EB885B-369F-4A1E-BE4B-022393838198@oracle.com>

Hi Paul,

thank you. I'm holding off on pushing; a new webrev is at http://cr.openjdk.java.net/~mhaupt/8143410/webrev.01. See below for two specific comments.

> Am 22.02.2016 um 21:45 schrieb Paul Sandoz :
>> Webrev: http://cr.openjdk.java.net/~mhaupt/8143410/webrev.00
>> 
>> The change documents the type variables used in the pseudo-code in the API documentation of the following methods in MethodHandles: catchException(), collectArguments(), filterArguments(), filterReturnValue(), foldArguments(), and guardWithTest().
> 
> Just minor stuff:
> 
> 2642      * 
        Here is pseudocode for the resulting adapter. In the code, {@code T}
> 2643      * denotes the return type of both the {@code target} and resulting adapter.
> 2644      * {@code P}/{@code p} and {@code B}/{@code b} represent the types / values
> 2645      * of the parameters / arguments that precede / follow the filter position
> 2646      * {@code pos}. {@code A[i]}/{@code a[i]} stand for the types / values of
> 
> 
> "precede / follow ..." -> "proceed and follow ?, respectively? ? like that for the last modified method doc.

That's "precede" as in "coming before in the list". "Proceed" wouldn't make sense in this context. Or did you mean "... respectively" should be appended? (In the webrev, I've applied the latter.)

> 2884      * }
      > 2885 *

      Here is pseudocode for the resulting adapter. In the code, {@code V} > 2886 * represents the result type of the {@code target}; {@code T}, that of the > 2887 * {@code filter}; and {@code A}/{@code a}, the types / values of the > 2888 * parameters / arguments of the {@code target} as well as the resulting > 2889 * adapter. > 2890 * 

      {@code
> 2891      * V target(A...);
> 2892      * T filter(V);
> 
> 
> Should target return a type of T, and filter V for consistency with the other pseudocode examples? That might be more changes than you wanna apply in this case :-)

You're right, the naming should be consistent. I've applied this throughout.

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603

Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
 	Oracle is committed to developing practices and products that help protect the environment



From xueming.shen at oracle.com  Tue Feb 23 07:01:35 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Mon, 22 Feb 2016 23:01:35 -0800
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: 
References: <56CB5F76.3030102@oracle.com>
	
	<56CB9B9B.8070509@oracle.com>
	
Message-ID: <56CC03CF.4000101@oracle.com>

On 2/22/16 10:23 PM, Martin Buchholz wrote:
> Thanks, Ivan.
>
> I see that the capacity growth x -> 2*x + 2 is mandated by the spec,
> so we can't change that.  Growing by more than a factor of two may
> mean that the "overflow-conscious" code I originally wrote may need
> more checks than ArrayList, where growing by 3/2 provides some sanity.
>
> ---
>
> Looking at the compact string code for the first time, I wonder that
> replacing a char[] with a byte[] has effectively reduced the capacity
> by 2, when non-LATIN1 is stored.  If you wanted to preserve the
> maximum capacity, you would need to use a char[] for storage as
> before?

 From certain perspective it's a kinda of "regression" that the maximum 
capacity for a non-latin1
buffer/builder is reduced by 2 . But arguably it's really an 
implementation detail that how big a
StringBuffer/Builder can really go, as the spec and the implementation 
don't/can't guarantee
you can really have a buffer/build with a Integer.MAX_VALUE capacity.  
On the other hand on
certain system you might be able to have a bigger buffer/builder for 
latin-1 only characters, as
it only requires half the space with the compact string implementation. 
That said, I was debating
whether or not the constructor (with the capacity parameter) should 
check the capacity, with the
assumption that the buffer/builder might be for non-latin1 input. But it 
doesn't like the check
will bring in any benefit...

Sherman

>
> ----
>
> Anyways, AbstractStringBuilder looks even trickier to get right than
> ArrayList or Vector.
>
> On Mon, Feb 22, 2016 at 3:36 PM, Ivan Gerasimov
>  wrote:
>>
>> On 22.02.2016 22:49, Martin Buchholz wrote:
>>> Can you make the code look more like the recently optimized code in
>>> ArrayList and Vector?
>>
>> Sure.  Please find the updated webrev below.
>> Of course, there are nuances, as we need to deal with the compact strings,
>> so the code isn't quite the same as in ArrayList.
>>
>> http://cr.openjdk.java.net/~igerasim/8149330/01/webrev/
>>
>> Sincerely yours,
>> Ivan
>>
>>
>>> On Mon, Feb 22, 2016 at 11:20 AM, Ivan Gerasimov
>>>  wrote:
>>>> Hello!
>>>>
>>>> When the capacity of a StringBuilder needs to be increased (either due to
>>>> append() or due to explicit call to ensureCapacity()), the new capacity
>>>> is
>>>> calculated as "twice the old capacity, plus 2", rounded down to
>>>> Integer.MAX_VALUE:
>>>>
>>>> http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int-
>>>> Because of that, StringBuilder throws OOM early, even though there may be
>>>> room to grow.
>>>>
>>>> The proposed solution is to reserve a few bytes at the top of the range
>>>> and
>>>> only try to allocate them if absolutely required.
>>>>
>>>> The regression test is @ignored by default, as it is too greedy for
>>>> memory.
>>>>
>>>> Would you please help review the fix?
>>>>
>>>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330
>>>> WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/
>>>>
>>>> Sincerely yours,
>>>> Ivan
>>



From paul.sandoz at oracle.com  Tue Feb 23 08:13:20 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Tue, 23 Feb 2016 09:13:20 +0100
Subject: RFR(S): 8143410: augment pseudo-code descriptions in
	MethodHandles API
In-Reply-To: <85EB885B-369F-4A1E-BE4B-022393838198@oracle.com>
References: <4C5FD37F-D976-414F-97FA-4EB9731A7675@oracle.com>
	
	<85EB885B-369F-4A1E-BE4B-022393838198@oracle.com>
Message-ID: <24AA7FEF-B174-4250-BC78-12A6D0A86A5D@oracle.com>


> On 23 Feb 2016, at 07:39, Michael Haupt  wrote:
> 
> Hi Paul,
> 
> thank you. I'm holding off on pushing; a new webrev is at http://cr.openjdk.java.net/~mhaupt/8143410/webrev.01. See below for two specific comments.
> 
>> Am 22.02.2016 um 21:45 schrieb Paul Sandoz :
>>> Webrev: http://cr.openjdk.java.net/~mhaupt/8143410/webrev.00
>>> 
>>> The change documents the type variables used in the pseudo-code in the API documentation of the following methods in MethodHandles: catchException(), collectArguments(), filterArguments(), filterReturnValue(), foldArguments(), and guardWithTest().
>> 
>> Just minor stuff:
>> 
>> 2642      * 
      Here is pseudocode for the resulting adapter. In the code, {@code T}
>> 2643      * denotes the return type of both the {@code target} and resulting adapter.
>> 2644      * {@code P}/{@code p} and {@code B}/{@code b} represent the types / values
>> 2645      * of the parameters / arguments that precede / follow the filter position
>> 2646      * {@code pos}. {@code A[i]}/{@code a[i]} stand for the types / values of
>> 
>> 
>> "precede / follow ..." -> "proceed and follow ?, respectively? ? like that for the last modified method doc.
> 
> That's "precede" as in "coming before in the list". "Proceed" wouldn't make sense in this context. Or did you mean "... respectively" should be appended? (In the webrev, I've applied the latter.)
> 

Oops, I typed ?proceed and follow ?, respectively? but meant ?precede and follow?, respectively? :-)

Tis a minor thing, I find the use of ?and? rather than ?/? flows better, given the already heavy use of ?/?.


>> 2884      * }
      >> 2885 *

      Here is pseudocode for the resulting adapter. In the code, {@code V} >> 2886 * represents the result type of the {@code target}; {@code T}, that of the >> 2887 * {@code filter}; and {@code A}/{@code a}, the types / values of the >> 2888 * parameters / arguments of the {@code target} as well as the resulting >> 2889 * adapter. >> 2890 * 

      {@code
>> 2891      * V target(A...);
>> 2892      * T filter(V);
>> 
>> 
>> Should target return a type of T, and filter V for consistency with the other pseudocode examples? That might be more changes than you wanna apply in this case :-)
> 
> You're right, the naming should be consistent. I've applied this throughout.
> 

Ok.

Paul.


From michael.haupt at oracle.com  Tue Feb 23 08:15:36 2016
From: michael.haupt at oracle.com (Michael Haupt)
Date: Tue, 23 Feb 2016 09:15:36 +0100
Subject: RFR(S): 8143410: augment pseudo-code descriptions in
	MethodHandles API
In-Reply-To: <24AA7FEF-B174-4250-BC78-12A6D0A86A5D@oracle.com>
References: <4C5FD37F-D976-414F-97FA-4EB9731A7675@oracle.com>
	
	<85EB885B-369F-4A1E-BE4B-022393838198@oracle.com>
	<24AA7FEF-B174-4250-BC78-12A6D0A86A5D@oracle.com>
Message-ID: 

Hi Paul,

thanks - all right; I'll consider using "and" rather than "/" and push the result.

Best,

Michael

> Am 23.02.2016 um 09:13 schrieb Paul Sandoz :
> 
>> 
>> On 23 Feb 2016, at 07:39, Michael Haupt  wrote:
>> 
>> Hi Paul,
>> 
>> thank you. I'm holding off on pushing; a new webrev is at http://cr.openjdk.java.net/~mhaupt/8143410/webrev.01. See below for two specific comments.
>> 
>>> Am 22.02.2016 um 21:45 schrieb Paul Sandoz :
>>>> Webrev: http://cr.openjdk.java.net/~mhaupt/8143410/webrev.00
>>>> 
>>>> The change documents the type variables used in the pseudo-code in the API documentation of the following methods in MethodHandles: catchException(), collectArguments(), filterArguments(), filterReturnValue(), foldArguments(), and guardWithTest().
>>> 
>>> Just minor stuff:
>>> 
>>> 2642      * 
      Here is pseudocode for the resulting adapter. In the code, {@code T}
>>> 2643      * denotes the return type of both the {@code target} and resulting adapter.
>>> 2644      * {@code P}/{@code p} and {@code B}/{@code b} represent the types / values
>>> 2645      * of the parameters / arguments that precede / follow the filter position
>>> 2646      * {@code pos}. {@code A[i]}/{@code a[i]} stand for the types / values of
>>> 
>>> 
>>> "precede / follow ..." -> "proceed and follow ?, respectively? ? like that for the last modified method doc.
>> 
>> That's "precede" as in "coming before in the list". "Proceed" wouldn't make sense in this context. Or did you mean "... respectively" should be appended? (In the webrev, I've applied the latter.)
>> 
> 
> Oops, I typed ?proceed and follow ?, respectively? but meant ?precede and follow?, respectively? :-)
> 
> Tis a minor thing, I find the use of ?and? rather than ?/? flows better, given the already heavy use of ?/?.
> 
> 
>>> 2884      * }
      >>> 2885 *

      Here is pseudocode for the resulting adapter. In the code, {@code V} >>> 2886 * represents the result type of the {@code target}; {@code T}, that of the >>> 2887 * {@code filter}; and {@code A}/{@code a}, the types / values of the >>> 2888 * parameters / arguments of the {@code target} as well as the resulting >>> 2889 * adapter. >>> 2890 * 

      {@code
>>> 2891      * V target(A...);
>>> 2892      * T filter(V);
>>> 
>>> 
>>> Should target return a type of T, and filter V for consistency with the other pseudocode examples? That might be more changes than you wanna apply in this case :-)
>> 
>> You're right, the naming should be consistent. I've applied this throughout.
>> 
> 
> Ok.
> 
> Paul.

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603

Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
 	Oracle is committed to developing practices and products that help protect the environment



From aleksey.shipilev at oracle.com  Tue Feb 23 10:24:41 2016
From: aleksey.shipilev at oracle.com (Aleksey Shipilev)
Date: Tue, 23 Feb 2016 13:24:41 +0300
Subject: RFR (S) 8150180: String.value contents should be trusted
In-Reply-To: <56C729D3.80404@oracle.com>
References: <56C702B6.5010903@oracle.com> <56C70DAA.6000806@oracle.com>
	<56C72407.2070301@oracle.com> <56C729D3.80404@oracle.com>
Message-ID: <56CC3369.2000804@oracle.com>

On 02/19/2016 05:42 PM, Vladimir Ivanov wrote:
>>    http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/
>>    http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/
> Looks good.

A few more reviews, please?

Cheers,
-Aleksey




From ivan.gerasimov at oracle.com  Tue Feb 23 11:39:27 2016
From: ivan.gerasimov at oracle.com (Ivan Gerasimov)
Date: Tue, 23 Feb 2016 14:39:27 +0300
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: 
References: <56CB5F76.3030102@oracle.com>
	
	<56CB9C87.7070605@oracle.com>
	
Message-ID: <56CC44EF.4070509@oracle.com>

Okay, Vitaly, you convinced me and I replaced the combined check with 
two ORed checks:

-        return (((SAFE_BOUND - newCapacity) | newCapacity) < 0)
+        return (newCapacity < 0 || SAFE_BOUND - newCapacity < 0)

The webrev was updated in place:
http://cr.openjdk.java.net/~igerasim/8149330/01/webrev/

Sincerely yours,
Ivan

On 23.02.2016 3:00, Vitaly Davidovich wrote:
> Yes, but my hope would be that code could be written in the 
> natural/"canonical" manner and the JIT could then optimize accordingly 
> (possibly based on the target).  I looked at 8u60 C2 generated asm and 
> it does not do a transformation similar to the manual elimination in 
> your code; I played around with similar looking C++ code using gcc 5.3 
> and clang 3.7, and they actually sometimes (depends on what's inside 
> the if/else blocks themselves) generate better code with the || present.
>
> But really, this is the JIT's domain -- it knows the target arch, it 
> knows whether it's an OoO cpu or not, it knows whether (highly 
> predictable) branch elimination can be worthwhile, it can estimate 
> whether logical OR version (as written above) that carries a 
> dependency on both sides of the expression (granted dependencies are 
> in registers/stack slots here) is better than letting cpu run both 
> sides in parallel (no data dependency), and so on.  I'm not even 
> entirely certain the above transform yields any benefit on modern 
> cpus, yet it contorts the code somewhat (this particular example isn't 
> too bad though); that's not even taking into account that what follows 
> this nano optimization is an Arrays.copyOf operation, which will 
> almost certainly dominate the cost here.
>
> I recall seeing Martin doing similar gymnastics in the 
> ArrayList/Vector changes he alluded to, but my hope would be that code 
> can be written in a "canonical" manner and let the JIT optimize it as 
> it sees fit.
>
> Just my $.02 :)
>
>
> On Mon, Feb 22, 2016 at 6:40 PM, Ivan Gerasimov 
> > wrote:
>
>
>
>     On 22.02.2016 23:43, Vitaly Davidovich wrote:
>>     165 final int SAFE_BOUND = (MAX_ARRAY_SIZE >> coder);
>>     166 if (((SAFE_BOUND - newCapacity) | newCapacity) < 0) {
>>     Do the hotspot compiler engineers know you guys are doing manual
>>     branch elimination like this? :)
>>
>>
>     Well, both these checks will be performed in the common case (and
>     we know it in advance), so combining them together seems to be
>     worthwhile.
>
>     Sincerely yours,
>     Ivan
>
>>     On Mon, Feb 22, 2016 at 2:20 PM, Ivan Gerasimov
>>     > wrote:
>>
>>         Hello!
>>
>>         When the capacity of a StringBuilder needs to be increased
>>         (either due to append() or due to explicit call to
>>         ensureCapacity()), the new capacity is calculated as "twice
>>         the old capacity, plus 2", rounded down to Integer.MAX_VALUE:
>>         http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int-
>>         Because of that, StringBuilder throws OOM early, even though
>>         there may be room to grow.
>>
>>         The proposed solution is to reserve a few bytes at the top of
>>         the range and only try to allocate them if absolutely required.
>>
>>         The regression test is @ignored by default, as it is too
>>         greedy for memory.
>>
>>         Would you please help review the fix?
>>
>>         BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330
>>         WEBREV:
>>         http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/
>>         
>>
>>         Sincerely yours,
>>         Ivan
>>
>>
>
>



From scolebourne at joda.org  Tue Feb 23 11:47:01 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Tue, 23 Feb 2016 11:47:01 +0000
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: <56BCED89.7040007@oracle.com>
References: <56BCED89.7040007@oracle.com>
Message-ID: 

Thanks for the changes.

In `DateTimeFormatter`, the code should be

.parseLenient()
.appendOffsetId()
.parseStrict()

and the same in the other case. This ensures that existing callers who
then embed the formatter in another formatter (like the
ZONED_DATE_TIME constant) are unaffected.


The logic for lenient parsing does not look right as it only handles
types 5 and 6. This table shows the mappings needed:

"+HH",  -> "+HHmmss" or "+HH:mm:ss"
"+HHmm",  -> "+HHmmss",
"+HH:mm",  -> "+HH:mm:ss",
"+HHMM",  -> "+HHmmss",
"+HH:MM",  -> "+HH:mm:ss",
"+HHMMss",  -> "+HHmmss",
"+HH:MM:ss",  -> "+HH:mm:ss",
"+HHMMSS",  -> "+HHmmss",
"+HH:MM:SS",  -> "+HH:mm:ss",
"+HHmmss",
"+HH:mm:ss",

Note that the "+HH" pattern is a special case, as we don't know
whether to use the colon or non-colon pattern. Whether to require
colon or not is based on whether the next character after the HH is a
colon or not.

Proposed appendOffsetId() Javadoc:

* Appends the zone offset, such as '+01:00', to the formatter.
* 
      
* This appends an instruction to format/parse the offset ID to the builder.
* This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
* See {@link #appendOffset(String, String)} for details on formatting
and parsing.

Proposed appendOffset(String, String) Javadoc:

* During parsing, the offset...

changed to:

* When parsing in strict mode, the input must contain the mandatory
and optional elements are defined by the specified pattern.
* If the offset cannot be parsed then an exception is thrown unless
the section of the formatter is optional.
* 
      
* When parsing in lenient mode, only the hours are mandatory - minutes
and seconds are optional.
* The colons are required if the specified pattern contains a colon.
* If the specified pattern is "+HH", the presence of colons is
determined by whether the character after the hour digits is a colon
or not.
* If the offset cannot be parsed then an exception is thrown unless
the section of the formatter is optional.

thanks and sorry for delay
Stephen



On 11 February 2016 at 20:22, nadeesh tv  wrote:
> Hi all,
>
> Please review a fix for
>
> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>
> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>
> --
> Thanks and Regards,
> Nadeesh TV
>


From claes.redestad at oracle.com  Tue Feb 23 14:39:24 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Tue, 23 Feb 2016 15:39:24 +0100
Subject: RFR (S) 8150180: String.value contents should be trusted
In-Reply-To: <56CC3369.2000804@oracle.com>
References: <56C702B6.5010903@oracle.com> <56C70DAA.6000806@oracle.com>
	<56C72407.2070301@oracle.com> <56C729D3.80404@oracle.com>
	<56CC3369.2000804@oracle.com>
Message-ID: <56CC6F1C.2050207@oracle.com>

Looks good to me, but I assume you mean to push 
http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.02 (which looks 
even better)?

Thanks!

/Claes

On 2016-02-23 11:24, Aleksey Shipilev wrote:
> On 02/19/2016 05:42 PM, Vladimir Ivanov wrote:
>>>     http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/
>>>     http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/
>> Looks good.
> A few more reviews, please?
>
> Cheers,
> -Aleksey
>
>



From aleksey.shipilev at oracle.com  Tue Feb 23 14:52:12 2016
From: aleksey.shipilev at oracle.com (Aleksey Shipilev)
Date: Tue, 23 Feb 2016 17:52:12 +0300
Subject: RFR (S) 8150180: String.value contents should be trusted
In-Reply-To: <56CC6F1C.2050207@oracle.com>
References: <56C702B6.5010903@oracle.com> <56C70DAA.6000806@oracle.com>
	<56C72407.2070301@oracle.com> <56C729D3.80404@oracle.com>
	<56CC3369.2000804@oracle.com> <56CC6F1C.2050207@oracle.com>
Message-ID: <56CC721C.7070008@oracle.com>

Yes, sorry, pushing these two:
  http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.02
  http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/

-Aleksey

On 02/23/2016 05:39 PM, Claes Redestad wrote:
> Looks good to me, but I assume you mean to push
> http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.02 (which looks
> even better)?
> 
> Thanks!
> 
> /Claes
> 
> On 2016-02-23 11:24, Aleksey Shipilev wrote:
>> On 02/19/2016 05:42 PM, Vladimir Ivanov wrote:
>>>>     http://cr.openjdk.java.net/~shade/8150180/webrev.jdk.01/
>>>>     http://cr.openjdk.java.net/~shade/8150180/webrev.hs.02/
>>> Looks good.
>> A few more reviews, please?
>>
>> Cheers,
>> -Aleksey
>>
>>
> 




From ivan.gerasimov at oracle.com  Tue Feb 23 15:20:26 2016
From: ivan.gerasimov at oracle.com (Ivan Gerasimov)
Date: Tue, 23 Feb 2016 18:20:26 +0300
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: 
References: <56CB5F76.3030102@oracle.com>
	
	<56CB9B9B.8070509@oracle.com>
	
Message-ID: <56CC78BA.5010409@oracle.com>



On 23.02.2016 9:23, Martin Buchholz wrote:
> Thanks, Ivan.
>
> I see that the capacity growth x -> 2*x + 2 is mandated by the spec,
> so we can't change that.  Growing by more than a factor of two may
> mean that the "overflow-conscious" code I originally wrote may need
> more checks than ArrayList, where growing by 3/2 provides some sanity.

Right.
Though checking the newCapacity is non-positive should suffice to catch 
overflow in *2+2 formula.

While writing this, I just noticed that I actually made a mistake when 
did newCapacity < 0 check.
This wouldn't catch the overflow when the oldCapacity happens to be 
Integer.MAX_VALUE (which is not possible with the current hotspot, but 
may become an issue one day).

The webrev is updated with this correction:
http://cr.openjdk.java.net/~igerasim/8149330/02/webrev/


To make myself confident with the fix, I extracted the new code for 
calculating new capacity and ran it in a loop for each value from 0 to 
Integer.MAX_VALUE and checked that the result is as expected.
I don't see how it can be turned into a test, however.  So, I created a 
test that just verifies the specific bug is fixed.

> ---
>
> Looking at the compact string code for the first time, I wonder that
> replacing a char[] with a byte[] has effectively reduced the capacity
> by 2, when non-LATIN1 is stored.  If you wanted to preserve the
> maximum capacity, you would need to use a char[] for storage as
> before?
Probably yes.
I don't plan to address this at the moment.
The goals of the fix is to make the code work better and to keep the 
solution local, so it can be easier to port it back to jdk8u.

Sincerely yours,
Ivan

> ----
>
> Anyways, AbstractStringBuilder looks even trickier to get right than
> ArrayList or Vector.
>
> On Mon, Feb 22, 2016 at 3:36 PM, Ivan Gerasimov
>  wrote:
>>
>> On 22.02.2016 22:49, Martin Buchholz wrote:
>>> Can you make the code look more like the recently optimized code in
>>> ArrayList and Vector?
>>
>> Sure.  Please find the updated webrev below.
>> Of course, there are nuances, as we need to deal with the compact strings,
>> so the code isn't quite the same as in ArrayList.
>>
>> http://cr.openjdk.java.net/~igerasim/8149330/01/webrev/
>>
>> Sincerely yours,
>> Ivan
>>
>>
>>> On Mon, Feb 22, 2016 at 11:20 AM, Ivan Gerasimov
>>>  wrote:
>>>> Hello!
>>>>
>>>> When the capacity of a StringBuilder needs to be increased (either due to
>>>> append() or due to explicit call to ensureCapacity()), the new capacity
>>>> is
>>>> calculated as "twice the old capacity, plus 2", rounded down to
>>>> Integer.MAX_VALUE:
>>>>
>>>> http://docs.oracle.com/javase/8/docs/api/java/lang/StringBuilder.html#ensureCapacity-int-
>>>> Because of that, StringBuilder throws OOM early, even though there may be
>>>> room to grow.
>>>>
>>>> The proposed solution is to reserve a few bytes at the top of the range
>>>> and
>>>> only try to allocate them if absolutely required.
>>>>
>>>> The regression test is @ignored by default, as it is too greedy for
>>>> memory.
>>>>
>>>> Would you please help review the fix?
>>>>
>>>> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8149330
>>>> WEBREV: http://cr.openjdk.java.net/~igerasim/8149330/00/webrev/
>>>>
>>>> Sincerely yours,
>>>> Ivan
>>



From Roger.Riggs at Oracle.com  Tue Feb 23 15:29:11 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Tue, 23 Feb 2016 10:29:11 -0500
Subject: RFR 9: 8150337 : sun/misc/SunMiscSignalTest.java failed intermittently
Message-ID: <56CC7AC7.6090004@Oracle.com>

Please review a test update to skip tests using SIGINT, SIGTERM, and 
SIGHUP when
run with those signals are ignored.  For example, using nohup or trap in 
a shell.
It also includes a small enhancement adding sun.misc.Signal.toString 
ease debugging.

Webrev:
http://cr.openjdk.java.net/~rriggs/webrev-nohup-8150337/index.html

Thanks, Roger



From Alan.Bateman at oracle.com  Tue Feb 23 15:33:57 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Tue, 23 Feb 2016 15:33:57 +0000
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: 
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
Message-ID: <56CC7BE5.6040600@oracle.com>



On 22/02/2016 21:41, Mandy Chung wrote:
> :
>> I added new method to Cleaner:
>>
>>     public boolean helpClean() { }
>>
> I?m in two minds of making this a public method. An explicit way to enqueue pending references as well as invoke Cleanable::clean.  If it goes in, the method name needs to be renamed.
>
I'm also not sure that it's a good idea to have this be part of the 
Cleaner API. If feels like it's exposing too much of the internals. Is 
it really needed?

-Alan.


From peter.levart at gmail.com  Tue Feb 23 16:35:22 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 23 Feb 2016 17:35:22 +0100
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CB83FF.4010808@Oracle.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com>
Message-ID: <56CC8A4A.9080303@gmail.com>

Hi Roger, Mandy,

Here's another webrev:

http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.05/

I renamed the method and reworded the specification. Is this better now?


On 02/22/2016 10:56 PM, Roger Riggs wrote:
> Hi Mandy,
>
> On 2/22/2016 4:41 PM, Mandy Chung wrote:
>
>>
>> The existing way to do that is to register phantom references in your 
>> own ReferenceQueue and then drain the queue at appropriate point.  
>> Would you consider having a method to return ReferenceQueue 
>> maintained by the cleaner instead?
> If the queue is exposed, there is no assurance that the cleanable 
> function would be called.
> Any caller could introduce a bug by not doing the proper cleaning.
>
> I was more concerned with the crossing of Reference.tryHandlePending 
> with the cleaning thread.
> The method description does not mention anything related to the 
> Reference processing thread
> though that is all implementation.  The @implNote might be a bit more 
> concise and less informal.
>
> Roger

Yes, ReferenceHandler thread is just an implementation detail. The 
specification of java.lang.Reference subclasses doesn't even mention it. 
It talks about GC enqueueing Reference objects:

"Suppose that the garbage collector determines at a certain point in time...
...At the same time or at some later time it will enqueue those 
newly-cleared weak references that are registered with reference queues."

So in this respect ReferenceHandler is just an extension of GC Reference 
discovery/enqueuing logic, delegated to a background thread on Java 
side. The Cleaner.cleanNextPending() method tries to be true to its 
specification - it tries to invoke next cleanup action for which GC has 
determined that its monitored object is phantom reachable without 
waiting for ReferenceHandler thread to transfer it from pending list to 
the queue.

Since Reference.tryHandlePending is just transfering Reference objects 
from one list to another and does that using two locks (the 
Reference.lock and the ReferenceQueue.lock) but never holds them both 
together or calls any outside code while holding any of the locks, 
there's no danger of dead-locking, if that was your concern.

Regards, Peter

>
>>
>> Other than this new method, the change looks good to me.
>>
>> Mandy
>>
>>> I think this form of method that just does one quantum of work and 
>>> returns a boolean indicating whether there's more work waiting is a 
>>> better fit for some clients that might want to do just a limited 
>>> amount of work at once (like for example 
>>> sun.java2d.Disposer.pollRemove that you mentioned). This method also 
>>> deals with helping the ReferenceHandler thread, which is necessary 
>>> to be able to "squeeze" out all outstanding work. As Cleaner is in 
>>> the same package as Reference and helping ReferenceHandler thread is 
>>> implicitly included in Cleaner.helpClean(), the JavaLangRefAccess 
>>> interface and a field in SharedSecrets can be removed.
>>>
>>> I also simplified the API in sun.nio.fs.NativeBuffer and replaced 
>>> the accessor of Cleanable with a simple void free() method (called 
>>> free because it deallocates memory).
>>>
>>> I think this will have to be submitted to CCC for approval, right? 
>>> Can you help me with it?
>>>
>>>
>>> Regards, Peter
>>>
>>> On 02/17/2016 11:34 PM, Kim Barrett wrote:
>>>>> On Feb 17, 2016, at 4:06 AM, Peter Levart  
>>>>> wrote:
>>>>>
>>>>> Hi Kim,
>>>>>
>>>>> Thanks for looking into this. Answers inline?
>>>> Peter,
>>>>
>>>> Thanks for the explanations.
>>>>
>>>>> On 02/17/2016 01:20 AM, Kim Barrett wrote:
>>>>>>> However, I recall Roger saying there were existing tests that
>>>>>> failed when certain uses of sun.misc.Cleaner were replaced with
>>>>>> java.lang.ref.Cleaner. I don't remember the details, but maybe Roger
>>>>>> does.
>>>>> If the failing test was the following:
>>>>>
>>>>> java/nio/Buffer/DirectBufferAllocTest.java
>>>>>
>>>>> ...then it has been taken care of (see Bits.java).
>>>> That looks familiar. And yes, I see what you did there, and I don't
>>>> think Roger's initial prototype testing did anything similar, so
>>>> indeed this is likely the failure he encountered.
>>>>
>>>> Though I'm still inclined to object to that form of drainQueue (see
>>>> below).
>>>>
>>>>>> I don't understand why CleanerImpl needs to be changed to public in
>>>>>> order to provide access to the new drainQueue. Wouldn't it be better
>>>>>> to add Cleaner.drainQueue?
>>>>> An interesting idea. But I don't know if such functionality is 
>>>>> generally useful enough to commit to it in a public API.
>>>> java.desktop:sun.java2d.Disposer.pollRemove seems to me to be
>>>> addressing an essentially similar requirement, with
>>>> java.desktop:sun.font.StrikeCache being the user of that API.
>>>>
>>>> Of course, that's already got all the scaffolding for using phantom
>>>> references, and there's no need to rewrite it to use
>>>> java.lang.ref.Cleaner.  But maybe there are others like this?
>>>>
>>>> In any case, I really dislike the approach of exposing the CleanerImpl
>>>> object to get at this functionality.
>>>>
>>>>>> Some of the other changes here don't seem related to the problem at
>>>>>> hand. ...
>>>>> One thing that this change unfortunately requires is to get rid of 
>>>>> lambdas and method references in the implementation and 
>>>>> dependencies of java.land.ref.Cleaner API, because it gets used 
>>>>> early in the boot-up sequence when java.lang.invoke infrastructure 
>>>>> is not ready yet.
>>>> Oh, right!  Bootstrapping issues!
>>>>
>>>>> The alternative to CleanerCleanable is a no-op Runnable 
>>>>> implementation passed to PhantomCleanableRef constructor. ?
>>>> OK.  Now I understand.
>>>>
>>>>> Making CleanerImpl implement Runnable ?
>>>> I'd be fine with this if the CleanerImpl wasn't exposed as part of the
>>>> drainQueue functionality.
>>>>
>>>>>> ------------------------------------------------------------------------------ 
>>>>>>
>>>>>> src/java.base/share/classes/sun/nio/fs/NativeBuffer.java
>>>>>>    76     Cleaner.Cleanable cleanable() {
>>>>>>    77         return cleanable;
>>>>>>    78     }
>>>>>>
>>>>>> [pre-existing, but if we're changing things anyway...]
>>>>>>
>>>>>> I'm kind of surprised by an accessor function (both here and in the
>>>>>> original code) rather than a cleanup function.  Is there a use case
>>>>>> for anything other than buffer.cleanable().clean()?
>>>>> It can't be, since both old Cleaner and new Cleanable have only 
>>>>> got a single method - clean().
>>>> So this could be replaced with
>>>>
>>>>    void clean() {
>>>>      cleanable.clean();
>>>>    }
>>>>
>>>> To me, that seems better.
>>>>
>>>>>> Similarly for the DirectBuffer interface.
>>>>> This one is a critical method, used by various 3rd party softwares...
>>>> I want to cover my ears when people start talking about some of the
>>>> things that have done?  OK.
>>>>
>>>>
>>>>
>>>>
>



From peter.levart at gmail.com  Tue Feb 23 17:04:01 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Tue, 23 Feb 2016 18:04:01 +0100
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CC7BE5.6040600@oracle.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CC7BE5.6040600@oracle.com>
Message-ID: <56CC9101.2090101@gmail.com>

Hi Alan,

On 02/23/2016 04:33 PM, Alan Bateman wrote:
>
>
> On 22/02/2016 21:41, Mandy Chung wrote:
>> :
>>> I added new method to Cleaner:
>>>
>>>     public boolean helpClean() { }
>>>
>> I?m in two minds of making this a public method. An explicit way to 
>> enqueue pending references as well as invoke Cleanable::clean.  If it 
>> goes in, the method name needs to be renamed.
>>
> I'm also not sure that it's a good idea to have this be part of the 
> Cleaner API. If feels like it's exposing too much of the internals. Is 
> it really needed?

Without it, the DirectBufferAllocTest fails.

As John Cage (the character from Ally McBeal) would say: "I'm puzzled!". 
My previous iterations used similar method on the internal CleanerImpl 
class and there were suggestions to expose the functionality as public API.

I don't think this method exposes too much of internals. In the sense 
where it would prevent any possible future rearrangement of the 
internals or evolving of the Cleaner API. It doesn't talk about any 
queue(s) or ReferenceHandler thread. It keeps the Cleaner implementation 
details private. It just enables giving CPU time to assist with cleanup 
and if used as hinted in @implNote, provides back-pressure to allocating 
threads. I think this is a nice way of solving the problem of balancing 
allocation with deallocation. The same problem concurrent GC algorithms 
have, like G1 or CMS. If they can not keep-up with allocation threads, 
they pause them and do the cleanup while the world is stopped.

Regards, Peter



From martinrb at google.com  Tue Feb 23 17:29:40 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 23 Feb 2016 09:29:40 -0800
Subject: RFR: 8149330: Capacity of StringBuilder should not get close to
	Integer.MAX_VALUE unless necessary
In-Reply-To: <56CC78BA.5010409@oracle.com>
References: <56CB5F76.3030102@oracle.com>
	
	<56CB9B9B.8070509@oracle.com>
	
	<56CC78BA.5010409@oracle.com>
Message-ID: 

On Tue, Feb 23, 2016 at 7:20 AM, Ivan Gerasimov
 wrote:
> While writing this, I just noticed that I actually made a mistake when did
> newCapacity < 0 check.
> This wouldn't catch the overflow when the oldCapacity happens to be
> Integer.MAX_VALUE (which is not possible with the current hotspot, but may
> become an issue one day).

Well done!

One interesting way that capacity may end up being Integer.MAX_VALUE
is if we switch to char[] for storage.  Then in LATIN1 mode you could
store Integer.MAX_VALUE elements even without help from hotspot!


From naoto.sato at oracle.com  Tue Feb 23 18:47:24 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Tue, 23 Feb 2016 10:47:24 -0800
Subject: RFR 8124977 cmdline encoding challenges on Windows
In-Reply-To: <56CB3099.4030108@oracle.com>
References: 
	
	<558C017A.2020708@oracle.com>
	
	<5651E9AA.9060908@oracle.com>
	
	
	<56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com>
	
	
	<56CB3099.4030108@oracle.com>
Message-ID: <56CCA93C.6020807@oracle.com>

Hello,

Sorry if this has already been discussed, but this is my first time 
looking at the fix. In java_props_md.c, sprops.sun_jnu_encoding is now 
always "UTF-8". Is it always the case? What if the system admin switches 
the locale for "non-Unicode" applications in the Windows control panel?

Naoto

On 2/22/16 8:00 AM, Kumar Srinivasan wrote:
>
> Hi Naoto, Sherman,  can you please take a look.
> I tested with the jprt build and test all tests pass.
>
> Hi Vladimir, et. al.,
>
> It appears that there has been more simplifications from the previous
> webrev.04. :-)
>
> It would've helped if you highlight the changes you have made from the
> previous
> revision, unfortunately this is one of the deficiencies of webrev.
>
> There are some inconsistencies in the coding conventions:
>
> parse_manifest.c
> + if (q == 0) return -1;
>
> we expect the return to be on the next line.
>
> similarly main.c
>
> if (0 == q)
> {
>
> I can fix those up. If I were to push this change, who should I
> attribute the
> changes to ? ie. in the Contributed-by: line of the commit info ?
> Please note these have to be email addresses of the contributors.
>
> Thanks
> Kumar
>
>> Hi Kumar,
>>
>> We posted another web review here:
>> http://cr.openjdk.java.net/~kshoop/8124977/webrev.05/
>>
>> The patch was successfully tested.
>>
>> Test details:
>> * Regression tests folder: jdk/test/tools/launcher/
>> * Builds were used: windows-x86_64-normal-server-fastdebug,
>> windows-x86_64-normal-server-release, windows-x86-normal-server-release;
>> * Platforms were used:  Windows 7(64 bit), Windows 8.1, Windows Server
>> 2012 R2 DC, Windows 10 ;
>> * System locales were used: English (United States), Persian, Japanese
>> (Japan), Chinese (Traditional, Taiwan), Russian (Russia);
>>
>> Thanks,
>> Vladimir.
>>
>> -----Original Message-----
>> From: Martin Sawicki
>> Sent: Thursday, January 14, 2016 11:34 AM
>> To: Kumar Srinivasan ; Vladimir
>> Shcherbakov 
>> Cc: core-libs-dev Libs ; Naoto Sato
>> 
>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Thanks for the feedback.
>> Investigating the regression failure.
>> We'll get back as soon as we figure this out.  (and yes, we'll run
>> this through some localized Windows VMs)
>>
>> Cheers
>>
>> -----Original Message-----
>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>> Sent: Tuesday, January 12, 2016 2:35 PM
>> To: Martin Sawicki ; Vladimir Shcherbakov
>> 
>> Cc: core-libs-dev Libs ; Naoto Sato
>> 
>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Hi Martin, Vladimir,
>>
>> It was suggested that this patch be tested on localized Windows
>> machines and/or trying with the various Windows native encodings,
>> appreciate if you can verify this as well.
>>
>> Thanks
>> Kumar
>>
>> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote:
>>> Hi,
>>>
>>> Was on vacation, I started to prepare the patch from webrev.04 for
>>> integration. Please note: made some adjustments to your patch to pass
>>> jcheck, ie. usage of tabs and space at line endings, and modifications
>>> to Copyright dates.
>>>
>>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE.
>>> I have attached a patch to for your reference.
>>>
>>> However, there is a regression test failure on Windows,
>>> jdk/test/tools/launcher/I18NTest.java
>>>
>>> ---Test info----
>>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java
>>>
>>> ++++Test Output++++
>>> javac: file not found: i18nH?lloWorld.java ----End test info-----
>>>
>>> Have you run all the launcher regression tests with this changeset ?
>>>
>>> Thanks
>>> Kumar
>>>
>>>> Hi Kumar, just wondering if there are any updates on processing this
>>>> submission.
>>>> Thanks!
>>>>
>>>> -----Original Message-----
>>>> From: Vladimir Shcherbakov
>>>> Sent: Wednesday, November 25, 2015 2:38 PM
>>>> To: Kumar Srinivasan ; Martin Sawicki
>>>> 
>>>> Cc: Kirk Shoop ; core-libs-dev Libs
>>>> 
>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>> Hi Kumar,
>>>>
>>>> Please find updated webreview here:
>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open
>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarcin
>>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f141
>>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNfwn
>>>> bIw%3d
>>>>
>>>> Thanks,
>>>> Vladimir.
>>>>
>>>> -----Original Message-----
>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>> Sent: Sunday, November 22, 2015 8:14 AM
>>>> To: Martin Sawicki 
>>>> Cc: Kirk Shoop ; Vladimir Shcherbakov
>>>> ; core-libs-dev Libs
>>>> 
>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>>
>>>> Hi Martin, et. al.,
>>>>
>>>> Sorry for not getting back earlier, I am very busy right now with my
>>>> other large commitments for JDK9.
>>>>
>>>> I will sponsor this "enhancement/bug fix" sometime in the new year,
>>>> meanwhile, there is the changeset  [1] which is likely to cause merge
>>>> conflicts, and perhaps logic issues.
>>>>
>>>> Thanks
>>>> Kumar
>>>>
>>>> [1]
>>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.op
>>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01%7
>>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988b
>>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk7M
>>>> g4lt2P0sdzLS%2fT9Q%3d
>>>>> Hi all
>>>>> Here's an updated webrev attempting to take into account the various
>>>>> pieces of feedback we have received:
>>>>>
>>>>> Issue:
>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs.
>>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40mic
>>>>> ro
>>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab2d
>>>>> 7c
>>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq9E
>>>>> %3
>>>>> d
>>>>> Webrev:
>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.ope
>>>>> nj
>>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlashc
>>>>> h%
>>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141a
>>>>> f9
>>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667BJn
>>>>> WY
>>>>> 0%3d
>>>>>
>>>>> (Vladimir Shcherbakov is now working on this from our side)
>>>>>
>>>>> Looking forward to any other feedback.
>>>>> Thanks
>>>>>
>>>>> -----Original Message-----
>>>>> From: core-libs-dev [mailto:core-libs-dev-bounces at openjdk.java.net]
>>>>> On Behalf Of Kumar Srinivasan
>>>>> Sent: Thursday, June 25, 2015 6:26 AM
>>>>> To: Kirk Shoop (MS OPEN TECH) 
>>>>> Cc: Valery Kopylov (Akvelon) ; core-libs-dev
>>>>> Libs 
>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>
>>>>> Hi Kirk,
>>>>>
>>>>> Thanks for proposing this change.
>>>>>
>>>>> If you notice all the posix calls are wrapped in JLI_* this gives us
>>>>> the ability to use "W" functions.  I almost got it done, several
>>>>> years ago, but we upgraded to VS2010 and my work based on VS2003
>>>>> keeled over, meanwhile my focus was  "shifted" to something else.
>>>>>
>>>>> main.c: is really envisioned to be a stub  compiled by the tool
>>>>> launchers, like java, javac, javah, jar etc. I prefer to see all the
>>>>> heavy logic in this file moved to the platform specific file
>>>>> windows/java_md.*
>>>>>
>>>>> For the reason specified above we need to move fprintf or any naked
>>>>> posix calls to JLI_* indirections.
>>>>>
>>>>> I don't see any tests ? The tests must be written in java and placed
>>>>> in jdk/test/tools/launcher, there is a helper framework
>>>>> TestHelper.java.
>>>>>
>>>>> There are other changes in nio, charsets etc, this will be reviewed
>>>>> by my colleague specializing in that area (Sherman) cc'ed.
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>> Kumar
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Issue:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbu
>>>>>> gs
>>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m
>>>>>> ic
>>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91a
>>>>>> b2
>>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsU
>>>>>> Iq
>>>>>> 9E%3d
>>>>>>
>>>>>> Webrev:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op
>>>>>> en
>>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micros
>>>>>> of
>>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d7c
>>>>>> d0
>>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0M%
>>>>>> 3d
>>>>>>
>>>>>> This webrev intends to address interaction between Windows console
>>>>>> and java apps.
>>>>>>
>>>>>> Two switches were added that change the behavior of the launcher.
>>>>>> The defaults do not change the launcher behavior.
>>>>>>
>>>>>>       -Dwindows.UnicodeConsole=true - switches on Unicode support in
>>>>>> the Windows console. This optional switch causes the launcher to
>>>>>> call GetCommandLineW() and parse the arguments in unicode. It also
>>>>>> modifies how the codepage for console output is selected.
>>>>>>
>>>>>>       -Dfile.encoding.unicode="UTF-8" - identifies Unicode charset
>>>>>> to use; If not specified, UTF-8 is used by default. Ignored when
>>>>>> windows.UnicodeConsole is not set to true. When the first switch is
>>>>>> used, this optional switch allows the codepage for console output
>>>>>> to be controlled.
>>>>>>
>>>>>> I would like to get feedback on the approach here and any
>>>>>> additional work that is required solve these particular Unicode
>>>>>> issues on Windows.
>>>>>>
>>>>>> Kirk
>


From vlashch at microsoft.com  Tue Feb 23 01:21:53 2016
From: vlashch at microsoft.com (Vladimir Shcherbakov)
Date: Tue, 23 Feb 2016 01:21:53 +0000
Subject: RFR 8124977 cmdline encoding challenges on Windows
In-Reply-To: <56CB34BD.4010707@oracle.com>
References: 
	
	<558C017A.2020708@oracle.com>
	
	<5651E9AA.9060908@oracle.com>
	
	
	<56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com>
	
	
	<56CB3099.4030108@oracle.com> <56CB34BD.4010707@oracle.com>
Message-ID: 

Hi Kumar,

(*) Highlights of the changes from the previous revision:

1) Conversion from UTF-16 to UTF-8.

    a) in webrev 4  was implemented in a standalone function JLI_DecodeArgs - the declaration and definition and usage affected 5 files: 

        src/java.base/share/native/libjli/jli_util.h
        make/lib/CoreLibraries.gmk
        src/java.base/unix/native/libjli/java_md_common.c
        src/java.base/windows/native/libjli/java_md.c
        src/java.base/share/native/launcher/main.c

    b) webrew 5: was implemented directly in the main method and affected only 1 file:  
    
        src/java.base/share/native/launcher/main.c
        
2) sprops.sun_stdout_encoding property 
    
    setting this property turned out to be unnecessary to address the problem and all related changed were abolished - this affected 2 files:
    
        src/java.base/windows/native/libjava/java_props_md.c
        src/java.base/windows/native/libjava/Console_md.c
        
3) Conversion from UTF-8 to UTF-16.

    to interact from Java launcher with Windows file system - changes to the file were added in webrew 5: 
        src/java.base/share/native/libjli/parse_manifest.c  


(*) All your comments have been fixed.
	Our only representative who has the right to post web reviews (Kirk Shoop) is on vacation now and will return on Monday 02/29/2016.
	We'll post the updated web review (6) as soon as he is back.

(*) We should give credit to:
    - Vladimir Shcherbakov  
    - Valeriy Kopylov 
    - Kirk Shoop 

Thanks,
Vladimir.

-----Original Message-----
From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] 
Sent: Monday, February 22, 2016 8:18 AM
To: Vladimir Shcherbakov 
Cc: Naoto Sato ; SHEN,XUEMING ; core-libs-dev Libs 
Subject: Re: RFR 8124977 cmdline encoding challenges on Windows


Few more comments on the tests:
UnicodeCmdTestRun.java:

The summary is fine, but it would be good to add a comment explaining what this test really does,  random folks look at the tests when a test failure occurs, such a comment should help, similar to what you have in https://na01.safelinks.protection.outlook.com/?url=UnicodeCmdTest.java&data=01%7c01%7cvlashch%40microsoft.com%7c3ec7d35c50cc44070a4108d33ba3d6ea%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=cBKAOpf5sqd1yS3JK2EX44E41f%2fd3gq9fOua6R2Jr38%3d

UnicodeCmdTest.java:
it would be good add an error at line 33,
+ System.out.println("Error: 0 length argument");
System.exit(1);

Thanks
Kumar

>
> Hi Naoto, Sherman,  can you please take a look.
> I tested with the jprt build and test all tests pass.
>
> Hi Vladimir, et. al.,
>
> It appears that there has been more simplifications from the previous 
> webrev.04. :-)
>
> It would've helped if you highlight the changes you have made from the 
> previous revision, unfortunately this is one of the deficiencies of 
> webrev.
>
> There are some inconsistencies in the coding conventions:
>
> parse_manifest.c
> + if (q == 0) return -1;
>
> we expect the return to be on the next line.
>
> similarly main.c
>
> if (0 == q)
> {
>
> I can fix those up. If I were to push this change, who should I 
> attribute the changes to ? ie. in the Contributed-by: line of the 
> commit info ?
> Please note these have to be email addresses of the contributors.
>
> Thanks
> Kumar
>
>> Hi Kumar,
>>
>> We posted another web review here: 
>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open
>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.05%2f&data=01%7C01%7Cvlashc
>> h%40microsoft.com%7C3ec7d35c50cc44070a4108d33ba3d6ea%7C72f988bf86f141
>> af91ab2d7cd011db47%7C1&sdata=UWa60mcs9nyfqxB1CpJA%2b6h%2f6fPks0aDofCv
>> k9ojssU%3d
>>
>> The patch was successfully tested.
>>
>> Test details:
>> * Regression tests folder: jdk/test/tools/launcher/
>> * Builds were used: windows-x86_64-normal-server-fastdebug,
>> windows-x86_64-normal-server-release, 
>> windows-x86-normal-server-release;
>> * Platforms were used:  Windows 7(64 bit), Windows 8.1, Windows 
>> Server 2012 R2 DC, Windows 10 ;
>> * System locales were used: English (United States), Persian, 
>> Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia);
>>
>> Thanks,
>> Vladimir.
>>
>> -----Original Message-----
>> From: Martin Sawicki
>> Sent: Thursday, January 14, 2016 11:34 AM
>> To: Kumar Srinivasan ; Vladimir 
>> Shcherbakov 
>> Cc: core-libs-dev Libs ; Naoto Sato 
>> 
>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Thanks for the feedback.
>> Investigating the regression failure.
>> We'll get back as soon as we figure this out.  (and yes, we'll run 
>> this through some localized Windows VMs)
>>
>> Cheers
>>
>> -----Original Message-----
>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>> Sent: Tuesday, January 12, 2016 2:35 PM
>> To: Martin Sawicki ; Vladimir Shcherbakov 
>> 
>> Cc: core-libs-dev Libs ; Naoto Sato 
>> 
>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Hi Martin, Vladimir,
>>
>> It was suggested that this patch be tested on localized Windows 
>> machines and/or trying with the various Windows native encodings, 
>> appreciate if you can verify this as well.
>>
>> Thanks
>> Kumar
>>
>> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote:
>>> Hi,
>>>
>>> Was on vacation, I started to prepare the patch from webrev.04 for 
>>> integration. Please note: made some adjustments to your patch to 
>>> pass jcheck, ie. usage of tabs and space at line endings, and 
>>> modifications to Copyright dates.
>>>
>>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE.
>>> I have attached a patch to for your reference.
>>>
>>> However, there is a regression test failure on Windows, 
>>> jdk/test/tools/launcher/I18NTest.java
>>>
>>> ---Test info----
>>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java
>>>
>>> ++++Test Output++++
>>> javac: file not found: i18nH?lloWorld.java ----End test info-----
>>>
>>> Have you run all the launcher regression tests with this changeset ?
>>>
>>> Thanks
>>> Kumar
>>>
>>>> Hi Kumar, just wondering if there are any updates on processing 
>>>> this submission.
>>>> Thanks!
>>>>
>>>> -----Original Message-----
>>>> From: Vladimir Shcherbakov
>>>> Sent: Wednesday, November 25, 2015 2:38 PM
>>>> To: Kumar Srinivasan ; Martin 
>>>> Sawicki 
>>>> Cc: Kirk Shoop ; core-libs-dev Libs 
>>>> 
>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>> Hi Kumar,
>>>>
>>>> Please find updated webreview here:
>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op
>>>> en 
>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarc
>>>> in
>>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f1
>>>> 41 
>>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNf
>>>> wn
>>>> bIw%3d
>>>>
>>>> Thanks,
>>>> Vladimir.
>>>>
>>>> -----Original Message-----
>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>> Sent: Sunday, November 22, 2015 8:14 AM
>>>> To: Martin Sawicki 
>>>> Cc: Kirk Shoop ; Vladimir Shcherbakov 
>>>> ; core-libs-dev Libs 
>>>> 
>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>>
>>>> Hi Martin, et. al.,
>>>>
>>>> Sorry for not getting back earlier, I am very busy right now with 
>>>> my other large commitments for JDK9.
>>>>
>>>> I will sponsor this "enhancement/bug fix" sometime in the new year, 
>>>> meanwhile, there is the changeset  [1] which is likely to cause 
>>>> merge conflicts, and perhaps logic issues.
>>>>
>>>> Thanks
>>>> Kumar
>>>>
>>>> [1]
>>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.
>>>> op
>>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01
>>>> %7 
>>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f98
>>>> 8b 
>>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk
>>>> 7M
>>>> g4lt2P0sdzLS%2fT9Q%3d
>>>>> Hi all
>>>>> Here's an updated webrev attempting to take into account the 
>>>>> various pieces of feedback we have received:
>>>>>
>>>>> Issue:
>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. 
>>>>>
>>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m
>>>>> ic
>>>>> ro
>>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab
>>>>> 2d
>>>>> 7c
>>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq
>>>>> 9E
>>>>> %3
>>>>> d
>>>>> Webrev:
>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.o
>>>>> pe
>>>>> nj
>>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlas
>>>>> hc
>>>>> h%
>>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f14
>>>>> 1a
>>>>> f9
>>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667B
>>>>> Jn
>>>>> WY
>>>>> 0%3d
>>>>>
>>>>> (Vladimir Shcherbakov is now working on this from our side)
>>>>>
>>>>> Looking forward to any other feedback.
>>>>> Thanks
>>>>>
>>>>> -----Original Message-----
>>>>> From: core-libs-dev 
>>>>> [mailto:core-libs-dev-bounces at openjdk.java.net]
>>>>> On Behalf Of Kumar Srinivasan
>>>>> Sent: Thursday, June 25, 2015 6:26 AM
>>>>> To: Kirk Shoop (MS OPEN TECH) 
>>>>> Cc: Valery Kopylov (Akvelon) ; 
>>>>> core-libs-dev Libs 
>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>
>>>>> Hi Kirk,
>>>>>
>>>>> Thanks for proposing this change.
>>>>>
>>>>> If you notice all the posix calls are wrapped in JLI_* this gives 
>>>>> us the ability to use "W" functions.  I almost got it done, 
>>>>> several years ago, but we upgraded to VS2010 and my work based on 
>>>>> VS2003 keeled over, meanwhile my focus was  "shifted" to something else.
>>>>>
>>>>> main.c: is really envisioned to be a stub  compiled by the tool 
>>>>> launchers, like java, javac, javah, jar etc. I prefer to see all 
>>>>> the heavy logic in this file moved to the platform specific file
>>>>> windows/java_md.*
>>>>>
>>>>> For the reason specified above we need to move fprintf or any 
>>>>> naked posix calls to JLI_* indirections.
>>>>>
>>>>> I don't see any tests ? The tests must be written in java and 
>>>>> placed in jdk/test/tools/launcher, there is a helper framework 
>>>>> TestHelper.java.
>>>>>
>>>>> There are other changes in nio, charsets etc, this will be 
>>>>> reviewed by my colleague specializing in that area (Sherman) cc'ed.
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>> Kumar
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Issue:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2f
>>>>>> bu
>>>>>> gs
>>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%4
>>>>>> 0m
>>>>>> ic
>>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af9
>>>>>> 1a
>>>>>> b2
>>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2f
>>>>>> sU
>>>>>> Iq
>>>>>> 9E%3d
>>>>>>
>>>>>> Webrev:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>> op
>>>>>> en
>>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micr
>>>>>> os
>>>>>> of
>>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d
>>>>>> 7c
>>>>>> d0
>>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0
>>>>>> M%
>>>>>> 3d
>>>>>>
>>>>>> This webrev intends to address interaction between Windows 
>>>>>> console and java apps.
>>>>>>
>>>>>> Two switches were added that change the behavior of the launcher.
>>>>>> The defaults do not change the launcher behavior.
>>>>>>
>>>>>>       -Dwindows.UnicodeConsole=true - switches on Unicode support 
>>>>>> in the Windows console. This optional switch causes the launcher 
>>>>>> to call GetCommandLineW() and parse the arguments in unicode. It 
>>>>>> also modifies how the codepage for console output is selected.
>>>>>>
>>>>>>       -Dfile.encoding.unicode="UTF-8" - identifies Unicode 
>>>>>> charset to use; If not specified, UTF-8 is used by default. 
>>>>>> Ignored when windows.UnicodeConsole is not set to true. When the 
>>>>>> first switch is used, this optional switch allows the codepage 
>>>>>> for console output to be controlled.
>>>>>>
>>>>>> I would like to get feedback on the approach here and any 
>>>>>> additional work that is required solve these particular Unicode 
>>>>>> issues on Windows.
>>>>>>
>>>>>> Kirk
>


From forax at univ-mlv.fr  Tue Feb 23 20:14:39 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Tue, 23 Feb 2016 21:14:39 +0100 (CET)
Subject: Collectors.minBy/maxBy behavior for null value
In-Reply-To: <56CB24B9.4010907@gmail.com>
References: 
	<56CB24B9.4010907@gmail.com>
Message-ID: <1552250860.1046795.1456258479619.JavaMail.zimbra@u-pem.fr>

I agree with Peter,
nulls in stream are tolerated for backward compatibility with existing code,
null is not something that should trigger a specific behavior apart a NPE.

regards,
R?mi

----- Mail original -----
> De: "Peter Levart" 
> ?: "Tagir Valeev" , "core-libs-dev" 
> Envoy?: Lundi 22 F?vrier 2016 16:09:45
> Objet: Re: Collectors.minBy/maxBy behavior for null value
> 
> 
> 
> On 02/22/2016 11:06 AM, Tagir Valeev wrote:
> > Hello!
> >
> > Consider the following code:
> >
> > Comparator cmp = Comparator.nullsFirst(Comparator.naturalOrder());
> > System.out.println(Stream.of("a", "b",
> > null).collect(Collectors.minBy(cmp)));
> >
> > It prints Optional.empty, so the result is indistinguishable from empty
> > Stream (siimlar for maxBy). This behavior is not consistent with
> > Stream.findFirst() (which throws NPE if the resulting element is null).
> > Currently minBy spec says:
> >
> > This produces a result equivalent to:
> >       reducing(BinaryOperator.minBy(comparator))
> >
> > However reducing spec says nothing what will occur if the result of
> > reduction is null. To my opinion at very least the spec of reducing should
> > be updated to specify the behavior when reducing result is null. It
> > probably would also be nice to add a note into minBy/maxBy spec about this.
> >
> > What do you think?
> 
> My personal opinion is that it would be better to throw NPE in this case
> (like with findFirst()). Silently converting null element to
> Optional.empty() is probably not the desired behavior and I doubt anyone
> is exploiting this implementation detail in such way.
> 
> Regards, Peter
> 
> >
> > With best regards,
> > Tagir Valeev.
> 
> 


From gil at azul.com  Tue Feb 23 21:20:04 2016
From: gil at azul.com (Gil Tene)
Date: Tue, 23 Feb 2016 21:20:04 +0000
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <20160222101133.946427992eggemoggin.niobe.net>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<20160222101133.946427992eggemoggin.niobe.net>
Message-ID: <45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>


> On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com wrote:
> 
> 2016/1/28 9:25 -0800, gil at azul.com:
>> This thread seems to have "hopped away" to the concurrency-interest
>> list in mid-Dec-2015. This posting is intended to capture a summary of
>> reasoning and some of the discussion there so that we have it in the
>> record in core-libs-dev. Mostly by including the contents of several
>> posts in the continuations of the original thread.
>> 
>> See thread continuations here:
>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
>> and here:
>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
>> 
>> Summary:
>> 
>> ...
> 
> Thanks for the summary.
> 
> I still don't buy the argument that this method belongs in j.l.Runtime.
> 
> To say that this method should go there because it's an instruction to
> the run-time system is pretty weak.  I agree with Vitaly [1] that if
> that's the threshold for adding methods to the Runtime class then lots
> of other stuff belongs there as well, including much of what's now in
> java.lang.Thread and java.util.concurrent and, arguably, anything else
> related to interacting with the environment in which the application
> runs (file and network I/O, process manipulation, etc.).
> 
> This thread-related method really belongs in either java.lang.Thread or
> java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
> expert-level static methods related to the current thread, one of which
> (Thread::yield) is even a hint, just like this one.  j.u.c.LockSupport
> is even more obviously intended for expert users and hence may be the
> best choice, but I could live with either one.

Ok. In the interest of moving forward, lets settle on:

Thread.onSpinWait()

Same logic for the name, different receiver for the event. I can certainly live with it, and Doug seems ok with it as well.

? Gil.


From vitalyd at gmail.com  Tue Feb 23 21:30:45 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 23 Feb 2016 16:30:45 -0500
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
Message-ID: 

Why not drop the (superfluous, IMO) "on" prefix while you're changing the
receiver? Thread::spinWait would be in the same style/spirit as
Thread::yield, and I don't think there will be much contention.

On Tue, Feb 23, 2016 at 4:20 PM, Gil Tene  wrote:

>
> > On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com wrote:
> >
> > 2016/1/28 9:25 -0800, gil at azul.com:
> >> This thread seems to have "hopped away" to the concurrency-interest
> >> list in mid-Dec-2015. This posting is intended to capture a summary of
> >> reasoning and some of the discussion there so that we have it in the
> >> record in core-libs-dev. Mostly by including the contents of several
> >> posts in the continuations of the original thread.
> >>
> >> See thread continuations here:
> >>
> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
> >> and here:
> >>
> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
> >>
> >> Summary:
> >>
> >> ...
> >
> > Thanks for the summary.
> >
> > I still don't buy the argument that this method belongs in j.l.Runtime.
> >
> > To say that this method should go there because it's an instruction to
> > the run-time system is pretty weak.  I agree with Vitaly [1] that if
> > that's the threshold for adding methods to the Runtime class then lots
> > of other stuff belongs there as well, including much of what's now in
> > java.lang.Thread and java.util.concurrent and, arguably, anything else
> > related to interacting with the environment in which the application
> > runs (file and network I/O, process manipulation, etc.).
> >
> > This thread-related method really belongs in either java.lang.Thread or
> > java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
> > expert-level static methods related to the current thread, one of which
> > (Thread::yield) is even a hint, just like this one.  j.u.c.LockSupport
> > is even more obviously intended for expert users and hence may be the
> > best choice, but I could live with either one.
>
> Ok. In the interest of moving forward, lets settle on:
>
> Thread.onSpinWait()
>
> Same logic for the name, different receiver for the event. I can certainly
> live with it, and Doug seems ok with it as well.
>
> ? Gil.
>


From brian.burkhalter at oracle.com  Tue Feb 23 22:18:10 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Tue, 23 Feb 2016 14:18:10 -0800
Subject: RFR 9: 8150337 : sun/misc/SunMiscSignalTest.java failed
	intermittently
In-Reply-To: <56CC7AC7.6090004@Oracle.com>
References: <56CC7AC7.6090004@Oracle.com>
Message-ID: <6BEB9950-D649-44B6-B67D-AA9A33B397FF@oracle.com>

Hi Roger,

I am no expert in this area but the changes appear to be reasonable and consistent. One thumb up.

Brian

On Feb 23, 2016, at 7:29 AM, Roger Riggs  wrote:

> Please review a test update to skip tests using SIGINT, SIGTERM, and SIGHUP when
> run with those signals are ignored.  For example, using nohup or trap in a shell.
> It also includes a small enhancement adding sun.misc.Signal.toString ease debugging.
> 
> Webrev:
> http://cr.openjdk.java.net/~rriggs/webrev-nohup-8150337/index.html
> 
> Thanks, Roger
> 



From Roger.Riggs at Oracle.com  Tue Feb 23 22:30:39 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Tue, 23 Feb 2016 17:30:39 -0500
Subject: RFR 9: 8150346 : java/lang/ProcessHandle/Infotest.java failed:
	startTime after process spawn completed
Message-ID: <56CCDD8F.3070807@Oracle.com>

Please review a test fix to resolve a timing race with the start time of 
a spawned process.

Webrev:
   http://cr.openjdk.java.net/~rriggs/webrev-start-8150346/

Jira:
    https://bugs.openjdk.java.net/browse/JDK-8150346

Thanks, Roger



From kim.barrett at oracle.com  Tue Feb 23 23:22:05 2016
From: kim.barrett at oracle.com (Kim Barrett)
Date: Tue, 23 Feb 2016 18:22:05 -0500
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CC8A4A.9080303@gmail.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com>
	<56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com> <56CC8A4A.9080303@gmail.com>
Message-ID: 

> On Feb 23, 2016, at 11:35 AM, Peter Levart  wrote:
> 
> Hi Roger, Mandy,
> 
> Here's another webrev:
> 
> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.05/
> 
> I renamed the method and reworded the specification. Is this better now?
> 
> 
> On 02/22/2016 10:56 PM, Roger Riggs wrote:
>> Hi Mandy,
>> 
>> On 2/22/2016 4:41 PM, Mandy Chung wrote:
>> 
>>> 
>>> The existing way to do that is to register phantom references in your own ReferenceQueue and then drain the queue at appropriate point.  Would you consider having a method to return ReferenceQueue maintained by the cleaner instead?
>> If the queue is exposed, there is no assurance that the cleanable function would be called.
>> Any caller could introduce a bug by not doing the proper cleaning.
>> 
>> I was more concerned with the crossing of Reference.tryHandlePending with the cleaning thread.
>> The method description does not mention anything related to the Reference processing thread
>> though that is all implementation.  The @implNote might be a bit more concise and less informal.
>> 
>> Roger
> 
> Yes, ReferenceHandler thread is just an implementation detail. The specification of java.lang.Reference subclasses doesn't even mention it. It talks about GC enqueueing Reference objects:
> 
> "Suppose that the garbage collector determines at a certain point in time...
> ...At the same time or at some later time it will enqueue those newly-cleared weak references that are registered with reference queues."
> 
> So in this respect ReferenceHandler is just an extension of GC Reference discovery/enqueuing logic, delegated to a background thread on Java side. The Cleaner.cleanNextPending() method tries to be true to its specification - it tries to invoke next cleanup action for which GC has determined that its monitored object is phantom reachable without waiting for ReferenceHandler thread to transfer it from pending list to the queue.
> 
> Since Reference.tryHandlePending is just transfering Reference objects from one list to another and does that using two locks (the Reference.lock and the ReferenceQueue.lock) but never holds them both together or calls any outside code while holding any of the locks, there's no danger of dead-locking, if that was your concern.
> 
> Regards, Peter

------------------------------------------------------------------------------ 
src/java.base/share/classes/java/lang/ref/Cleaner.java
 242     public boolean cleanNextPending() {
 243         while (!impl.cleanNextEnqueuedCleanable()) {
 244             if (!Reference.tryHandlePending(false)) {
 245                 return false;
 246             }
 247         }
 248         return true;
 249     }

This can loop for an arbitrarily long time if there are many pending
references that are unrelated to the invoking Cleaner.  It could even
loop arbitrarily long if there are lots of pending references and only
a few scattered ones are for the invoking Cleaner, because the
Cleaner's thread might take care of each of them before the helping
thread re-checks the queue.

The Disposer class that was mentioned as a use-case for this new
function doesn't try to do anything like that; it just loops until
there aren't any more in the queue.  The caller there is only affected
by the number of enqueued objects, not by the number of unrelated
pending objects.

OTOH, while not structured exactly the same, I think this has a
similar effect to the existing Direct-X-Buffer support provided by
Bits.  Helping the single reference processing thread could be useful,
though contention for the pending list lock could be a problem.  And
this isn't *quite* the same as what we have now, because the reference
will be enqueued and the Cleaner will be notified, so that the helper
and the Cleaner compete to process the queued cleanup; with
sun.misc.Cleaner the helper just directly invokes the cleanup.

I'm not sure what to do about any of this; I'm just feeling like maybe
the proposed API / implementation isn't quite right yet.

------------------------------------------------------------------------------ 
src/java.base/share/classes/java/nio/Direct-X-Buffer.java
  34 import jdk.internal.ref.CleanerFactory;
&etc.

It might make sense to give Direct-X-Buffer its own dedicated Cleaner,
rather than sharing the jdk.internal Cleaner.

------------------------------------------------------------------------------



From christoph.langer at sap.com  Tue Feb 23 23:49:41 2016
From: christoph.langer at sap.com (Langer, Christoph)
Date: Tue, 23 Feb 2016 23:49:41 +0000
Subject: RFR 8150470: JCK: api/xsl/conf/copy/copy19 test failure
Message-ID: <7d93f07319f54ba084fdd94adfac5e88@derote13de11.global.corp.sap>

Hi,



the changeset for 8149915 (enabling validate-annotations feature for xsd schema with annotation causes NPE) introduced a regression. Here is the fix.



Bugid: https://bugs.openjdk.java.net/browse/JDK-8150470

Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8150470.1/



Please review and push this one liner.



Thanks & Best regards

Christoph





From christoph.langer at sap.com  Tue Feb 23 23:51:40 2016
From: christoph.langer at sap.com (Langer, Christoph)
Date: Tue, 23 Feb 2016 23:51:40 +0000
Subject: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature
	for xsd schema with annotation causes NPE
References: 
	<56C38060.2030703@oracle.com>
	<80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap>
	<56C3AF17.6050103@oracle.com>
	
	<56C4D342.9070705@oracle.com>
	<56c7676f64ad49bd9dc9cf8cf1ed617f@DEWDFE13DE11.global.corp.sap>
	<56C60B19.3040508@oracle.com>
	<8f47645d4d074224bf30c08237df7cdf@DEWDFE13DE11.global.corp.sap>
	<56CC40B1.9040905@oracle.com>
	<2e459a12d41449faaa8a11522184782e@derote13de11.global.corp.sap>
	<56CC7EDF.30209@oracle.com> <56CC8588.1070403@oracle.com>
	<56CC8DEC.7050500@oracle.com> <56CC961B.7010000@oracle.com> 
Message-ID: <8c0aeda413f5455db06c4de4200c464f@derote13de11.global.corp.sap>

Sorry, this one is the right webrev:

http://cr.openjdk.java.net/~clanger/webrevs/8150470.1/





From: Langer, Christoph
Sent: Dienstag, 23. Februar 2016 21:13
To: 'Aleksej Efimov' ; huizhe wang 
Subject: RE: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE



Hi Aleksej, Joe,



sorry, I already had a bad feeling with that line...



Here is a bug: https://bugs.openjdk.java.net/browse/JDK-8150470

And here is a webrev for fix: http://cr.openjdk.java.net/~clanger/webrevs/8150470.0/



Feel free to add more data to the bug and do more tests. I only had the jtreg tests at hand.



Shall I post this in the mailing list?



Best regards

Christoph



From: Aleksej Efimov [mailto:aleksej.efimov at oracle.com]
Sent: Dienstag, 23. Februar 2016 18:26
To: huizhe wang >
Cc: Langer, Christoph >
Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE



Sure!
Will do it tomorrow.

Best Regards,
Aleksej

On 02/23/2016 07:50 PM, huizhe wang wrote:

   Hi Aleksej,

   Thanks for catching the issue and identifying the fix!  Once you come back from the holiday, could you submit a review of the fix?

   Best,
   Joe

   On 2/23/2016 8:15 AM, Aleksej Efimov wrote:

      Hi again,
      Got an update on this issue:
      The fix introduced back this one issue: https://bugs.openjdk.java.net/browse/JDK-8021366

      The root cause is that Entity.DEFAULT_XMLDECL_BUFFER_SIZE and XMLEntityManager.DEFAULT_XMLDECL_BUFFER_SIZE have different values: 32 and 64. This change erased the JDK-8021366 change (64->32) and introduced back the issue:

      -                        len = fCurrentEntity.DEFAULT_XMLDECL_BUFFER_SIZE;
      +                        len = DEFAULT_XMLDECL_BUFFER_SIZE;

      Tested the reversion of the fix for only this line and JCK test passes now: -                        len = DEFAULT_XMLDECL_BUFFER_SIZE; +                        len = fCurrentEntity.DEFAULT_XMLDECL_BUFFER_SIZE; With Best Regards, Aleksej On 02/23/2016 06:46 PM, Aleksej Efimov wrote:

         Hi Christoph, Joe, I think we may have a problem with this fix. I'm observing one new JCK test failure with the fix. If the fix is stripped from JDK9 repo - no failure observed. Failed test is api/xsl/conf/copy/copy19.html#copy19 The failure message is:

            MultiJVM group agent ID: 1 java version "9-internal" OpenJDK Runtime Environment (build 9-internal+0-2016-02-23-012828.aefimov.9) OpenJDK 64-Bit Server VM (build 9-internal+0-2016-02-23-012828.aefimov.9, mixed mode) Failed to parse as XML. Cause: warning;org.xml.sax.SAXParseException; lineNumber: 2; columnNumber: 10; XML document structures must start and end within the same entity. Parsing as text. Failed to parse as XML. Cause: warning;org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 53; XML document structures must start and end within the same entity. Parsing as text. mismatch-value;TEXT()len=43;TEXT()len=62;lengths do not match mismatch-value-gold;TEXT();  mismatch-value-text;TEXT(); abcd\u00e8fgh XHTFileCheckService results were not equal. Expected result:  abcd\u00e8fgh Actual result: abcd\u00e8fgh result: Failed. test cases: 1; all failed; first test case failure: testTransformation test result: Failed. test cases: 1; all failed; first test case failure: testTransformation

         Today is a holiday in SPb. Tomorrow I can log a bug to track this failure (new line is missing in actual result output). Best Regards, Aleksej On 02/23/2016 02:27 PM, Langer, Christoph wrote:

            Hi Aleksej,

            that's great, please go ahead. Sooner or later I would have started this effort, too. But right now I'm working on another fix for the XALAN which I'll post soon :-)

            Best regards
            Christoph

            PS: @Joe: I noticed that in the commit I was not the author of the change but only mentioned in the "Contributed-by" tag. Couldn't you push the change on my behalf but with maintaining me as the author?

            -----Original Message-----
            From: Aleksej Efimov [mailto:aleksej.efimov at oracle.com]
            Sent: Dienstag, 23. Februar 2016 12:21
            To: Langer, Christoph 
            Cc: Joe Wang 
            Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE

            Hi Christoph,
            I would like to backport your fix to JDK8.
            Just wanted to get your confirmation that you're not working on that and
            we are not doing the same task =)

            With Best Regards,
            Aleksej

            On 02/18/2016 10:33 PM, Langer, Christoph wrote:

               Hi Joe,

               I fixed the copyright in http://cr.openjdk.java.net/~clanger/webrevs/8149915.3/. So I think I'm done. Can you review and push the change then?

               Thanks
               Christoph


               -----Original Message-----
               From: huizhe wang [mailto:huizhe.wang at oracle.com]
               Sent: Donnerstag, 18. Februar 2016 19:19
               To: Langer, Christoph 
               Cc: core-libs-dev at openjdk.java.net
               Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE


               On 2/18/2016 5:15 AM, Langer, Christoph wrote:

                  Hi Joe,

                  here is the next version:
                  http://cr.openjdk.java.net/~clanger/webrevs/8149915.2/

               Looks good!  Thanks.

               Copyright year in XSAttributeChecker.java is still incorrect. I meant to
               say the year "2016" needs to have a comma, so in this case, it needs to
               be "2015, 2016, " rather than "2015, 2016".

               Best,
               Joe


                  Hopefully the final one :-)


                     Yes, it's good to have some cleanups. For the generic initialization,
                     it's preferable to have no redundant type arguments, for example,
                     instead of:

                     protected Stack fEntityStack = new Stack();

                     do:
                     protected Stack fEntityStack = new Stack<>();

                     Some of the "cleanup" in the webrev were reversing <> with added type
                     argument(s), that would be unnecessary.

                  I think I did that on all the places that I touched now.


                     For the obsolete Vector, it would be good to replace it with ArrayList.

                  There are so many Vector instances in this code that it's out of scope for me now to touch them all...


                     Copyright year "2016" in XSAttributeChecker needs to be "2016," or else
                     the script would complain.

                  For that one I thought that it would look like "2015,2016" if the initial copyright was done in 2015. But I changed it as you suggested.


                     In XSDHandler, note that there's another call to get SECURITY_MANAGER
                     towards the end of the reset method (at 3572 in your new version) that
                     may be removed. You may also consolidate most of those
                     setFeature/Property statements in one try-catch block if you want.

                  OK, I did some further refacturing here. However, I've left the try/catch blocks as I think it's the intention to silently catch some exceptions but then try to go on with the next property.


                     For the test, the @bug tag is still desirable in the general comment
                     section. For now, it's @bug 8149915. As we add more test to the class in
                     the future, we'd then append more bug ids, @bug 8149915, xxxxxxx.

                  Done. I also left the @bug in the comments for the test method.


                        I'm also wondering why some files have a copyright header like this, e.g.

                     src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11D
                     TDScannerImpl.java:

                        /*
                            * reserved comment block
                            * DO NOT REMOVE OR ALTER!
                            */

                        Shouldn't there be the standard copyright header or is there some reason

                     behind it?

                     The block is used by the licensing script to swap with license
                     information required by licensees. For code that came from Xerces, we
                     should keep the block as is with the Apache License. But when we make
                     changes, we're required to put in the copyright header. Since the
                     changes you made are cleanup / removing unused fields, it's okay to keep
                     the block as is.

                  OK :)

                  Thanks again for reviewing.

                  All the Best
                  Christoph





From huizhe.wang at oracle.com  Wed Feb 24 00:06:58 2016
From: huizhe.wang at oracle.com (huizhe wang)
Date: Tue, 23 Feb 2016 16:06:58 -0800
Subject: JDK-8149915 -> Re: JDK 9 RFR of JDK-8149915: enabling
	validate-annotations feature for xsd schema with annotation causes NPE
In-Reply-To: <8c0aeda413f5455db06c4de4200c464f@derote13de11.global.corp.sap>
References: 
	<56C38060.2030703@oracle.com>
	<80b3957f5d9146dba2ebb20a020190e9@DEWDFE13DE11.global.corp.sap>
	<56C3AF17.6050103@oracle.com>
	
	<56C4D342.9070705@oracle.com>
	<56c7676f64ad49bd9dc9cf8cf1ed617f@DEWDFE13DE11.global.corp.sap>
	<56C60B19.3040508@oracle.com>
	<8f47645d4d074224bf30c08237df7cdf@DEWDFE13DE11.global.corp.sap>
	<56CC40B1.9040905@oracle.com>
	<2e459a12d41449faaa8a11522184782e@derote13de11.global.corp.sap>
	<56CC7EDF.30209@oracle.com> <56CC8588.1070403@oracle.com>
	<56CC8DEC.7050500@oracle.com> <56CC961B.7010000@oracle.com>
	<8c0aeda413f5455db06c4de4200c464f@derote13de11.global.corp.sap>
Message-ID: <56CCF422.9030404@oracle.com>

The change looks good.

Thanks Aleksej, again, for catching the issue. For future changes, we'll 
need to make sure we run the JCK tests.

Best,
Joe

On 2/23/2016 3:51 PM, Langer, Christoph wrote:
>
> Sorry, this one is the right webrev:
>
> http://cr.openjdk.java.net/~clanger/webrevs/8150470.1/ 
> 
>
> *From:*Langer, Christoph
> *Sent:* Dienstag, 23. Februar 2016 21:13
> *To:* 'Aleksej Efimov' ; huizhe wang 
> 
> *Subject:* RE: JDK 9 RFR of JDK-8149915: enabling validate-annotations 
> feature for xsd schema with annotation causes NPE
>
> Hi Aleksej, Joe,
>
> sorry, I already had a bad feeling with that line...
>
> Here is a bug: https://bugs.openjdk.java.net/browse/JDK-8150470
>
> And here is a webrev for fix: 
> http://cr.openjdk.java.net/~clanger/webrevs/8150470.0/ 
> 
>
> Feel free to add more data to the bug and do more tests. I only had 
> the jtreg tests at hand.
>
> Shall I post this in the mailing list?
>
> Best regards
>
> Christoph
>
> *From:*Aleksej Efimov [mailto:aleksej.efimov at oracle.com]
> *Sent:* Dienstag, 23. Februar 2016 18:26
> *To:* huizhe wang >
> *Cc:* Langer, Christoph  >
> *Subject:* Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations 
> feature for xsd schema with annotation causes NPE
>
> Sure!
> Will do it tomorrow.
>
> Best Regards,
> Aleksej
>
> On 02/23/2016 07:50 PM, huizhe wang wrote:
>
>     Hi Aleksej,
>
>     Thanks for catching the issue and identifying the fix!  Once you
>     come back from the holiday, could you submit a review of the fix?
>
>     Best,
>     Joe
>
>     On 2/23/2016 8:15 AM, Aleksej Efimov wrote:
>
>         Hi again,
>         Got an update on this issue:
>         The fix introduced back this one issue:
>         https://bugs.openjdk.java.net/browse/JDK-8021366
>         
>
>         The root cause is that Entity.DEFAULT_XMLDECL_BUFFER_SIZE and
>         XMLEntityManager.DEFAULT_XMLDECL_BUFFER_SIZE have different
>         values: 32 and 64. This change erased the JDK-8021366 change
>         (64->32) and introduced back the issue:
>
>         -                        len = fCurrentEntity.DEFAULT_XMLDECL_BUFFER_SIZE;
>
>         +                        len = DEFAULT_XMLDECL_BUFFER_SIZE;
>
>         Tested the reversion of the fix for only this line and JCK
>         test passes now: -                        len =
>         DEFAULT_XMLDECL_BUFFER_SIZE; + len =
>         fCurrentEntity.DEFAULT_XMLDECL_BUFFER_SIZE; With Best Regards,
>         Aleksej On 02/23/2016 06:46 PM, Aleksej Efimov wrote:
>
>             Hi Christoph, Joe, I think we may have a problem with this
>             fix. I'm observing one new JCK test failure with the fix.
>             If the fix is stripped from JDK9 repo - no failure
>             observed. Failed test is
>             api/xsl/conf/copy/copy19.html#copy19 The failure message is:
>
>                 MultiJVM group agent ID: 1 java version "9-internal"
>                 OpenJDK Runtime Environment (build
>                 9-internal+0-2016-02-23-012828.aefimov.9) OpenJDK
>                 64-Bit Server VM (build
>                 9-internal+0-2016-02-23-012828.aefimov.9, mixed mode)
>                 Failed to parse as XML. Cause:
>                 warning;org.xml.sax.SAXParseException; lineNumber: 2;
>                 columnNumber: 10; XML document structures must start
>                 and end within the same entity. Parsing as text.
>                 Failed to parse as XML. Cause:
>                 warning;org.xml.sax.SAXParseException; lineNumber: 1;
>                 columnNumber: 53; XML document structures must start
>                 and end within the same entity. Parsing as text.
>                 mismatch-value;TEXT()len=43;TEXT()len=62;lengths do
>                 not match mismatch-value-gold;TEXT();                  version="1.0" encoding="ISO-8859-1"?>
>                 mismatch-value-text;TEXT();                  encoding="ISO-8859-1"?>abcd\u00e8fgh
>                 XHTFileCheckService results were not equal. Expected
>                 result: 
>                 abcd\u00e8fgh Actual result:                  version="1.0"
>                 encoding="ISO-8859-1"?>abcd\u00e8fgh
>                 result: Failed. test cases: 1; all failed; first test
>                 case failure: testTransformation test result: Failed.
>                 test cases: 1; all failed; first test case failure:
>                 testTransformation
>
>             Today is a holiday in SPb. Tomorrow I can log a bug to
>             track this failure (new line is missing in actual result
>             output). Best Regards, Aleksej On 02/23/2016 02:27 PM,
>             Langer, Christoph wrote:
>
>                 Hi Aleksej,
>
>                   
>
>                 that's great, please go ahead. Sooner or later I would have started this effort, too. But right now I'm working on another fix for the XALAN which I'll post soon :-)
>
>                   
>
>                 Best regards
>
>                 Christoph
>
>                   
>
>                 PS: @Joe: I noticed that in the commit I was not the author of the change but only mentioned in the "Contributed-by" tag. Couldn't you push the change on my behalf but with maintaining me as the author?
>
>                   
>
>                 -----Original Message-----
>
>                 From: Aleksej Efimov [mailto:aleksej.efimov at oracle.com]
>
>                 Sent: Dienstag, 23. Februar 2016 12:21
>
>                 To: Langer, Christoph  
>
>                 Cc: Joe Wang  
>
>                 Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE
>
>                   
>
>                 Hi Christoph,
>
>                 I would like to backport your fix to JDK8.
>
>                 Just wanted to get your confirmation that you're not working on that and
>
>                 we are not doing the same task =)
>
>                   
>
>                 With Best Regards,
>
>                 Aleksej
>
>                   
>
>                 On 02/18/2016 10:33 PM, Langer, Christoph wrote:
>
>                     Hi Joe,
>
>                       
>
>                     I fixed the copyright inhttp://cr.openjdk.java.net/~clanger/webrevs/8149915.3/  . So I think I'm done. Can you review and push the change then?
>
>                       
>
>                     Thanks
>
>                     Christoph
>
>                       
>
>                       
>
>                     -----Original Message-----
>
>                     From: huizhe wang [mailto:huizhe.wang at oracle.com]
>
>                     Sent: Donnerstag, 18. Februar 2016 19:19
>
>                     To: Langer, Christoph  
>
>                     Cc:core-libs-dev at openjdk.java.net  
>
>                     Subject: Re: JDK 9 RFR of JDK-8149915: enabling validate-annotations feature for xsd schema with annotation causes NPE
>
>                       
>
>                       
>
>                     On 2/18/2016 5:15 AM, Langer, Christoph wrote:
>
>                         Hi Joe,
>
>                           
>
>                         here is the next version:
>
>                         http://cr.openjdk.java.net/~clanger/webrevs/8149915.2/  
>
>                     Looks good!  Thanks.
>
>                       
>
>                     Copyright year in XSAttributeChecker.java is still incorrect. I meant to
>
>                     say the year "2016" needs to have a comma, so in this case, it needs to
>
>                     be "2015, 2016, " rather than "2015, 2016".
>
>                       
>
>                     Best,
>
>                     Joe
>
>                       
>
>                         Hopefully the final one :-)
>
>                           
>
>                             Yes, it's good to have some cleanups. For the generic initialization,
>
>                             it's preferable to have no redundant type arguments, for example,
>
>                             instead of:
>
>                               
>
>                             protected Stack fEntityStack = new Stack();
>
>                               
>
>                             do:
>
>                             protected Stack fEntityStack = new Stack<>();
>
>                               
>
>                             Some of the "cleanup" in the webrev were reversing <> with added type
>
>                             argument(s), that would be unnecessary.
>
>                         I think I did that on all the places that I touched now.
>
>                           
>
>                             For the obsolete Vector, it would be good to replace it with ArrayList.
>
>                         There are so many Vector instances in this code that it's out of scope for me now to touch them all...
>
>                           
>
>                             Copyright year "2016" in XSAttributeChecker needs to be "2016," or else
>
>                             the script would complain.
>
>                         For that one I thought that it would look like "2015,2016" if the initial copyright was done in 2015. But I changed it as you suggested.
>
>                           
>
>                             In XSDHandler, note that there's another call to get SECURITY_MANAGER
>
>                             towards the end of the reset method (at 3572 in your new version) that
>
>                             may be removed. You may also consolidate most of those
>
>                             setFeature/Property statements in one try-catch block if you want.
>
>                         OK, I did some further refacturing here. However, I've left the try/catch blocks as I think it's the intention to silently catch some exceptions but then try to go on with the next property.
>
>                           
>
>                             For the test, the @bug tag is still desirable in the general comment
>
>                             section. For now, it's @bug 8149915. As we add more test to the class in
>
>                             the future, we'd then append more bug ids, @bug 8149915, xxxxxxx.
>
>                         Done. I also left the @bug in the comments for the test method.
>
>                           
>
>                                 I'm also wondering why some files have a copyright header like this, e.g.
>
>                             src/java.xml/share/classes/com/sun/org/apache/xerces/internal/impl/XML11D
>
>                             TDScannerImpl.java:
>
>                                 /*
>
>                                      * reserved comment block
>
>                                      * DO NOT REMOVE OR ALTER!
>
>                                      */
>
>                                   
>
>                                 Shouldn't there be the standard copyright header or is there some reason
>
>                             behind it?
>
>                               
>
>                             The block is used by the licensing script to swap with license
>
>                             information required by licensees. For code that came from Xerces, we
>
>                             should keep the block as is with the Apache License. But when we make
>
>                             changes, we're required to put in the copyright header. Since the
>
>                             changes you made are cleanup / removing unused fields, it's okay to keep
>
>                             the block as is.
>
>                         OK :)
>
>                           
>
>                         Thanks again for reviewing.
>
>                           
>
>                         All the Best
>
>                         Christoph
>
>                           
>
>                           
>



From dl at cs.oswego.edu  Wed Feb 24 00:23:49 2016
From: dl at cs.oswego.edu (Doug Lea)
Date: Tue, 23 Feb 2016 19:23:49 -0500
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: 
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
Message-ID: <56CCF815.7010105@cs.oswego.edu>

On 02/23/2016 04:30 PM, Vitaly Davidovich wrote:
> Why not drop the (superfluous, IMO) "on" prefix while you're changing the
> receiver?

Because then it reads as if the method itself is doing a spinWait.
"onSpinWait" is the only proposed name that no one has said they cannot
live with. So, live with it :-)

-Doug


>
> On Tue, Feb 23, 2016 at 4:20 PM, Gil Tene  wrote:
>
>>
>>> On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com wrote:
>>>
>>> 2016/1/28 9:25 -0800, gil at azul.com:
>>>> This thread seems to have "hopped away" to the concurrency-interest
>>>> list in mid-Dec-2015. This posting is intended to capture a summary of
>>>> reasoning and some of the discussion there so that we have it in the
>>>> record in core-libs-dev. Mostly by including the contents of several
>>>> posts in the continuations of the original thread.
>>>>
>>>> See thread continuations here:
>>>>
>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
>>>> and here:
>>>>
>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
>>>>
>>>> Summary:
>>>>
>>>> ...
>>>
>>> Thanks for the summary.
>>>
>>> I still don't buy the argument that this method belongs in j.l.Runtime.
>>>
>>> To say that this method should go there because it's an instruction to
>>> the run-time system is pretty weak.  I agree with Vitaly [1] that if
>>> that's the threshold for adding methods to the Runtime class then lots
>>> of other stuff belongs there as well, including much of what's now in
>>> java.lang.Thread and java.util.concurrent and, arguably, anything else
>>> related to interacting with the environment in which the application
>>> runs (file and network I/O, process manipulation, etc.).
>>>
>>> This thread-related method really belongs in either java.lang.Thread or
>>> java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
>>> expert-level static methods related to the current thread, one of which
>>> (Thread::yield) is even a hint, just like this one.  j.u.c.LockSupport
>>> is even more obviously intended for expert users and hence may be the
>>> best choice, but I could live with either one.
>>
>> Ok. In the interest of moving forward, lets settle on:
>>
>> Thread.onSpinWait()
>>
>> Same logic for the name, different receiver for the event. I can certainly
>> live with it, and Doug seems ok with it as well.
>>
>> ? Gil.
>>
>




From naoto.sato at oracle.com  Wed Feb 24 00:34:07 2016
From: naoto.sato at oracle.com (Naoto Sato)
Date: Tue, 23 Feb 2016 16:34:07 -0800
Subject: RFR 8124977 cmdline encoding challenges on Windows
In-Reply-To: 
References: 
	
	<558C017A.2020708@oracle.com>
	
	<5651E9AA.9060908@oracle.com>
	
	
	<56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com>
	
	
	<56CB3099.4030108@oracle.com> <56CCA93C.6020807@oracle.com>
	
Message-ID: <56CCFA7F.705@oracle.com>

Hi Vladimir,

I think it would work fine with the Java launcher, but what about other 
areas, which may rely on the native encodings? Java runtime is in itself 
a "non-Unicode" application, so still there may be the area affected by 
hardcoding "UTF-8" as the native encoding. Have you checked in such 
cases? Sherman, will you comment on this too?

Naoto

On 2/23/16 2:12 PM, Vladimir Shcherbakov wrote:
> Hi Naoto,
>
> 1) The system locale determines which code page is used on the system by default on operating systems that use Unicode as their native encoding (all OSes  from Windows 2000 to Windows 10) to convert text data from Unicode to code page whenever dealing with legacy non-Unicode applications. Only applications that do not use Unicode as their default character-encoding mechanism are affected by this setting; therefore, applications that are already Unicode-encoded can safely ignore the value and functionality of this setting.
>
> 2) The fundamental representation of text in Windows NT-based operating systems is UTF-16, and the WCHAR data type is a UTF-16 code unit. Java launcher, from the other side, uses CHAR as a code unit - so to use UNICODE charset with Java launcher we had to encode entire command line with UTF-8 (convert from UTF-16 to UTF-8). After that step we can state that Java launcher is Unicode-encoded and can safely ignore the value and functionality of the system locale. To let JVM know that we use UTF-8 as a default UNICODE encoding for platform string  - we assign the value to sprops.sun_jnu_encoding property (mac osx does the same) instead of reading system locale code page.
>
> The main idea of the fix was to change the way of how java and javac works with so called platform string on Windows. Before the fix the platform string was read as ANSI encoded - that's why the system locale code page was very important. The sun.jnu.encoding property is responsible for storing the platform string encoding. On Windows the property could be set with the system locale but the system locale doesn't support (by design) UTF-8 or with -Dsun.jnu.encoding switch, but the switch only works with java not with javac, and the switch was useless for ANSI encoded platform string.
>
> Thanks,
> Vladimir.
>
> -----Original Message-----
> From: Naoto Sato [mailto:naoto.sato at oracle.com]
> Sent: Tuesday, February 23, 2016 10:47 AM
> To: Kumar Srinivasan ; Vladimir Shcherbakov ; SHEN,XUEMING 
> Cc: Martin Sawicki ; core-libs-dev Libs 
> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>
> Hello,
>
> Sorry if this has already been discussed, but this is my first time looking at the fix. In java_props_md.c, sprops.sun_jnu_encoding is now always "UTF-8". Is it always the case? What if the system admin switches the locale for "non-Unicode" applications in the Windows control panel?
>
> Naoto
>
> On 2/22/16 8:00 AM, Kumar Srinivasan wrote:
>>
>> Hi Naoto, Sherman,  can you please take a look.
>> I tested with the jprt build and test all tests pass.
>>
>> Hi Vladimir, et. al.,
>>
>> It appears that there has been more simplifications from the previous
>> webrev.04. :-)
>>
>> It would've helped if you highlight the changes you have made from the
>> previous revision, unfortunately this is one of the deficiencies of
>> webrev.
>>
>> There are some inconsistencies in the coding conventions:
>>
>> parse_manifest.c
>> + if (q == 0) return -1;
>>
>> we expect the return to be on the next line.
>>
>> similarly main.c
>>
>> if (0 == q)
>> {
>>
>> I can fix those up. If I were to push this change, who should I
>> attribute the changes to ? ie. in the Contributed-by: line of the
>> commit info ?
>> Please note these have to be email addresses of the contributors.
>>
>> Thanks
>> Kumar
>>
>>> Hi Kumar,
>>>
>>> We posted another web review here:
>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open
>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.05%2f&data=01%7C01%7Cvlashc
>>> h%40microsoft.com%7Cf33316507f214e013a4008d33c81c785%7C72f988bf86f141
>>> af91ab2d7cd011db47%7C1&sdata=%2fTQaWH0KGurgvZcdCQRZHSyaftjlMsW5FVc%2f
>>> 14Wc5fA%3d
>>>
>>> The patch was successfully tested.
>>>
>>> Test details:
>>> * Regression tests folder: jdk/test/tools/launcher/
>>> * Builds were used: windows-x86_64-normal-server-fastdebug,
>>> windows-x86_64-normal-server-release,
>>> windows-x86-normal-server-release;
>>> * Platforms were used:  Windows 7(64 bit), Windows 8.1, Windows
>>> Server
>>> 2012 R2 DC, Windows 10 ;
>>> * System locales were used: English (United States), Persian,
>>> Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia);
>>>
>>> Thanks,
>>> Vladimir.
>>>
>>> -----Original Message-----
>>> From: Martin Sawicki
>>> Sent: Thursday, January 14, 2016 11:34 AM
>>> To: Kumar Srinivasan ; Vladimir
>>> Shcherbakov 
>>> Cc: core-libs-dev Libs ; Naoto Sato
>>> 
>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>
>>> Thanks for the feedback.
>>> Investigating the regression failure.
>>> We'll get back as soon as we figure this out.  (and yes, we'll run
>>> this through some localized Windows VMs)
>>>
>>> Cheers
>>>
>>> -----Original Message-----
>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>> Sent: Tuesday, January 12, 2016 2:35 PM
>>> To: Martin Sawicki ; Vladimir Shcherbakov
>>> 
>>> Cc: core-libs-dev Libs ; Naoto Sato
>>> 
>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>
>>> Hi Martin, Vladimir,
>>>
>>> It was suggested that this patch be tested on localized Windows
>>> machines and/or trying with the various Windows native encodings,
>>> appreciate if you can verify this as well.
>>>
>>> Thanks
>>> Kumar
>>>
>>> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote:
>>>> Hi,
>>>>
>>>> Was on vacation, I started to prepare the patch from webrev.04 for
>>>> integration. Please note: made some adjustments to your patch to
>>>> pass jcheck, ie. usage of tabs and space at line endings, and
>>>> modifications to Copyright dates.
>>>>
>>>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE.
>>>> I have attached a patch to for your reference.
>>>>
>>>> However, there is a regression test failure on Windows,
>>>> jdk/test/tools/launcher/I18NTest.java
>>>>
>>>> ---Test info----
>>>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java
>>>>
>>>> ++++Test Output++++
>>>> javac: file not found: i18nH?lloWorld.java ----End test info-----
>>>>
>>>> Have you run all the launcher regression tests with this changeset ?
>>>>
>>>> Thanks
>>>> Kumar
>>>>
>>>>> Hi Kumar, just wondering if there are any updates on processing
>>>>> this submission.
>>>>> Thanks!
>>>>>
>>>>> -----Original Message-----
>>>>> From: Vladimir Shcherbakov
>>>>> Sent: Wednesday, November 25, 2015 2:38 PM
>>>>> To: Kumar Srinivasan ; Martin
>>>>> Sawicki 
>>>>> Cc: Kirk Shoop ; core-libs-dev Libs
>>>>> 
>>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>>
>>>>> Hi Kumar,
>>>>>
>>>>> Please find updated webreview here:
>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op
>>>>> en
>>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarc
>>>>> in
>>>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f1
>>>>> 41
>>>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNf
>>>>> wn
>>>>> bIw%3d
>>>>>
>>>>> Thanks,
>>>>> Vladimir.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>>> Sent: Sunday, November 22, 2015 8:14 AM
>>>>> To: Martin Sawicki 
>>>>> Cc: Kirk Shoop ; Vladimir Shcherbakov
>>>>> ; core-libs-dev Libs
>>>>> 
>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>
>>>>>
>>>>> Hi Martin, et. al.,
>>>>>
>>>>> Sorry for not getting back earlier, I am very busy right now with
>>>>> my other large commitments for JDK9.
>>>>>
>>>>> I will sponsor this "enhancement/bug fix" sometime in the new year,
>>>>> meanwhile, there is the changeset  [1] which is likely to cause
>>>>> merge conflicts, and perhaps logic issues.
>>>>>
>>>>> Thanks
>>>>> Kumar
>>>>>
>>>>> [1]
>>>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.
>>>>> op
>>>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01
>>>>> %7
>>>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f98
>>>>> 8b
>>>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk
>>>>> 7M
>>>>> g4lt2P0sdzLS%2fT9Q%3d
>>>>>> Hi all
>>>>>> Here's an updated webrev attempting to take into account the
>>>>>> various pieces of feedback we have received:
>>>>>>
>>>>>> Issue:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs.
>>>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m
>>>>>> ic
>>>>>> ro
>>>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab
>>>>>> 2d
>>>>>> 7c
>>>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq
>>>>>> 9E
>>>>>> %3
>>>>>> d
>>>>>> Webrev:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.o
>>>>>> pe
>>>>>> nj
>>>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlas
>>>>>> hc
>>>>>> h%
>>>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f14
>>>>>> 1a
>>>>>> f9
>>>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667B
>>>>>> Jn
>>>>>> WY
>>>>>> 0%3d
>>>>>>
>>>>>> (Vladimir Shcherbakov is now working on this from our side)
>>>>>>
>>>>>> Looking forward to any other feedback.
>>>>>> Thanks
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: core-libs-dev
>>>>>> [mailto:core-libs-dev-bounces at openjdk.java.net]
>>>>>> On Behalf Of Kumar Srinivasan
>>>>>> Sent: Thursday, June 25, 2015 6:26 AM
>>>>>> To: Kirk Shoop (MS OPEN TECH) 
>>>>>> Cc: Valery Kopylov (Akvelon) ;
>>>>>> core-libs-dev Libs 
>>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>
>>>>>> Hi Kirk,
>>>>>>
>>>>>> Thanks for proposing this change.
>>>>>>
>>>>>> If you notice all the posix calls are wrapped in JLI_* this gives
>>>>>> us the ability to use "W" functions.  I almost got it done,
>>>>>> several years ago, but we upgraded to VS2010 and my work based on
>>>>>> VS2003 keeled over, meanwhile my focus was  "shifted" to something else.
>>>>>>
>>>>>> main.c: is really envisioned to be a stub  compiled by the tool
>>>>>> launchers, like java, javac, javah, jar etc. I prefer to see all
>>>>>> the heavy logic in this file moved to the platform specific file
>>>>>> windows/java_md.*
>>>>>>
>>>>>> For the reason specified above we need to move fprintf or any
>>>>>> naked posix calls to JLI_* indirections.
>>>>>>
>>>>>> I don't see any tests ? The tests must be written in java and
>>>>>> placed in jdk/test/tools/launcher, there is a helper framework
>>>>>> TestHelper.java.
>>>>>>
>>>>>> There are other changes in nio, charsets etc, this will be
>>>>>> reviewed by my colleague specializing in that area (Sherman) cc'ed.
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> Kumar
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Issue:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2f
>>>>>>> bu
>>>>>>> gs
>>>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%4
>>>>>>> 0m
>>>>>>> ic
>>>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af9
>>>>>>> 1a
>>>>>>> b2
>>>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2f
>>>>>>> sU
>>>>>>> Iq
>>>>>>> 9E%3d
>>>>>>>
>>>>>>> Webrev:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>>> op
>>>>>>> en
>>>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micr
>>>>>>> os
>>>>>>> of
>>>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d
>>>>>>> 7c
>>>>>>> d0
>>>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0
>>>>>>> M%
>>>>>>> 3d
>>>>>>>
>>>>>>> This webrev intends to address interaction between Windows
>>>>>>> console and java apps.
>>>>>>>
>>>>>>> Two switches were added that change the behavior of the launcher.
>>>>>>> The defaults do not change the launcher behavior.
>>>>>>>
>>>>>>>        -Dwindows.UnicodeConsole=true - switches on Unicode support
>>>>>>> in the Windows console. This optional switch causes the launcher
>>>>>>> to call GetCommandLineW() and parse the arguments in unicode. It
>>>>>>> also modifies how the codepage for console output is selected.
>>>>>>>
>>>>>>>        -Dfile.encoding.unicode="UTF-8" - identifies Unicode
>>>>>>> charset to use; If not specified, UTF-8 is used by default.
>>>>>>> Ignored when windows.UnicodeConsole is not set to true. When the
>>>>>>> first switch is used, this optional switch allows the codepage
>>>>>>> for console output to be controlled.
>>>>>>>
>>>>>>> I would like to get feedback on the approach here and any
>>>>>>> additional work that is required solve these particular Unicode
>>>>>>> issues on Windows.
>>>>>>>
>>>>>>> Kirk
>>


From stuart.marks at oracle.com  Wed Feb 24 01:29:14 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Tue, 23 Feb 2016 17:29:14 -0800
Subject: RFR: 8072727 - add variation of Stream.iterate() that's finite
In-Reply-To: <494585613.20160217114835@gmail.com>
References: <6728448.20160214205343@gmail.com> <56C123D3.8000608@oracle.com>
	<494585613.20160217114835@gmail.com>
Message-ID: <56CD076A.9090703@oracle.com>

Finally getting back to this.

So, now that this has been clarified to model a for-loop, this seems fine. It 
looks like Paul is sponsoring this for you. Great, let's move ahead with it.

As the author of this particular RFE, though, I should say that this isn't what 
I had in mind when I wrote the RFE. :-)

There are two main issues I'm concerned about:

1) modeling loops that aren't for-loops; and

2) modeling loops where the termination condition is known to the stream source 
but isn't part of the stream value.

If covering these requires a different API and RFE, then that's fine. I just 
want to make sure these cases aren't dropped.

For case (1) I think the Collatz example is a good one, since you basically had 
to cheat in order to get it to work with this API, either by appending 1, or by 
introducing a sentinel value. It's not always possible to do this.

The kind of loops I'm interested in look like this:

     do {
         emitValue();
     } while (condition);

or

     while (true) {
         emitValue();
         if (condition)
             break;
         stuff();
     }

Clearly it's possible to rearrange these to fit into the 3-arg iterate() method, 
but it's unnatural to do so. For example, to implement the craps dice game, the 
obvious Java approach is to maintain the game state in an object or even local 
variables that are mutated as the game progresses. To use the 3-arg iterate() 
method, you wrote a State object (a value), which knows how to generate its 
successor State, and from which the int values are extracted. This is a very 
nice functional approach, but I expect that many people will have data sources 
that rely on mutation or external resources. It doesn't seem reasonable to 
require rewriting the source logic in order to produce a stream from it. And in 
general it might be arbitrarily difficult.

It's probably easier to wrap a Spliterator around the source logic and simply 
call it from tryAdvance(). In fact, this is what I'd like to see in a 
stream-source API, to make it easier to create such stream sources without 
having to subclass AbstractSpliterator.

For (2) it's critical that the stream source have control over production of 
values. Placing takeWhile() / takeWhileInclusive() downstream doesn't work, 
since the stream machinery (particularly in parallel) may aggressively pull 
elements from the source before they're presented to takeWhile().

For example, consider parsing int values from a Scanner into an IntStream, until 
there are no more, leaving the scanner at a known state. Suppose there were an 
iterate(Supplier) overload. Then something like this would work:

     Scanner sc = new Scanner("1 2 3 4 foo");
     IntStream.iterate(
             () -> sc.hasNextInt() ? OptionalInt.of(sc.nextInt())
                                   : OptionalInt.empty())
         .forEach(System.out::println);

     assert sc.hasNext();
     System.out.println(sc.next()); // prints "foo"

Boxing things into optionals is a bit cumbersome, and I'm also concerned about 
the overhead. But it illustrates the point that the termination condition is 
separate from the values that go into the stream. In particular there's no 
"seed" value to which a termination condition can be applied. (And "iterate" 
might not be the best name for this method, either.)

Oh well, I think this is turning into another RFE.

s'marks




On 2/16/16 9:48 PM, Tagir F. Valeev wrote:
> Hello, Stuart!
>
> Thank you for your comments.
>
> SM> I'd suggest focusing on the API first before worrying about how to track the
> SM> stream state with booleans, etc. Is the API convenient to use, and how well does
> SM> it support the use cases we envision for it?
>
> As Brian already noted, the most benefit of such signature is the
> resemblance to the good old for loop. Also it's good, because the
> lambdas don't need to maintain external mutable state in this case
> (the state is encapsulated in the current element). Most of your
> proposed examples, however, need to do it as they don't receive the
> existing state. Also I see no reason to create a method which is
> compatible with iterator::hasNext/iterator::next or even
> spliterator::tryAdvance. If you already have a spliterator, you can
> create a stream using StreamSupport.stream(spliterator, false). If you
> have an iterator, you can convert it to spliterator using
> Spliterators.spliterator[UnknownSize]. Well, this probably looks ugly,
> but more flexible and signals that some low-level stuff is performed.
>
> Supplier> is definitely bad in terms of performance.
> Creating a new stream is not nearly free. To illustrate this I wrote a
> simple benchmark which compares .flatMapToInt(OptionalInt::stream)
> with Java8 way .filter(OptionalInt::isPresent).mapToInt(OptionalInt::getAsInt)
>
> Here's source code and full output:
> http://cr.openjdk.java.net/~tvaleev/jmh/optionalstream/
>
> Benchmark                               (n)  Mode  Cnt      Score     Error  Units
> OptionalTest.testOptionalFilterGet       10  avgt   30      0,171 ?   0,011  us/op
> OptionalTest.testOptionalFilterGet     1000  avgt   30      6,295 ?   0,046  us/op
> OptionalTest.testOptionalFilterGet  1000000  avgt   30  12597,706 ?  69,214  us/op
> OptionalTest.testOptionalStream          10  avgt   30      0,330 ?   0,002  us/op
> OptionalTest.testOptionalStream        1000  avgt   30     27,552 ?   0,577  us/op
> OptionalTest.testOptionalStream     1000000  avgt   30  30837,240 ? 812,420  us/op
>
> Involving intermediate streams makes the thing at least twice slower.
> Surely this delay could become negligible in some scenarios, but I
> think it's inacceptable to enforce users to create new source with a
> bunch of streams. At least primitive specializations will become
> meaningless in this case: boxing would eat much less time compared to
> stream creation.
>
> As for elements drawn from the queue, it's much better to use existing
> takeWhile method:
>
> queue.stream().takeWhile(x -> x.equals(sentinel));
>
> True, such approach will not include the sentinel element to the
> result, and there's no easy way to do it with current API. Probably
> additional method (takeWhileInclusive?) could be considered to solve
> such problems. Still, I think, drawing from the queue is not the
> problem which should be solved with new iterate() method.
>
> As for Collatz conjecture, it's quite easy to iterate without trailing
> one:
>
> IntStream.iterate(start, val -> val != 1,
>    val -> val % 2 == 0 ? val / 2 : val * 3 + 1)
>
> If having one is desired, then it would be easier just to append one
> to the stream (even if Collatz conjecture is false we will have an
> infinite stream, so appended one will never appear):
>
> IntStream.concat(
>    IntStream.iterate(start, val -> val != 1,
>        val -> val % 2 == 0 ? val / 2 : val * 3 + 1),
>    IntStream.of(1))
>
> A side note: having IntStream.append(int... numbers) would be really
> nice:
>
> IntStream.iterate(start, val -> val != 1,
>    val -> val % 2 == 0 ? val / 2 : val * 3 + 1).append(1)
>
> Another approach would be to introduce a special stop value (for
> example, -1):
>
> IntStream.iterate(start, val -> val != -1,
>    val -> val == 1 ? -1 : val % 2 == 0 ? val / 2 : val * 3 + 1)
>
> This stream produces Collatz series, including the trailing one.
>
> As for Craps, I never heard about such game. If I understood the rules
> correctly, it's good to represent the state as separate object and
> define state transition via its method. Something like this should
> work:
>
> Random r = new Random();
> IntSupplier dice = () -> r.nextInt(6)+r.nextInt(6)+2;
> class State {
>    int roll, point;
>
>    State(int roll, int point) {
>      this.roll = roll;
>      this.point = point;
>    }
>
>    State() {
>      this(dice.getAsInt(), 0);
>    }
>
>    boolean isStopRound() {
>      return roll == 7 || (point == 0 && (roll > 10 || roll < 4)) || (point != 0 && roll == point);
>    }
>
>    State next() {
>      return isStopRound() ? null : new State(dice.getAsInt(), point == 0 ? roll : point);
>    }
> }
> Stream.iterate(new State(), Objects::nonNull, State::next)
>        .mapToInt(state -> state.roll)
>        .forEach(System.out::println);
>
> With best regards,
> Tagir Valeev.
>
>
> SM> In particular, I can imagine a number of cases where it would be very helpful to
> SM> be able to support an empty stream, or where the computation to produce the
> SM> first element is the same as the computation to produce subsequent elements.
> SM> Requiring a value for the first stream element is at odds with that.
>
> SM> Here are some ideas for use cases to try out:
>
> SM>   - a series of dice rolls representing a round of craps [1]
> SM>   - elements drawn from a queue until the queue is empty or until
> SM>     a sentinel is reached
> SM>   - a sequence of numbers that (probably) terminates but whose length
> SM>     isn't necessarily known in advance (e.g. Collatz sequence [2])
>
> SM> [1] https://en.wikipedia.org/wiki/Craps
>
> SM> [2] https://en.wikipedia.org/wiki/Collatz_conjecture
>
> SM> Note that in some cases the sentinel value that terminates the stream should be
> SM> part of the stream, and in other cases it's not.
>
> SM> I'm sure you can find more uses cases by perusing Stack Overflow. :-)
>
> SM> I'm a bit skeptical of the use of "iterate" for producing a finite stream. There
> SM> are the usual issues with overloading, but there's also potential confusion as
> SM> some forms of iterate() are infinite and others finite. I'll suggest the name
> SM> "produce" instead, but there are surely better terms.
>
> SM> One thing to think about is where the state of the producer is stored. Is it
> SM> expected to be in an argument that's passed to each invocation of the functional
> SM> argument, or is it expected to be captured? I don't think there's an answer in
> SM> isolation; examining use cases would probably shed some light here.
>
> SM> Here are a few API ideas (wildcards elided):
>
> SM> --
>
> SM>  Stream iterate(T seed, Predicate predicate, UnaryOperator f)
>
> SM> The API from your proposal, for comparison purposes.
>
> SM> --
>
> SM>  Stream produce(Supplier>)
>
> SM> Produces elements until empty Optional is returned. This box/unboxes every
> SM> element, maybe(?) alleviated by Valhalla.
>
> SM> --
>
> SM>  Stream produce(BooleanSupplier, Supplier)
>
> SM> Calls the BooleanSupplier; if true the next stream element is what's returned by
> SM> calling the Supplier. If BooleanSupplier returns false, end of stream. If you
> SM> have an iterator already, this enables
>
> SM> produce(iterator::hasNext, iterator::next)
>
> SM> But if you don't have an iterator already, coming up with the functions to
> SM> satisfy the iterator-style protocol is sometimes painful.
>
> SM> --
>
> SM>  Stream produce(Predicate> advancer)
>
> SM> This has an odd signature, but the function is like Spliterator.tryAdvance(). It
> SM> must either call the consumer once and return true, or return false without
> SM> calling the consumer.
>
> SM> --
>
> SM>  Stream produce(Consumer> advancer)
>
> SM> A variation of the above, without a boolean return. The advancer calls the
> SM> consumer one or more times to add elements to the stream. End of stream occurs
> SM> when the advancer doesn't call the consumer.
>
> SM> --
>
> SM>  Stream produce(Supplier>)
>
> SM> A variation of Supplier> where the supplier returns a stream
> SM> containing zero or more elements. The stream terminates if the supplier returns
> SM> an empty stream. There "boxing" overhead here, but we don't seem to be bothered
> SM> by this with flatMap().
>
> SM> --
>
> SM> s'marks
>
>
> SM> On 2/14/16 6:53 AM, Tagir F. Valeev wrote:
>>> Hello!
>>>
>>> I wanted to work on foldLeft, but Brian asked me to take this issue
>>> instead. So here's webrev:
>>> http://cr.openjdk.java.net/~tvaleev/webrev/8072727/r1/
>>>
>>> I don't like iterator-based Stream source implementations, so I made
>>> them AbstractSpliterator-based. I also implemented manually
>>> forEachRemaining as, I believe, this improves the performance in
>>> non-short-circuiting cases.
>>>
>>> I also decided to keep two flags (started and finished) to track the
>>> state. Currently existing implementation of infinite iterate() does
>>> not use started flag, but instead reads one element ahead for
>>> primitive streams. This seems wrong to me and may even lead to
>>> unexpected exceptions (*). I could get rid of "started" flag for
>>> Stream.iterate() using Streams.NONE, but this would make object
>>> implementation different from primitive implementations. It would also
>>> be possible to keep single three-state variable (byte or int,
>>> NOT_STARTED, STARTED, FINISHED), but I doubt that this would improve
>>> the performance or footprint. Having two flags looks more readable to
>>> me.
>>>
>>> Currently existing two-arg iterate methods can now be expressed as a
>>> partial case of the new method:
>>>
>>> public static Stream iterate(final T seed, final UnaryOperator f) {
>>>       return iterate(seed, x -> true, f);
>>> }
>>> (same for primitive streams). I may do this if you think it's
>>> reasonable.
>>>
>>> I created new test class and added new iterate sources to existing
>>> data providers.
>>>
>>> Please review and sponsor!
>>>
>>> With best regards,
>>> Tagir Valeev.
>>>
>>> (*) Consider the following code:
>>>
>>> int[] data = {1,2,3,4,-1};
>>> IntStream.iterate(0, x -> data[x])
>>>            .takeWhile(x -> x >= 0)
>>>            .forEach(System.out::println);
>>>
>>> Currently this unexpectedly throws an AIOOBE, because
>>> IntStream.iterate unnecessarily tries to read one element ahead.
>>>
>


From indeliblebluepen at gmail.com  Wed Feb 24 02:05:35 2016
From: indeliblebluepen at gmail.com (Jason C. McDonald)
Date: Wed, 24 Feb 2016 02:05:35 +0000 (UTC)
Subject: Replacement of Quicksort in java.util.Arrays with new
	=?utf-8?b?RHVhbC1QaXZvdAlRdWlja3NvcnQ=?=
References: <4AAA27EA.8090902@Sun.COM>
Message-ID: 

I think this is the best place to contact the original authors.

The link to Vladimir Yaroslavskiy's original whitepaper describing the
algorithm and its proofs was, unfortunately, broken. Using Archive.org's
Wayback Machine, I was able to get the last known revision. I reformatted
the document in LibreOffice for ease of reading, and fixed some minor
grammatical errors.

I also implemented the algorithm in an open-source (MIT License) C++
library, which I hope to release in the coming few months.

In order to make this algorithm and its proofs more easily accessible, I
would like to make the revised whitepaper publicly and freely available from
my own web servers, but I wanted to check with the original author(s) first.
Furthermore, I wanted to find out if there have been any revisions since the
22 September 2009 version of the whitepaper I acquired.

Thank you in advance!



From mandy.chung at oracle.com  Wed Feb 24 02:28:42 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Tue, 23 Feb 2016 18:28:42 -0800
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: 
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com> <56B9EB17.7020303@oracle.com>
	<56C1E765.7080603@oracle.com> <56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com> <56CC8A4A.9080303@gmail.com>
	
Message-ID: <416730A7-88EE-4855-81E3-99B7E603793F@oracle.com>


> On Feb 23, 2016, at 3:22 PM, Kim Barrett  wrote:
> 
> I'm not sure what to do about any of this; I'm just feeling like maybe
> the proposed API / implementation isn't quite right yet.

I suggest restore the JavaLangRefAccess used by direct byte buffer code and revisit such explicit mechanism until there is a need.

Another concern is that this method gets to run in any arbitrary thread rather than the dedicated reference handler thread / cleaner thread.

Mandy

From vitalyd at gmail.com  Wed Feb 24 02:48:40 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 23 Feb 2016 21:48:40 -0500
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <56CCF815.7010105@cs.oswego.edu>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
Message-ID: 

On Tuesday, February 23, 2016, Doug Lea 
       wrote:

> On 02/23/2016 04:30 PM, Vitaly Davidovich wrote:
>
>> Why not drop the (superfluous, IMO) "on" prefix while you're changing the
>> receiver?
>>
>
> Because then it reads as if the method itself is doing a spinWait.

vs who else logically speaking? We all know there's a runtime underneath
Java, there's no point in explicitly calling that out here.  Again, how is
this different from Thread::yield or any of the other mentioned examples?
This is splitting hairs perhaps but there's no onXXX precedent to follow
and this just throws an oddly looking method name into the mix.

> "onSpinWait" is the only proposed name that no one has said they cannot
> live with. So, live with it :-)

Perhaps that's because the Runtime placement was a more glaring issue? :)
It's livable but I just don't see the point of the prefix (and yes, I read
the description of the intent in the original mail).

>
> -Doug
>
>
>
>> On Tue, Feb 23, 2016 at 4:20 PM, Gil Tene  wrote:
>>
>>
>>> On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com wrote:
>>>>
>>>> 2016/1/28 9:25 -0800, gil at azul.com:
>>>>
>>>>> This thread seems to have "hopped away" to the concurrency-interest
>>>>> list in mid-Dec-2015. This posting is intended to capture a summary of
>>>>> reasoning and some of the discussion there so that we have it in the
>>>>> record in core-libs-dev. Mostly by including the contents of several
>>>>> posts in the continuations of the original thread.
>>>>>
>>>>> See thread continuations here:
>>>>>
>>>>>
>>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
>>>
>>>> and here:
>>>>>
>>>>>
>>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
>>>
>>>>
>>>>> Summary:
>>>>>
>>>>> ...
>>>>>
>>>>
>>>> Thanks for the summary.
>>>>
>>>> I still don't buy the argument that this method belongs in j.l.Runtime.
>>>>
>>>> To say that this method should go there because it's an instruction to
>>>> the run-time system is pretty weak.  I agree with Vitaly [1] that if
>>>> that's the threshold for adding methods to the Runtime class then lots
>>>> of other stuff belongs there as well, including much of what's now in
>>>> java.lang.Thread and java.util.concurrent and, arguably, anything else
>>>> related to interacting with the environment in which the application
>>>> runs (file and network I/O, process manipulation, etc.).
>>>>
>>>> This thread-related method really belongs in either java.lang.Thread or
>>>> java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
>>>> expert-level static methods related to the current thread, one of which
>>>> (Thread::yield) is even a hint, just like this one.  j.u.c.LockSupport
>>>> is even more obviously intended for expert users and hence may be the
>>>> best choice, but I could live with either one.
>>>>
>>>
>>> Ok. In the interest of moving forward, lets settle on:
>>>
>>> Thread.onSpinWait()
>>>
>>> Same logic for the name, different receiver for the event. I can
>>> certainly
>>> live with it, and Doug seems ok with it as well.
>>>
>>> ? Gil.
>>>
>>>
>>
>
>

-- 
Sent from my phone


From martinrb at google.com  Wed Feb 24 02:58:01 2016
From: martinrb at google.com (Martin Buchholz)
Date: Tue, 23 Feb 2016 18:58:01 -0800
Subject: RFR: jsr166 jdk9 integration wave 5
Message-ID: 

A very boring jsr166 integration, focused on reliability.
This one has the promised "even more unnecessarily robust" ThreadLocalRandom.

http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/


From pbenedict at apache.org  Wed Feb 24 03:54:27 2016
From: pbenedict at apache.org (Paul Benedict)
Date: Tue, 23 Feb 2016 21:54:27 -0600
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: 
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
Message-ID: 

The onXXX prefix does have precedent as event handler callbacks, but this
method does not fit that purpose. Thus, I agree dropping "on" is sensible.
On Feb 23, 2016 8:48 PM, "Vitaly Davidovich"  wrote:

> On Tuesday, February 23, 2016, Doug Lea 
       wrote:
>
> > On 02/23/2016 04:30 PM, Vitaly Davidovich wrote:
> >
> >> Why not drop the (superfluous, IMO) "on" prefix while you're changing
> the
> >> receiver?
> >>
> >
> > Because then it reads as if the method itself is doing a spinWait.
>
> vs who else logically speaking? We all know there's a runtime underneath
> Java, there's no point in explicitly calling that out here.  Again, how is
> this different from Thread::yield or any of the other mentioned examples?
> This is splitting hairs perhaps but there's no onXXX precedent to follow
> and this just throws an oddly looking method name into the mix.
>
> > "onSpinWait" is the only proposed name that no one has said they cannot
> > live with. So, live with it :-)
>
> Perhaps that's because the Runtime placement was a more glaring issue? :)
> It's livable but I just don't see the point of the prefix (and yes, I read
> the description of the intent in the original mail).
>
> >
> > -Doug
> >
> >
> >
> >> On Tue, Feb 23, 2016 at 4:20 PM, Gil Tene  wrote:
> >>
> >>
> >>> On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com wrote:
> >>>>
> >>>> 2016/1/28 9:25 -0800, gil at azul.com:
> >>>>
> >>>>> This thread seems to have "hopped away" to the concurrency-interest
> >>>>> list in mid-Dec-2015. This posting is intended to capture a summary
> of
> >>>>> reasoning and some of the discussion there so that we have it in the
> >>>>> record in core-libs-dev. Mostly by including the contents of several
> >>>>> posts in the continuations of the original thread.
> >>>>>
> >>>>> See thread continuations here:
> >>>>>
> >>>>>
> >>>
> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
> >>>
> >>>> and here:
> >>>>>
> >>>>>
> >>>
> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
> >>>
> >>>>
> >>>>> Summary:
> >>>>>
> >>>>> ...
> >>>>>
> >>>>
> >>>> Thanks for the summary.
> >>>>
> >>>> I still don't buy the argument that this method belongs in
> j.l.Runtime.
> >>>>
> >>>> To say that this method should go there because it's an instruction to
> >>>> the run-time system is pretty weak.  I agree with Vitaly [1] that if
> >>>> that's the threshold for adding methods to the Runtime class then lots
> >>>> of other stuff belongs there as well, including much of what's now in
> >>>> java.lang.Thread and java.util.concurrent and, arguably, anything else
> >>>> related to interacting with the environment in which the application
> >>>> runs (file and network I/O, process manipulation, etc.).
> >>>>
> >>>> This thread-related method really belongs in either java.lang.Thread
> or
> >>>> java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
> >>>> expert-level static methods related to the current thread, one of
> which
> >>>> (Thread::yield) is even a hint, just like this one.  j.u.c.LockSupport
> >>>> is even more obviously intended for expert users and hence may be the
> >>>> best choice, but I could live with either one.
> >>>>
> >>>
> >>> Ok. In the interest of moving forward, lets settle on:
> >>>
> >>> Thread.onSpinWait()
> >>>
> >>> Same logic for the name, different receiver for the event. I can
> >>> certainly
> >>> live with it, and Doug seems ok with it as well.
> >>>
> >>> ? Gil.
> >>>
> >>>
> >>
> >
> >
>
> --
> Sent from my phone
>


From vitalyd at gmail.com  Wed Feb 24 04:03:44 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Tue, 23 Feb 2016 23:03:44 -0500
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: 
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
Message-ID: 

Yes, I was going to mention onXXX being common for event handler names (e.g
onMouseClick), but didn't bother for the same reason you mentioned - it's
irrelevant to this situation.

On Tuesday, February 23, 2016, Paul Benedict  wrote:

> The onXXX prefix does have precedent as event handler callbacks, but this
> method does not fit that purpose. Thus, I agree dropping "on" is sensible.
> On Feb 23, 2016 8:48 PM, "Vitaly Davidovich"  > wrote:
>
>> On Tuesday, February 23, 2016, Doug Lea 
      > > wrote:
>>
>> > On 02/23/2016 04:30 PM, Vitaly Davidovich wrote:
>> >
>> >> Why not drop the (superfluous, IMO) "on" prefix while you're changing
>> the
>> >> receiver?
>> >>
>> >
>> > Because then it reads as if the method itself is doing a spinWait.
>>
>> vs who else logically speaking? We all know there's a runtime underneath
>> Java, there's no point in explicitly calling that out here.  Again, how is
>> this different from Thread::yield or any of the other mentioned examples?
>> This is splitting hairs perhaps but there's no onXXX precedent to follow
>> and this just throws an oddly looking method name into the mix.
>>
>> > "onSpinWait" is the only proposed name that no one has said they cannot
>> > live with. So, live with it :-)
>>
>> Perhaps that's because the Runtime placement was a more glaring issue? :)
>> It's livable but I just don't see the point of the prefix (and yes, I read
>> the description of the intent in the original mail).
>>
>> >
>> > -Doug
>> >
>> >
>> >
>> >> On Tue, Feb 23, 2016 at 4:20 PM, Gil Tene > > wrote:
>> >>
>> >>
>> >>> On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com
>>  wrote:
>> >>>>
>> >>>> 2016/1/28 9:25 -0800, gil at azul.com
>> :
>> >>>>
>> >>>>> This thread seems to have "hopped away" to the concurrency-interest
>> >>>>> list in mid-Dec-2015. This posting is intended to capture a summary
>> of
>> >>>>> reasoning and some of the discussion there so that we have it in the
>> >>>>> record in core-libs-dev. Mostly by including the contents of several
>> >>>>> posts in the continuations of the original thread.
>> >>>>>
>> >>>>> See thread continuations here:
>> >>>>>
>> >>>>>
>> >>>
>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
>> >>>
>> >>>> and here:
>> >>>>>
>> >>>>>
>> >>>
>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
>> >>>
>> >>>>
>> >>>>> Summary:
>> >>>>>
>> >>>>> ...
>> >>>>>
>> >>>>
>> >>>> Thanks for the summary.
>> >>>>
>> >>>> I still don't buy the argument that this method belongs in
>> j.l.Runtime.
>> >>>>
>> >>>> To say that this method should go there because it's an instruction
>> to
>> >>>> the run-time system is pretty weak.  I agree with Vitaly [1] that if
>> >>>> that's the threshold for adding methods to the Runtime class then
>> lots
>> >>>> of other stuff belongs there as well, including much of what's now in
>> >>>> java.lang.Thread and java.util.concurrent and, arguably, anything
>> else
>> >>>> related to interacting with the environment in which the application
>> >>>> runs (file and network I/O, process manipulation, etc.).
>> >>>>
>> >>>> This thread-related method really belongs in either java.lang.Thread
>> or
>> >>>> java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
>> >>>> expert-level static methods related to the current thread, one of
>> which
>> >>>> (Thread::yield) is even a hint, just like this one.
>> j.u.c.LockSupport
>> >>>> is even more obviously intended for expert users and hence may be the
>> >>>> best choice, but I could live with either one.
>> >>>>
>> >>>
>> >>> Ok. In the interest of moving forward, lets settle on:
>> >>>
>> >>> Thread.onSpinWait()
>> >>>
>> >>> Same logic for the name, different receiver for the event. I can
>> >>> certainly
>> >>> live with it, and Doug seems ok with it as well.
>> >>>
>> >>> ? Gil.
>> >>>
>> >>>
>> >>
>> >
>> >
>>
>> --
>> Sent from my phone
>>
>

-- 
Sent from my phone


From xueming.shen at oracle.com  Wed Feb 24 04:53:40 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Tue, 23 Feb 2016 20:53:40 -0800
Subject: RFR 8124977 cmdline encoding challenges on Windows
In-Reply-To: <56CCFA7F.705@oracle.com>
References: 
	
	<558C017A.2020708@oracle.com>
	
	<5651E9AA.9060908@oracle.com>
	
	
	<56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com>
	
	
	<56CB3099.4030108@oracle.com> <56CCA93C.6020807@oracle.com>
	
	<56CCFA7F.705@oracle.com>
Message-ID: <56CD3754.1050106@oracle.com>

Vladimir,

sun.jnu.encoding is used by 
JNU_NewStringPlatform/JNU_GetStringPlatformChars. The JNU_ pair
is "widely" used by the various native library code to convert between 
the jstring and native char*,
with the assumption that the "platform encoding" for the native char* is 
the "default" encoding
used by the underlying platform/os APIs that takes char* parameters or 
return char* values, in
case of Windows, it's the code page decided by the system locale. We 
have migrated certain areas
completely to use the "W" version/WChar APIs, such as the java.io, the 
system properties initialization,
but I'm think lots of areas still work on the "A" apis, especially I 
think the "char*" interface between
the jvm and the libraries is still the the "ansi" codepage, not the 
utf8. Those work on utf8 have
their names explicitly named as "xyzUTF" or similar.

For example, the "java_home_dir" path used in 
libjava/TimeZone.c/getSystemTimeZoneID/
TimeZone_md.c/findJavaTZ_md is encoded frm jstring java_home to char* 
via JNU_GetSTringPlatformChars.
Simply change/hardcode the jnu_sun.encoding to utf8 probably will cause 
the timezone code
stop to work if the java_home_dir path has some non-ascii characters in 
it (the jdk/jre is installed
in a Japanese/Chinese directory, for example).

A quick "grep" indicates java.desktop/windows/native/libawt/windows 
package has a heavily
usage of the JNU_ pair as well. I'm not sure if this awt implementation 
is still being used though :-)

Before we clear all these internal "StringPlatform" use cases (I'm not 
sure if they are also used
by external), I don't think we can simply set the sun.jnu.encoding to 
utf8, though it's very attractive.

Thanks,
-Sherman

On 2/23/16 4:34 PM, Naoto Sato wrote:
> Hi Vladimir,
>
> I think it would work fine with the Java launcher, but what about 
> other areas, which may rely on the native encodings? Java runtime is 
> in itself a "non-Unicode" application, so still there may be the area 
> affected by hardcoding "UTF-8" as the native encoding. Have you 
> checked in such cases? Sherman, will you comment on this too?
>
> Naoto
>
> On 2/23/16 2:12 PM, Vladimir Shcherbakov wrote:
>> Hi Naoto,
>>
>> 1) The system locale determines which code page is used on the system 
>> by default on operating systems that use Unicode as their native 
>> encoding (all OSes  from Windows 2000 to Windows 10) to convert text 
>> data from Unicode to code page whenever dealing with legacy 
>> non-Unicode applications. Only applications that do not use Unicode 
>> as their default character-encoding mechanism are affected by this 
>> setting; therefore, applications that are already Unicode-encoded can 
>> safely ignore the value and functionality of this setting.
>>
>> 2) The fundamental representation of text in Windows NT-based 
>> operating systems is UTF-16, and the WCHAR data type is a UTF-16 code 
>> unit. Java launcher, from the other side, uses CHAR as a code unit - 
>> so to use UNICODE charset with Java launcher we had to encode entire 
>> command line with UTF-8 (convert from UTF-16 to UTF-8). After that 
>> step we can state that Java launcher is Unicode-encoded and can 
>> safely ignore the value and functionality of the system locale. To 
>> let JVM know that we use UTF-8 as a default UNICODE encoding for 
>> platform string  - we assign the value to sprops.sun_jnu_encoding 
>> property (mac osx does the same) instead of reading system locale 
>> code page.
>>
>> The main idea of the fix was to change the way of how java and javac 
>> works with so called platform string on Windows. Before the fix the 
>> platform string was read as ANSI encoded - that's why the system 
>> locale code page was very important. The sun.jnu.encoding property is 
>> responsible for storing the platform string encoding. On Windows the 
>> property could be set with the system locale but the system locale 
>> doesn't support (by design) UTF-8 or with -Dsun.jnu.encoding switch, 
>> but the switch only works with java not with javac, and the switch 
>> was useless for ANSI encoded platform string.
>>
>> Thanks,
>> Vladimir.
>>
>> -----Original Message-----
>> From: Naoto Sato [mailto:naoto.sato at oracle.com]
>> Sent: Tuesday, February 23, 2016 10:47 AM
>> To: Kumar Srinivasan ; Vladimir 
>> Shcherbakov ; SHEN,XUEMING 
>> 
>> Cc: Martin Sawicki ; core-libs-dev Libs 
>> 
>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Hello,
>>
>> Sorry if this has already been discussed, but this is my first time 
>> looking at the fix. In java_props_md.c, sprops.sun_jnu_encoding is 
>> now always "UTF-8". Is it always the case? What if the system admin 
>> switches the locale for "non-Unicode" applications in the Windows 
>> control panel?
>>
>> Naoto
>>
>> On 2/22/16 8:00 AM, Kumar Srinivasan wrote:
>>>
>>> Hi Naoto, Sherman,  can you please take a look.
>>> I tested with the jprt build and test all tests pass.
>>>
>>> Hi Vladimir, et. al.,
>>>
>>> It appears that there has been more simplifications from the previous
>>> webrev.04. :-)
>>>
>>> It would've helped if you highlight the changes you have made from the
>>> previous revision, unfortunately this is one of the deficiencies of
>>> webrev.
>>>
>>> There are some inconsistencies in the coding conventions:
>>>
>>> parse_manifest.c
>>> + if (q == 0) return -1;
>>>
>>> we expect the return to be on the next line.
>>>
>>> similarly main.c
>>>
>>> if (0 == q)
>>> {
>>>
>>> I can fix those up. If I were to push this change, who should I
>>> attribute the changes to ? ie. in the Contributed-by: line of the
>>> commit info ?
>>> Please note these have to be email addresses of the contributors.
>>>
>>> Thanks
>>> Kumar
>>>
>>>> Hi Kumar,
>>>>
>>>> We posted another web review here:
>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open
>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.05%2f&data=01%7C01%7Cvlashc
>>>> h%40microsoft.com%7Cf33316507f214e013a4008d33c81c785%7C72f988bf86f141
>>>> af91ab2d7cd011db47%7C1&sdata=%2fTQaWH0KGurgvZcdCQRZHSyaftjlMsW5FVc%2f
>>>> 14Wc5fA%3d
>>>>
>>>> The patch was successfully tested.
>>>>
>>>> Test details:
>>>> * Regression tests folder: jdk/test/tools/launcher/
>>>> * Builds were used: windows-x86_64-normal-server-fastdebug,
>>>> windows-x86_64-normal-server-release,
>>>> windows-x86-normal-server-release;
>>>> * Platforms were used:  Windows 7(64 bit), Windows 8.1, Windows
>>>> Server
>>>> 2012 R2 DC, Windows 10 ;
>>>> * System locales were used: English (United States), Persian,
>>>> Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia);
>>>>
>>>> Thanks,
>>>> Vladimir.
>>>>
>>>> -----Original Message-----
>>>> From: Martin Sawicki
>>>> Sent: Thursday, January 14, 2016 11:34 AM
>>>> To: Kumar Srinivasan ; Vladimir
>>>> Shcherbakov 
>>>> Cc: core-libs-dev Libs ; Naoto Sato
>>>> 
>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>> Thanks for the feedback.
>>>> Investigating the regression failure.
>>>> We'll get back as soon as we figure this out.  (and yes, we'll run
>>>> this through some localized Windows VMs)
>>>>
>>>> Cheers
>>>>
>>>> -----Original Message-----
>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>> Sent: Tuesday, January 12, 2016 2:35 PM
>>>> To: Martin Sawicki ; Vladimir Shcherbakov
>>>> 
>>>> Cc: core-libs-dev Libs ; Naoto Sato
>>>> 
>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>> Hi Martin, Vladimir,
>>>>
>>>> It was suggested that this patch be tested on localized Windows
>>>> machines and/or trying with the various Windows native encodings,
>>>> appreciate if you can verify this as well.
>>>>
>>>> Thanks
>>>> Kumar
>>>>
>>>> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote:
>>>>> Hi,
>>>>>
>>>>> Was on vacation, I started to prepare the patch from webrev.04 for
>>>>> integration. Please note: made some adjustments to your patch to
>>>>> pass jcheck, ie. usage of tabs and space at line endings, and
>>>>> modifications to Copyright dates.
>>>>>
>>>>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE.
>>>>> I have attached a patch to for your reference.
>>>>>
>>>>> However, there is a regression test failure on Windows,
>>>>> jdk/test/tools/launcher/I18NTest.java
>>>>>
>>>>> ---Test info----
>>>>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java
>>>>>
>>>>> ++++Test Output++++
>>>>> javac: file not found: i18nH?lloWorld.java ----End test info-----
>>>>>
>>>>> Have you run all the launcher regression tests with this changeset ?
>>>>>
>>>>> Thanks
>>>>> Kumar
>>>>>
>>>>>> Hi Kumar, just wondering if there are any updates on processing
>>>>>> this submission.
>>>>>> Thanks!
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Vladimir Shcherbakov
>>>>>> Sent: Wednesday, November 25, 2015 2:38 PM
>>>>>> To: Kumar Srinivasan ; Martin
>>>>>> Sawicki 
>>>>>> Cc: Kirk Shoop ; core-libs-dev Libs
>>>>>> 
>>>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>
>>>>>> Hi Kumar,
>>>>>>
>>>>>> Please find updated webreview here:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op
>>>>>> en
>>>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarc
>>>>>> in
>>>>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f1
>>>>>> 41
>>>>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNf
>>>>>> wn
>>>>>> bIw%3d
>>>>>>
>>>>>> Thanks,
>>>>>> Vladimir.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>>>> Sent: Sunday, November 22, 2015 8:14 AM
>>>>>> To: Martin Sawicki 
>>>>>> Cc: Kirk Shoop ; Vladimir Shcherbakov
>>>>>> ; core-libs-dev Libs
>>>>>> 
>>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>
>>>>>>
>>>>>> Hi Martin, et. al.,
>>>>>>
>>>>>> Sorry for not getting back earlier, I am very busy right now with
>>>>>> my other large commitments for JDK9.
>>>>>>
>>>>>> I will sponsor this "enhancement/bug fix" sometime in the new year,
>>>>>> meanwhile, there is the changeset  [1] which is likely to cause
>>>>>> merge conflicts, and perhaps logic issues.
>>>>>>
>>>>>> Thanks
>>>>>> Kumar
>>>>>>
>>>>>> [1]
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.
>>>>>> op
>>>>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01
>>>>>> %7
>>>>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f98
>>>>>> 8b
>>>>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk
>>>>>> 7M
>>>>>> g4lt2P0sdzLS%2fT9Q%3d
>>>>>>> Hi all
>>>>>>> Here's an updated webrev attempting to take into account the
>>>>>>> various pieces of feedback we have received:
>>>>>>>
>>>>>>> Issue:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. 
>>>>>>>
>>>>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m
>>>>>>> ic
>>>>>>> ro
>>>>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab
>>>>>>> 2d
>>>>>>> 7c
>>>>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq
>>>>>>> 9E
>>>>>>> %3
>>>>>>> d
>>>>>>> Webrev:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.o
>>>>>>> pe
>>>>>>> nj
>>>>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlas
>>>>>>> hc
>>>>>>> h%
>>>>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f14
>>>>>>> 1a
>>>>>>> f9
>>>>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667B
>>>>>>> Jn
>>>>>>> WY
>>>>>>> 0%3d
>>>>>>>
>>>>>>> (Vladimir Shcherbakov is now working on this from our side)
>>>>>>>
>>>>>>> Looking forward to any other feedback.
>>>>>>> Thanks
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: core-libs-dev
>>>>>>> [mailto:core-libs-dev-bounces at openjdk.java.net]
>>>>>>> On Behalf Of Kumar Srinivasan
>>>>>>> Sent: Thursday, June 25, 2015 6:26 AM
>>>>>>> To: Kirk Shoop (MS OPEN TECH) 
>>>>>>> Cc: Valery Kopylov (Akvelon) ;
>>>>>>> core-libs-dev Libs 
>>>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>>
>>>>>>> Hi Kirk,
>>>>>>>
>>>>>>> Thanks for proposing this change.
>>>>>>>
>>>>>>> If you notice all the posix calls are wrapped in JLI_* this gives
>>>>>>> us the ability to use "W" functions.  I almost got it done,
>>>>>>> several years ago, but we upgraded to VS2010 and my work based on
>>>>>>> VS2003 keeled over, meanwhile my focus was  "shifted" to 
>>>>>>> something else.
>>>>>>>
>>>>>>> main.c: is really envisioned to be a stub  compiled by the tool
>>>>>>> launchers, like java, javac, javah, jar etc. I prefer to see all
>>>>>>> the heavy logic in this file moved to the platform specific file
>>>>>>> windows/java_md.*
>>>>>>>
>>>>>>> For the reason specified above we need to move fprintf or any
>>>>>>> naked posix calls to JLI_* indirections.
>>>>>>>
>>>>>>> I don't see any tests ? The tests must be written in java and
>>>>>>> placed in jdk/test/tools/launcher, there is a helper framework
>>>>>>> TestHelper.java.
>>>>>>>
>>>>>>> There are other changes in nio, charsets etc, this will be
>>>>>>> reviewed by my colleague specializing in that area (Sherman) cc'ed.
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Kumar
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Issue:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2f
>>>>>>>> bu
>>>>>>>> gs
>>>>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%4
>>>>>>>> 0m
>>>>>>>> ic
>>>>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af9
>>>>>>>> 1a
>>>>>>>> b2
>>>>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2f
>>>>>>>> sU
>>>>>>>> Iq
>>>>>>>> 9E%3d
>>>>>>>>
>>>>>>>> Webrev:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>>>> op
>>>>>>>> en
>>>>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micr
>>>>>>>> os
>>>>>>>> of
>>>>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d
>>>>>>>> 7c
>>>>>>>> d0
>>>>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0
>>>>>>>> M%
>>>>>>>> 3d
>>>>>>>>
>>>>>>>> This webrev intends to address interaction between Windows
>>>>>>>> console and java apps.
>>>>>>>>
>>>>>>>> Two switches were added that change the behavior of the launcher.
>>>>>>>> The defaults do not change the launcher behavior.
>>>>>>>>
>>>>>>>>        -Dwindows.UnicodeConsole=true - switches on Unicode support
>>>>>>>> in the Windows console. This optional switch causes the launcher
>>>>>>>> to call GetCommandLineW() and parse the arguments in unicode. It
>>>>>>>> also modifies how the codepage for console output is selected.
>>>>>>>>
>>>>>>>>        -Dfile.encoding.unicode="UTF-8" - identifies Unicode
>>>>>>>> charset to use; If not specified, UTF-8 is used by default.
>>>>>>>> Ignored when windows.UnicodeConsole is not set to true. When the
>>>>>>>> first switch is used, this optional switch allows the codepage
>>>>>>>> for console output to be controlled.
>>>>>>>>
>>>>>>>> I would like to get feedback on the approach here and any
>>>>>>>> additional work that is required solve these particular Unicode
>>>>>>>> issues on Windows.
>>>>>>>>
>>>>>>>> Kirk
>>>



From gil at azul.com  Wed Feb 24 06:22:48 2016
From: gil at azul.com (Gil Tene)
Date: Wed, 24 Feb 2016 06:22:48 +0000
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: 
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
	
Message-ID: <7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>

onXXX is VERY relevant to this situation. And it's exactly that precedent and common convention for event notification calls that drove the choice of onSpinWait as a method name. onSpinWait() it is delivering a notification [from the calling code to the Runtime, or the Thread] about what is going on: a spin wait is in progress.

The fundamental difference between onSpinWait() and Thread.yield() is that yield() means "please yield here", while onSpinWait means "I am in the middle of busily executing a spin wait that I need no help with, and certainly don't want to be blocking in. But you might want to know about that and do something with that knowledge. Preferably something that improves my performance." Basically, onSpinWait() is delivering a notification of a situation, and the runtime has a chance to react to that.

Dropping the "on" would completely change the meaning. Something called "spinWait" *is* like yield(). It would/should mean "please spin wait here". But that's not what spin wait constructs need or want in a call. Unlike yield(), spin wait constructs use their own logic for spinning, and usually need to retain control over the logic. They just need to let someone know that spinning is going on, in the hope that performance of the spinwait construct (with whatever logic it uses) might be improved.

Note that the runtime reactions to being notified the calling code is in the middle of a spin wait could range from doing nothing, to doing something with instruction execution (e.g. execute an x86 pause instruction), to doing something more sophisticated (e.g. trying to dedicate a core to the spinning thread and steer all other thread and interrupts away from it to improve it's reaction time stats). What will actually happen is up to the runtime, and intentionally not specified. Per the JavaDoc, if the runtime does take action when it receives the onSpinWait() event, the action is intended "to improve the performance of invoking spin-wait loop constructions".

You can hopefully/probably see why Runtime was a logical choice as the receiver of the event. Since Thread is also acceptable and defensible as the receiver of the event, I'm happy to go with it if it gets us over this name-can-of-worms thing. But please please please don't let us go into a whole method name redo again. We've done that on and off for 3+ months, and what we have is both logical and "not unacceptable": onSpinWait().

Let's just go with it.

Thread.onSpinWait() FTW!

> On Feb 23, 2016, at 8:03 PM, Vitaly Davidovich  wrote:
> 
> Yes, I was going to mention onXXX being common for event handler names (e.g
> onMouseClick), but didn't bother for the same reason you mentioned - it's
> irrelevant to this situation.
> 
> On Tuesday, February 23, 2016, Paul Benedict  wrote:
> 
>> The onXXX prefix does have precedent as event handler callbacks, but this
>> method does not fit that purpose. Thus, I agree dropping "on" is sensible.
>> On Feb 23, 2016 8:48 PM, "Vitaly Davidovich" > > wrote:
>> 
>>> On Tuesday, February 23, 2016, Doug Lea 
      >> > wrote:
>>> 
>>>> On 02/23/2016 04:30 PM, Vitaly Davidovich wrote:
>>>> 
>>>>> Why not drop the (superfluous, IMO) "on" prefix while you're changing
>>> the
>>>>> receiver?
>>>>> 
>>>> 
>>>> Because then it reads as if the method itself is doing a spinWait.
>>> 
>>> vs who else logically speaking? We all know there's a runtime underneath
>>> Java, there's no point in explicitly calling that out here.  Again, how is
>>> this different from Thread::yield or any of the other mentioned examples?
>>> This is splitting hairs perhaps but there's no onXXX precedent to follow
>>> and this just throws an oddly looking method name into the mix.
>>> 
>>>> "onSpinWait" is the only proposed name that no one has said they cannot
>>>> live with. So, live with it :-)
>>> 
>>> Perhaps that's because the Runtime placement was a more glaring issue? :)
>>> It's livable but I just don't see the point of the prefix (and yes, I read
>>> the description of the intent in the original mail).
>>> 
>>>> 
>>>> -Doug
>>>> 
>>>> 
>>>> 
>>>>> On Tue, Feb 23, 2016 at 4:20 PM, Gil Tene >> > wrote:
>>>>> 
>>>>> 
>>>>>> On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com
>>>  wrote:
>>>>>>> 
>>>>>>> 2016/1/28 9:25 -0800, gil at azul.com
>>> :
>>>>>>> 
>>>>>>>> This thread seems to have "hopped away" to the concurrency-interest
>>>>>>>> list in mid-Dec-2015. This posting is intended to capture a summary
>>> of
>>>>>>>> reasoning and some of the discussion there so that we have it in the
>>>>>>>> record in core-libs-dev. Mostly by including the contents of several
>>>>>>>> posts in the continuations of the original thread.
>>>>>>>> 
>>>>>>>> See thread continuations here:
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
>>>>>> 
>>>>>>> and here:
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
>>>>>> 
>>>>>>> 
>>>>>>>> Summary:
>>>>>>>> 
>>>>>>>> ...
>>>>>>>> 
>>>>>>> 
>>>>>>> Thanks for the summary.
>>>>>>> 
>>>>>>> I still don't buy the argument that this method belongs in
>>> j.l.Runtime.
>>>>>>> 
>>>>>>> To say that this method should go there because it's an instruction
>>> to
>>>>>>> the run-time system is pretty weak.  I agree with Vitaly [1] that if
>>>>>>> that's the threshold for adding methods to the Runtime class then
>>> lots
>>>>>>> of other stuff belongs there as well, including much of what's now in
>>>>>>> java.lang.Thread and java.util.concurrent and, arguably, anything
>>> else
>>>>>>> related to interacting with the environment in which the application
>>>>>>> runs (file and network I/O, process manipulation, etc.).
>>>>>>> 
>>>>>>> This thread-related method really belongs in either java.lang.Thread
>>> or
>>>>>>> java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
>>>>>>> expert-level static methods related to the current thread, one of
>>> which
>>>>>>> (Thread::yield) is even a hint, just like this one.
>>> j.u.c.LockSupport
>>>>>>> is even more obviously intended for expert users and hence may be the
>>>>>>> best choice, but I could live with either one.
>>>>>>> 
>>>>>> 
>>>>>> Ok. In the interest of moving forward, lets settle on:
>>>>>> 
>>>>>> Thread.onSpinWait()
>>>>>> 
>>>>>> Same logic for the name, different receiver for the event. I can
>>>>>> certainly
>>>>>> live with it, and Doug seems ok with it as well.
>>>>>> 
>>>>>> ? Gil.
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> --
>>> Sent from my phone
>>> 
>> 
> 
> --
> Sent from my phone



From xueming.shen at oracle.com  Wed Feb 24 06:43:04 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Tue, 23 Feb 2016 22:43:04 -0800
Subject: RFR 8124977 cmdline encoding challenges on Windows
In-Reply-To: 
References: 
	
	<558C017A.2020708@oracle.com>
	
	<5651E9AA.9060908@oracle.com>
	
	
	<56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com>
	
	
	<56CB3099.4030108@oracle.com> <56CCA93C.6020807@oracle.com>
	
	<56CCFA7F.705@oracle.com> <56CD3754.1050106@oracle.com>
	
Message-ID: <56CD50F8.9030805@oracle.com>

On 2/23/16 9:52 PM, Vladimir Shcherbakov wrote:
> Hi Sherman,
>
> 1) If you can point out the regression test cases that are compromised by the fix - it would be very helpful;

I don't have a specific regression test for now. Guess running a "old" 
awt app with
Japanese/Chinese characters as menu item might help show the issue? if I 
read those
awt lines correctly.

> 2) From my understanding you can change default encoding by starting java with -Dsun.jnu.encoding=UTF-8 - this is well known feature that never caused problems (javac doesn't have such a switch );

I have been telling people for decade -Dfile.encoding is not a 
"supported" usage/feature :-) as it does
cause "inconsistent" behavior on different platforms with different use 
scenario. And then the sun.jnu.encoding,
definitely with no intention to be specified via -D. That's a pure 
contract between the java runtime and
the underlying os on how the string/text should be encoded when using 
those platform APIs. I think I had
forwarded the internal CCC doc for sun.jnu.encoding a while back, no 
-Dsun.jnu.encoding=XYZ please :-)

> 3) If you state that java is non-Unicode on Windows by nature - the issue  JDK-8124977 is a feature not a bug  :)

Ideally we should run the java runtime as a unicode app. launcher is not 
a big issue. The
concern is the interface with jvm for those "char*".

Sherman

>
> Thanks,
> Vladimir.
>
> -----Original Message-----
> From: Xueming Shen [mailto:xueming.shen at oracle.com]
> Sent: Tuesday, February 23, 2016 8:54 PM
> To: Vladimir Shcherbakov 
> Cc: Naoto Sato ; Kumar Srinivasan ; Martin Sawicki ; core-libs-dev Libs 
> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>
> Vladimir,
>
> sun.jnu.encoding is used by
> JNU_NewStringPlatform/JNU_GetStringPlatformChars. The JNU_ pair is "widely" used by the various native library code to convert between the jstring and native char*, with the assumption that the "platform encoding" for the native char* is the "default" encoding used by the underlying platform/os APIs that takes char* parameters or return char* values, in case of Windows, it's the code page decided by the system locale. We have migrated certain areas completely to use the "W" version/WChar APIs, such as the https://na01.safelinks.protection.outlook.com/?url=java.io&data=01%7c01%7cvlashch%40microsoft.com%7c635061d867af4ad4105008d33cd679e7%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=l4G1yzKKhniPRYJvBsGxchsBZvuWliVq8tILa0pLoY8%3d, the system properties initialization, but I'm think lots of areas still work on the "A" apis, especially I think the "char*" interface between the jvm and the libraries is still the the "ansi" codepage, not the utf8. Those work on utf8 have their names explicitly named as "xyzUTF" or similar.
>
> For example, the "java_home_dir" path used in libjava/TimeZone.c/getSystemTimeZoneID/
> TimeZone_md.c/findJavaTZ_md is encoded frm jstring java_home to char* via JNU_GetSTringPlatformChars.
> Simply change/hardcode the jnu_sun.encoding to utf8 probably will cause the timezone code stop to work if the java_home_dir path has some non-ascii characters in it (the jdk/jre is installed in a Japanese/Chinese directory, for example).
>
> A quick "grep" indicates java.desktop/windows/native/libawt/windows
> package has a heavily
> usage of the JNU_ pair as well. I'm not sure if this awt implementation is still being used though :-)
>
> Before we clear all these internal "StringPlatform" use cases (I'm not sure if they are also used by external), I don't think we can simply set the sun.jnu.encoding to utf8, though it's very attractive.
>
> Thanks,
> -Sherman
>
> On 2/23/16 4:34 PM, Naoto Sato wrote:
>> Hi Vladimir,
>>
>> I think it would work fine with the Java launcher, but what about
>> other areas, which may rely on the native encodings? Java runtime is
>> in itself a "non-Unicode" application, so still there may be the area
>> affected by hardcoding "UTF-8" as the native encoding. Have you
>> checked in such cases? Sherman, will you comment on this too?
>>
>> Naoto
>>
>> On 2/23/16 2:12 PM, Vladimir Shcherbakov wrote:
>>> Hi Naoto,
>>>
>>> 1) The system locale determines which code page is used on the system
>>> by default on operating systems that use Unicode as their native
>>> encoding (all OSes  from Windows 2000 to Windows 10) to convert text
>>> data from Unicode to code page whenever dealing with legacy
>>> non-Unicode applications. Only applications that do not use Unicode
>>> as their default character-encoding mechanism are affected by this
>>> setting; therefore, applications that are already Unicode-encoded can
>>> safely ignore the value and functionality of this setting.
>>>
>>> 2) The fundamental representation of text in Windows NT-based
>>> operating systems is UTF-16, and the WCHAR data type is a UTF-16 code
>>> unit. Java launcher, from the other side, uses CHAR as a code unit -
>>> so to use UNICODE charset with Java launcher we had to encode entire
>>> command line with UTF-8 (convert from UTF-16 to UTF-8). After that
>>> step we can state that Java launcher is Unicode-encoded and can
>>> safely ignore the value and functionality of the system locale. To
>>> let JVM know that we use UTF-8 as a default UNICODE encoding for
>>> platform string  - we assign the value to sprops.sun_jnu_encoding
>>> property (mac osx does the same) instead of reading system locale
>>> code page.
>>>
>>> The main idea of the fix was to change the way of how java and javac
>>> works with so called platform string on Windows. Before the fix the
>>> platform string was read as ANSI encoded - that's why the system
>>> locale code page was very important. The sun.jnu.encoding property is
>>> responsible for storing the platform string encoding. On Windows the
>>> property could be set with the system locale but the system locale
>>> doesn't support (by design) UTF-8 or with -Dsun.jnu.encoding switch,
>>> but the switch only works with java not with javac, and the switch
>>> was useless for ANSI encoded platform string.
>>>
>>> Thanks,
>>> Vladimir.
>>>
>>> -----Original Message-----
>>> From: Naoto Sato [mailto:naoto.sato at oracle.com]
>>> Sent: Tuesday, February 23, 2016 10:47 AM
>>> To: Kumar Srinivasan ; Vladimir
>>> Shcherbakov ; SHEN,XUEMING
>>> 
>>> Cc: Martin Sawicki ; core-libs-dev Libs
>>> 
>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>
>>> Hello,
>>>
>>> Sorry if this has already been discussed, but this is my first time
>>> looking at the fix. In java_props_md.c, sprops.sun_jnu_encoding is
>>> now always "UTF-8". Is it always the case? What if the system admin
>>> switches the locale for "non-Unicode" applications in the Windows
>>> control panel?
>>>
>>> Naoto
>>>
>>> On 2/22/16 8:00 AM, Kumar Srinivasan wrote:
>>>> Hi Naoto, Sherman,  can you please take a look.
>>>> I tested with the jprt build and test all tests pass.
>>>>
>>>> Hi Vladimir, et. al.,
>>>>
>>>> It appears that there has been more simplifications from the
>>>> previous webrev.04. :-)
>>>>
>>>> It would've helped if you highlight the changes you have made from
>>>> the previous revision, unfortunately this is one of the deficiencies
>>>> of webrev.
>>>>
>>>> There are some inconsistencies in the coding conventions:
>>>>
>>>> parse_manifest.c
>>>> + if (q == 0) return -1;
>>>>
>>>> we expect the return to be on the next line.
>>>>
>>>> similarly main.c
>>>>
>>>> if (0 == q)
>>>> {
>>>>
>>>> I can fix those up. If I were to push this change, who should I
>>>> attribute the changes to ? ie. in the Contributed-by: line of the
>>>> commit info ?
>>>> Please note these have to be email addresses of the contributors.
>>>>
>>>> Thanks
>>>> Kumar
>>>>
>>>>> Hi Kumar,
>>>>>
>>>>> We posted another web review here:
>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op
>>>>> en
>>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.05%2f&data=01%7C01%7Cvlas
>>>>> hc
>>>>> h%40microsoft.com%7Cf33316507f214e013a4008d33c81c785%7C72f988bf86f1
>>>>> 41
>>>>> af91ab2d7cd011db47%7C1&sdata=%2fTQaWH0KGurgvZcdCQRZHSyaftjlMsW5FVc%
>>>>> 2f
>>>>> 14Wc5fA%3d
>>>>>
>>>>> The patch was successfully tested.
>>>>>
>>>>> Test details:
>>>>> * Regression tests folder: jdk/test/tools/launcher/
>>>>> * Builds were used: windows-x86_64-normal-server-fastdebug,
>>>>> windows-x86_64-normal-server-release,
>>>>> windows-x86-normal-server-release;
>>>>> * Platforms were used:  Windows 7(64 bit), Windows 8.1, Windows
>>>>> Server
>>>>> 2012 R2 DC, Windows 10 ;
>>>>> * System locales were used: English (United States), Persian,
>>>>> Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia);
>>>>>
>>>>> Thanks,
>>>>> Vladimir.
>>>>>
>>>>> -----Original Message-----
>>>>> From: Martin Sawicki
>>>>> Sent: Thursday, January 14, 2016 11:34 AM
>>>>> To: Kumar Srinivasan ; Vladimir
>>>>> Shcherbakov 
>>>>> Cc: core-libs-dev Libs ; Naoto Sato
>>>>> 
>>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>>
>>>>> Thanks for the feedback.
>>>>> Investigating the regression failure.
>>>>> We'll get back as soon as we figure this out.  (and yes, we'll run
>>>>> this through some localized Windows VMs)
>>>>>
>>>>> Cheers
>>>>>
>>>>> -----Original Message-----
>>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>>> Sent: Tuesday, January 12, 2016 2:35 PM
>>>>> To: Martin Sawicki ; Vladimir Shcherbakov
>>>>> 
>>>>> Cc: core-libs-dev Libs ; Naoto Sato
>>>>> 
>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>
>>>>> Hi Martin, Vladimir,
>>>>>
>>>>> It was suggested that this patch be tested on localized Windows
>>>>> machines and/or trying with the various Windows native encodings,
>>>>> appreciate if you can verify this as well.
>>>>>
>>>>> Thanks
>>>>> Kumar
>>>>>
>>>>> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Was on vacation, I started to prepare the patch from webrev.04 for
>>>>>> integration. Please note: made some adjustments to your patch to
>>>>>> pass jcheck, ie. usage of tabs and space at line endings, and
>>>>>> modifications to Copyright dates.
>>>>>>
>>>>>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE.
>>>>>> I have attached a patch to for your reference.
>>>>>>
>>>>>> However, there is a regression test failure on Windows,
>>>>>> jdk/test/tools/launcher/I18NTest.java
>>>>>>
>>>>>> ---Test info----
>>>>>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java
>>>>>>
>>>>>> ++++Test Output++++
>>>>>> javac: file not found: i18nH?lloWorld.java ----End test info-----
>>>>>>
>>>>>> Have you run all the launcher regression tests with this changeset ?
>>>>>>
>>>>>> Thanks
>>>>>> Kumar
>>>>>>
>>>>>>> Hi Kumar, just wondering if there are any updates on processing
>>>>>>> this submission.
>>>>>>> Thanks!
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Vladimir Shcherbakov
>>>>>>> Sent: Wednesday, November 25, 2015 2:38 PM
>>>>>>> To: Kumar Srinivasan ; Martin
>>>>>>> Sawicki 
>>>>>>> Cc: Kirk Shoop ; core-libs-dev Libs
>>>>>>> 
>>>>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>>
>>>>>>> Hi Kumar,
>>>>>>>
>>>>>>> Please find updated webreview here:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>>> op
>>>>>>> en
>>>>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cma
>>>>>>> rc
>>>>>>> in
>>>>>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86
>>>>>>> f1
>>>>>>> 41
>>>>>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjp
>>>>>>> Nf
>>>>>>> wn
>>>>>>> bIw%3d
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Vladimir.
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>>>>> Sent: Sunday, November 22, 2015 8:14 AM
>>>>>>> To: Martin Sawicki 
>>>>>>> Cc: Kirk Shoop ; Vladimir Shcherbakov
>>>>>>> ; core-libs-dev Libs
>>>>>>> 
>>>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>>
>>>>>>>
>>>>>>> Hi Martin, et. al.,
>>>>>>>
>>>>>>> Sorry for not getting back earlier, I am very busy right now with
>>>>>>> my other large commitments for JDK9.
>>>>>>>
>>>>>>> I will sponsor this "enhancement/bug fix" sometime in the new
>>>>>>> year, meanwhile, there is the changeset  [1] which is likely to
>>>>>>> cause merge conflicts, and perhaps logic issues.
>>>>>>>
>>>>>>> Thanks
>>>>>>> Kumar
>>>>>>>
>>>>>>> [1]
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.
>>>>>>> op
>>>>>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c
>>>>>>> 01
>>>>>>> %7
>>>>>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f
>>>>>>> 98
>>>>>>> 8b
>>>>>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNO
>>>>>>> Jk
>>>>>>> 7M
>>>>>>> g4lt2P0sdzLS%2fT9Q%3d
>>>>>>>> Hi all
>>>>>>>> Here's an updated webrev attempting to take into account the
>>>>>>>> various pieces of feedback we have received:
>>>>>>>>
>>>>>>>> Issue:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs.
>>>>>>>>
>>>>>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%4
>>>>>>>> 0m
>>>>>>>> ic
>>>>>>>> ro
>>>>>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91
>>>>>>>> ab
>>>>>>>> 2d
>>>>>>>> 7c
>>>>>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsU
>>>>>>>> Iq
>>>>>>>> 9E
>>>>>>>> %3
>>>>>>>> d
>>>>>>>> Webrev:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr
>>>>>>>> .o
>>>>>>>> pe
>>>>>>>> nj
>>>>>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvl
>>>>>>>> as
>>>>>>>> hc
>>>>>>>> h%
>>>>>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f
>>>>>>>> 14
>>>>>>>> 1a
>>>>>>>> f9
>>>>>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN66
>>>>>>>> 7B
>>>>>>>> Jn
>>>>>>>> WY
>>>>>>>> 0%3d
>>>>>>>>
>>>>>>>> (Vladimir Shcherbakov is now working on this from our side)
>>>>>>>>
>>>>>>>> Looking forward to any other feedback.
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: core-libs-dev
>>>>>>>> [mailto:core-libs-dev-bounces at openjdk.java.net]
>>>>>>>> On Behalf Of Kumar Srinivasan
>>>>>>>> Sent: Thursday, June 25, 2015 6:26 AM
>>>>>>>> To: Kirk Shoop (MS OPEN TECH) 
>>>>>>>> Cc: Valery Kopylov (Akvelon) ;
>>>>>>>> core-libs-dev Libs 
>>>>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>>>
>>>>>>>> Hi Kirk,
>>>>>>>>
>>>>>>>> Thanks for proposing this change.
>>>>>>>>
>>>>>>>> If you notice all the posix calls are wrapped in JLI_* this
>>>>>>>> gives us the ability to use "W" functions.  I almost got it
>>>>>>>> done, several years ago, but we upgraded to VS2010 and my work
>>>>>>>> based on
>>>>>>>> VS2003 keeled over, meanwhile my focus was  "shifted" to
>>>>>>>> something else.
>>>>>>>>
>>>>>>>> main.c: is really envisioned to be a stub  compiled by the tool
>>>>>>>> launchers, like java, javac, javah, jar etc. I prefer to see all
>>>>>>>> the heavy logic in this file moved to the platform specific file
>>>>>>>> windows/java_md.*
>>>>>>>>
>>>>>>>> For the reason specified above we need to move fprintf or any
>>>>>>>> naked posix calls to JLI_* indirections.
>>>>>>>>
>>>>>>>> I don't see any tests ? The tests must be written in java and
>>>>>>>> placed in jdk/test/tools/launcher, there is a helper framework
>>>>>>>> TestHelper.java.
>>>>>>>>
>>>>>>>> There are other changes in nio, charsets etc, this will be
>>>>>>>> reviewed by my colleague specializing in that area (Sherman) cc'ed.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> Kumar
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Issue:
>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%
>>>>>>>>> 2f
>>>>>>>>> bu
>>>>>>>>> gs
>>>>>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch
>>>>>>>>> %4
>>>>>>>>> 0m
>>>>>>>>> ic
>>>>>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141a
>>>>>>>>> f9
>>>>>>>>> 1a
>>>>>>>>> b2
>>>>>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%
>>>>>>>>> 2f
>>>>>>>>> sU
>>>>>>>>> Iq
>>>>>>>>> 9E%3d
>>>>>>>>>
>>>>>>>>> Webrev:
>>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>>>>> op
>>>>>>>>> en
>>>>>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40mi
>>>>>>>>> cr
>>>>>>>>> os
>>>>>>>>> of
>>>>>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab
>>>>>>>>> 2d
>>>>>>>>> 7c
>>>>>>>>> d0
>>>>>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11
>>>>>>>>> B0
>>>>>>>>> M%
>>>>>>>>> 3d
>>>>>>>>>
>>>>>>>>> This webrev intends to address interaction between Windows
>>>>>>>>> console and java apps.
>>>>>>>>>
>>>>>>>>> Two switches were added that change the behavior of the launcher.
>>>>>>>>> The defaults do not change the launcher behavior.
>>>>>>>>>
>>>>>>>>>         -Dwindows.UnicodeConsole=true - switches on Unicode
>>>>>>>>> support in the Windows console. This optional switch causes the
>>>>>>>>> launcher to call GetCommandLineW() and parse the arguments in
>>>>>>>>> unicode. It also modifies how the codepage for console output is selected.
>>>>>>>>>
>>>>>>>>>         -Dfile.encoding.unicode="UTF-8" - identifies Unicode
>>>>>>>>> charset to use; If not specified, UTF-8 is used by default.
>>>>>>>>> Ignored when windows.UnicodeConsole is not set to true. When
>>>>>>>>> the first switch is used, this optional switch allows the
>>>>>>>>> codepage for console output to be controlled.
>>>>>>>>>
>>>>>>>>> I would like to get feedback on the approach here and any
>>>>>>>>> additional work that is required solve these particular Unicode
>>>>>>>>> issues on Windows.
>>>>>>>>>
>>>>>>>>> Kirk




From ramanand.patil at oracle.com  Wed Feb 24 07:40:48 2016
From: ramanand.patil at oracle.com (Ramanand Patil)
Date: Tue, 23 Feb 2016 23:40:48 -0800 (PST)
Subject: RFR: JDK-8087104: DateFormatSymbols triggers this.clone() in the
	constructor
Message-ID: 

Hi all,
Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8087104
Bug Description: DateFormatSymbols caches its own instance and calls this.clone() in the constructor. Because of this, any subclass implementation (which expects a field is always initialized to non-null in the constructor) will throw NPE in its overridden clone() method while using any instance variables which it assumed are initilaized in its contructor.
Webrev: http://cr.openjdk.java.net/~rpatil/8087104/webrev.00/
Fix: Instead of using its own instance for caching and calling clone in DateFormatSymbols, a nested class SymbolsCacheEntry is introduced.
 

Regards,

Ramanand.


From masayoshi.okutsu at oracle.com  Wed Feb 24 08:16:25 2016
From: masayoshi.okutsu at oracle.com (Masayoshi Okutsu)
Date: Wed, 24 Feb 2016 17:16:25 +0900
Subject: RFR: JDK-8087104: DateFormatSymbols triggers this.clone() in the
	constructor
In-Reply-To: 
References: 
Message-ID: <56CD66D9.9070605@oracle.com>

Looks good to me.

Masayoshi

On 2/24/2016 4:40 PM, Ramanand Patil wrote:
> Hi all,
> Please review the fix for bug: https://bugs.openjdk.java.net/browse/JDK-8087104
> Bug Description: DateFormatSymbols caches its own instance and calls this.clone() in the constructor. Because of this, any subclass implementation (which expects a field is always initialized to non-null in the constructor) will throw NPE in its overridden clone() method while using any instance variables which it assumed are initilaized in its contructor.
> Webrev: http://cr.openjdk.java.net/~rpatil/8087104/webrev.00/
> Fix: Instead of using its own instance for caching and calling clone in DateFormatSymbols, a nested class SymbolsCacheEntry is introduced.
>   
>
> Regards,
>
> Ramanand.



From vyom.tewari at oracle.com  Wed Feb 24 09:16:17 2016
From: vyom.tewari at oracle.com (vyom)
Date: Wed, 24 Feb 2016 14:46:17 +0530
Subject: RFR 8145635 : Add TCP_QUICKACK socket option
Message-ID: <56CD74E1.60309@oracle.com>

Hi All,

Please review my code changes for the below issue.

Bug: JDK-8145635 : Add TCP_QUICKACK socket option

Webrev: http://cr.openjdk.java.net/~vtewari/8145635/webrev0.0/index.html 


Currently TCP_QUICKACK is only supported on Linux( since Linux 2.4.4) so 
i implemented is as "ExtendedSocketOption".

Thanks,
Vyom



From Alan.Bateman at oracle.com  Wed Feb 24 09:46:50 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Wed, 24 Feb 2016 09:46:50 +0000
Subject: RFR 8145635 : Add TCP_QUICKACK socket option
In-Reply-To: <56CD74E1.60309@oracle.com>
References: <56CD74E1.60309@oracle.com>
Message-ID: <56CD7C0A.2080402@oracle.com>


On 24/02/2016 09:16, vyom wrote:
> Hi All,
>
> Please review my code changes for the below issue.
>
> Bug: JDK-8145635 : Add TCP_QUICKACK socket option
>
> Webrev: http://cr.openjdk.java.net/~vtewari/8145635/webrev0.0/index.html
>
> Currently TCP_QUICKACK is only supported on Linux( since Linux 2.4.4) 
> so i implemented is as "ExtendedSocketOption".

Is there any urgency on this one? I'm just wondering if we should try to 
the jdk.net API moved out of java.base before we add more socket 
options. This is currently tracked as JDK-8044773 and important to get 
into JDK 9.

-Alan


From aleksej.efimov at oracle.com  Wed Feb 24 12:36:24 2016
From: aleksej.efimov at oracle.com (Aleksej Efimov)
Date: Wed, 24 Feb 2016 15:36:24 +0300
Subject: RFR 8150470: JCK: api/xsl/conf/copy/copy19 test failure
In-Reply-To: <7d93f07319f54ba084fdd94adfac5e88@derote13de11.global.corp.sap>
References: <7d93f07319f54ba084fdd94adfac5e88@derote13de11.global.corp.sap>
Message-ID: <56CDA3C8.3040507@oracle.com>

Hi Christoph,
The changes looks good to me too. I will push them shortly today.

With Best Regards,
Aleksej

On 02/24/2016 02:49 AM, Langer, Christoph wrote:
>
> Hi,
>
> the changeset for 8149915 (enabling validate-annotations feature for 
> xsd schema with annotation causes NPE) introduced a regression. Here 
> is the fix.
>
> Bugid: https://bugs.openjdk.java.net/browse/JDK-8150470 
> 
>
> Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8150470.1/ 
> 
>
> Please review and push this one liner.
>
> Thanks & Best regards
>
> Christoph
>



From dcherepanov at azul.com  Wed Feb 24 14:07:28 2016
From: dcherepanov at azul.com (Dmitry Cherepanov)
Date: Wed, 24 Feb 2016 14:07:28 +0000
Subject: RFR [7] 8133206: "java.lang.OutOfMemoryError: unable to create
	new native thread" caused by upgrade to zlib 1.2.8
In-Reply-To: <56C6F49C.5090604@azulsystems.com>
References: <561E7C96.9010207@azulsystems.com>
	<5627B1B8.6080307@azulsystems.com> <5641CD9D.8080208@alexkasko.com>
	<566AB4AF.60902@oracle.com> <566ABCEC.90502@azulsystems.com>
	<56C6F49C.5090604@azulsystems.com>
Message-ID: 


On Feb 19, 2016, at 1:55 PM, Nikolay Gorshkov > wrote:

Hi Sherman, Sean,

Could you please help with making progress on this code review request?
This fix is waiting for review since October.

Webrev for jdk7u:
http://cr.openjdk.java.net/~nikgor/8133206/jdk7u-dev/webrev.01/
Original mail thread:
http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-October/035884.html

I?m not expert in this area but the changes look reasonable to me. +1 for pushing this into JDK9.

Thanks

Dmitry


Webrev for jdk9 (contributed by Alex Kashchenko):
http://cr.openjdk.java.net/~akasko/jdk9/8133206/webrev.00/
Original mail thread:
http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-November/003036.html

Thanks,
Nikolay

On 11.12.2015 15:09, Nikolay Gorshkov wrote:
Hi Sean,

Thank you for your attention to this matter!
Actually, the code review request was sent to core-libs-dev alias a month ago:
http://mail.openjdk.java.net/pipermail/core-libs-dev/2015-November/036463.html
Unfortunately, we haven't got any feedback yet.

Thanks,
Nikolay

On 11.12.2015 14:34, Se?n Coffey wrote:
Hi Alex,

I'm dropping jdk7u-dev mailing list for the moment. core-libs-dev is the
mailing list where this review should happen. This fix would be required in JDK
9 first as per process. I think Sherman would be best to review if possible.

Once it's soaked in JDK 9 for a few weeks, we could consider jdk8u and 7u
backports.

Regards,
Sean.

On 10/11/15 10:57, Alex Kashchenko wrote:
Hi,

On 10/21/2015 04:39 PM, Nikolay Gorshkov wrote:
Hi Sherman,

Thank you for your reply! My answers are inlined.

> Can you be more specific about the "class loading cases" above? Sounds
> more like we have a memory leaking here (the real root cause)? for
example
> the inflateEnd() never gets called?

I agree, the real root cause is probably the following issue that exists
since the end of 2002:
https://bugs.openjdk.java.net/browse/JDK-4797189
"Finalizers not called promptly enough"
And it is "the absence of a general solution to the non-heap resource
exhaustion problem".

zlib's inflateEnd() function is called by
void java.util.zip.Inflater.end(long addr)
native method only, and this method, in turn, is called only by
void java.util.zip.Inflater.end()
and
void java.util.zip.Inflater.finalize()
methods. According to the experiments, the typical stack trace for
instantiating java.util.zip.Inflater is:

java.util.zip.Inflater.(Inflater.java:116)
java.util.zip.ZipFile.getInflater(ZipFile.java:450)
java.util.zip.ZipFile.getInputStream(ZipFile.java:369)
java.util.jar.JarFile.getInputStream(JarFile.java:412)
org.jboss.virtual.plugins.context.zip.ZipFileWrapper.openStream(ZipFileWrapper.java:222)



  
org.jboss.classloader.spi.base.BaseClassLoader$2.run(BaseClassLoader.java:592)

java.security.AccessController.doPrivileged(Native Method)
org.jboss.classloader.spi.base.BaseClassLoader.loadClassLocally(BaseClassLoader.java:591)



  
org.jboss.classloader.spi.base.BaseClassLoader.loadClass(BaseClassLoader.java:447)



java.lang.ClassLoader.loadClass(ClassLoader.java:358)
java.lang.Class.forName0(Native Method)
java.lang.Class.forName(Class.java:278)
org.jboss.deployers.plugins.annotations.WeakClassLoaderHolder.loadClass(WeakClassLoaderHolder.java:72)



  

It's quite hard to understand who is responsible for not calling
Inflater.end()
method explicitly; probably, it is the jboss/application's code.
Unfortunately,
we were in "it worked before and is broken now" customer situation here, so
needed to fix it anyway.

>  From the doc/impl in inflate() it appears the proposed change should be
> fine, though it's a little hacky, as you never know if it starts to
return
> Z_OK from some future release(s). Since the "current" implementation
> never returns Z_OK, it might be worth considering to keep the Z_OK logic
> asis in Inflater.c, together with the Z_BUF_ERROR, just in case?

OK, I added handling of Z_OK code back.

> I would be desired to add some words in Inflater.c to remind the
> future maintainer why we switched from partial to finish and why to
> check z_buf_error.

I agree, added a comment.

The updated webrev is available here:

http://cr.openjdk.java.net/~nikgor/8133206/jdk7u-dev/webrev.01/


The change looks good to me (not a Reviewer/Committer).

Patched jdk7u also passes JCK-7 on RHEL 7.1.

I forward-ported this patch to jdk9 (consulted with Nikolay Gorshkov first),
jtreg reproducer for jdk9 also works with jdk7u -
http://mail.openjdk.java.net/pipermail/jdk9-dev/2015-November/003036.html




From vitalyd at gmail.com  Wed Feb 24 15:20:35 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Wed, 24 Feb 2016 10:20:35 -0500
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
	
	<7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
Message-ID: 

On Wednesday, February 24, 2016, Gil Tene  wrote:

> onXXX is VERY relevant to this situation. And it's exactly that precedent
> and common convention for event notification calls that drove the choice of
> onSpinWait as a method name. onSpinWait() it is delivering a notification
> [from the calling code to the Runtime, or the Thread] about what is going
> on: a spin wait is in progress.

onXXX is typically used for event *handler* names, not event firing.  But
at any rate, this isn't an eventing system.

>
> The fundamental difference between onSpinWait() and Thread.yield() is that
> yield() means "please yield here", while onSpinWait means "I am in the
> middle of busily executing a spin wait that I need no help with, and
> certainly don't want to be blocking in. But you might want to know about
> that and do something with that knowledge. Preferably something that
> improves my performance." Basically, onSpinWait() is delivering a
> notification of a situation, and the runtime has a chance to react to that.

You can just as well think of spinWait as"I'm spin waiting *right here,
right now*, please possibly do something or nothing", which is just like
yield.

Here's the bottom line as I see it.  This API is for advanced/expert users,
there's no need for additional fluff.These users will either already know
what this API translates to on platforms they care about or will seek that
info.  Users unfamiliar with the JDK will see spinWait or onSpinWait -
neither name alone is sufficient to figure out what it is, they'll look at
javadoc which is the right place for elaboration.  So the 'on' prefix adds
no additional *practical* value for the intended audience, it's just
noise.  Runtime has just gc(), not onGC, even though it also can do
nothing, run GC in foreground, run GC in background, or anything else.
Again, 'on' would add zero practical value.

If you're really keen on having something in the name indicate its
optionality/hint nature, spinWaitHint is better.

>
> Dropping the "on" would completely change the meaning. Something called
> "spinWait" *is* like yield(). It would/should mean "please spin wait here".
> But that's not what spin wait constructs need or want in a call. Unlike
> yield(), spin wait constructs use their own logic for spinning, and usually
> need to retain control over the logic. They just need to let someone know
> that spinning is going on, in the hope that performance of the spinwait
> construct (with whatever logic it uses) might be improved.
>
> Note that the runtime reactions to being notified the calling code is in
> the middle of a spin wait could range from doing nothing, to doing
> something with instruction execution (e.g. execute an x86 pause
> instruction), to doing something more sophisticated (e.g. trying to
> dedicate a core to the spinning thread and steer all other thread and
> interrupts away from it to improve it's reaction time stats). What will
> actually happen is up to the runtime, and intentionally not specified. Per
> the JavaDoc, if the runtime does take action when it receives the
> onSpinWait() event, the action is intended "to improve the performance of
> invoking spin-wait loop constructions".
>
> You can hopefully/probably see why Runtime was a logical choice as the
> receiver of the event. Since Thread is also acceptable and defensible as
> the receiver of the event, I'm happy to go with it if it gets us over this
> name-can-of-worms thing. But please please please don't let us go into a
> whole method name redo again. We've done that on and off for 3+ months, and
> what we have is both logical and "not unacceptable": onSpinWait().
>
> Let's just go with it.

If we're going to discuss naming at all, let's discuss it in earnest rather
than just glossing over it in the name of progress? Yes, if it's between
not having this API at all vs having it named suboptimally (IMO) then I
frankly almost don't care what it's called, as I mentioned before.  But if
we *are* subjecting naming to critique, then let's iron that out.

If you want, run a poll to get a better feel for what others may think.
Doug did that recently for some CompletableFuture behavior change
proposals.  Ultimately, I don't make the call on this anyway so I'm just
giving you my opinion as a would-be user of this API.

>
> Thread.onSpinWait() FTW!
>
> > On Feb 23, 2016, at 8:03 PM, Vitaly Davidovich  > wrote:
> >
> > Yes, I was going to mention onXXX being common for event handler names
> (e.g
> > onMouseClick), but didn't bother for the same reason you mentioned - it's
> > irrelevant to this situation.
> >
> > On Tuesday, February 23, 2016, Paul Benedict  > wrote:
> >
> >> The onXXX prefix does have precedent as event handler callbacks, but
> this
> >> method does not fit that purpose. Thus, I agree dropping "on" is
> sensible.
> >> On Feb 23, 2016 8:48 PM, "Vitaly Davidovich"  
> >> ');>>
> wrote:
> >>
> >>> On Tuesday, February 23, 2016, Doug Lea 
       
> >>> ');>>
> wrote:
> >>>
> >>>> On 02/23/2016 04:30 PM, Vitaly Davidovich wrote:
> >>>>
> >>>>> Why not drop the (superfluous, IMO) "on" prefix while you're changing
> >>> the
> >>>>> receiver?
> >>>>>
> >>>>
> >>>> Because then it reads as if the method itself is doing a spinWait.
> >>>
> >>> vs who else logically speaking? We all know there's a runtime
> underneath
> >>> Java, there's no point in explicitly calling that out here.  Again,
> how is
> >>> this different from Thread::yield or any of the other mentioned
> examples?
> >>> This is splitting hairs perhaps but there's no onXXX precedent to
> follow
> >>> and this just throws an oddly looking method name into the mix.
> >>>
> >>>> "onSpinWait" is the only proposed name that no one has said they
> cannot
> >>>> live with. So, live with it :-)
> >>>
> >>> Perhaps that's because the Runtime placement was a more glaring issue?
> :)
> >>> It's livable but I just don't see the point of the prefix (and yes, I
> read
> >>> the description of the intent in the original mail).
> >>>
> >>>>
> >>>> -Doug
> >>>>
> >>>>
> >>>>
> >>>>> On Tue, Feb 23, 2016 at 4:20 PM, Gil Tene  
> >>> ');>> wrote:
> >>>>>
> >>>>>
> >>>>>> On Feb 22, 2016, at 10:11 AM, mark.reinhold at oracle.com
> 
> >>> ');>
> wrote:
> >>>>>>>
> >>>>>>> 2016/1/28 9:25 -0800, gil at azul.com 
> >>> ');>:
> >>>>>>>
> >>>>>>>> This thread seems to have "hopped away" to the
> concurrency-interest
> >>>>>>>> list in mid-Dec-2015. This posting is intended to capture a
> summary
> >>> of
> >>>>>>>> reasoning and some of the discussion there so that we have it in
> the
> >>>>>>>> record in core-libs-dev. Mostly by including the contents of
> several
> >>>>>>>> posts in the continuations of the original thread.
> >>>>>>>>
> >>>>>>>> See thread continuations here:
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>
> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14576
> >>>>>>
> >>>>>>> and here:
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>
> http://cs.oswego.edu/pipermail/concurrency-interest/2015-December/thread.html#14580
> >>>>>>
> >>>>>>>
> >>>>>>>> Summary:
> >>>>>>>>
> >>>>>>>> ...
> >>>>>>>>
> >>>>>>>
> >>>>>>> Thanks for the summary.
> >>>>>>>
> >>>>>>> I still don't buy the argument that this method belongs in
> >>> j.l.Runtime.
> >>>>>>>
> >>>>>>> To say that this method should go there because it's an instruction
> >>> to
> >>>>>>> the run-time system is pretty weak.  I agree with Vitaly [1] that
> if
> >>>>>>> that's the threshold for adding methods to the Runtime class then
> >>> lots
> >>>>>>> of other stuff belongs there as well, including much of what's now
> in
> >>>>>>> java.lang.Thread and java.util.concurrent and, arguably, anything
> >>> else
> >>>>>>> related to interacting with the environment in which the
> application
> >>>>>>> runs (file and network I/O, process manipulation, etc.).
> >>>>>>>
> >>>>>>> This thread-related method really belongs in either
> java.lang.Thread
> >>> or
> >>>>>>> java.util.concurrent.LockSupport.  j.l.Thread already has plenty of
> >>>>>>> expert-level static methods related to the current thread, one of
> >>> which
> >>>>>>> (Thread::yield) is even a hint, just like this one.
> >>> j.u.c.LockSupport
> >>>>>>> is even more obviously intended for expert users and hence may be
> the
> >>>>>>> best choice, but I could live with either one.
> >>>>>>>
> >>>>>>
> >>>>>> Ok. In the interest of moving forward, lets settle on:
> >>>>>>
> >>>>>> Thread.onSpinWait()
> >>>>>>
> >>>>>> Same logic for the name, different receiver for the event. I can
> >>>>>> certainly
> >>>>>> live with it, and Doug seems ok with it as well.
> >>>>>>
> >>>>>> ? Gil.
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>> --
> >>> Sent from my phone
> >>>
> >>
> >
> > --
> > Sent from my phone
>
>

-- 
Sent from my phone


From mark.reinhold at oracle.com  Wed Feb 24 15:56:26 2016
From: mark.reinhold at oracle.com (mark.reinhold at oracle.com)
Date: Wed, 24 Feb 2016 07:56:26 -0800
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<561428BC.3060806@cs.oswego.edu>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
	
	<7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
Message-ID: <20160224075626.185057628eggemoggin.niobe.net>

2016/2/23 22:22 -0800, Gil Tene :
> ...
> 
> Let's just go with it.
> 
> Thread.onSpinWait() FTW!

I agree.  The "on" prefix is not exclusively reserved for APIs that
deliver event objects, and in this context it clarifies the meaning
of the method.

- Mark


From ivan at azulsystems.com  Wed Feb 24 16:06:37 2016
From: ivan at azulsystems.com (Ivan Krylov)
Date: Wed, 24 Feb 2016 19:06:37 +0300
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: 
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
	
	<7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
	
Message-ID: <56CDD50D.2070107@azulsystems.com>

On 24/02/2016 18:20, Vitaly Davidovich wrote:
> If you want, run a poll to get a better feel for what others may 
> think. Doug did that recently for some CompletableFuture behavior 
> change proposals. Ultimately, I don't make the call on this anyway so 
> I'm just giving you my opinion as a would-be user of this API. 
This discussion has been going on for nearly half of a year, everyone is 
saying "I don't care for the name, but..".
Another poll will not do any good. I would rather have a Project Lead to 
make a final decision on the API, a harmless one either way IMO.

Ivan



From vitalyd at gmail.com  Wed Feb 24 16:12:19 2016
From: vitalyd at gmail.com (Vitaly Davidovich)
Date: Wed, 24 Feb 2016 11:12:19 -0500
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <56CDD50D.2070107@azulsystems.com>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
	
	<7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
	
	<56CDD50D.2070107@azulsystems.com>
Message-ID: 

Looks like you have Mark's approval, so we're done :).

On Wed, Feb 24, 2016 at 11:06 AM, Ivan Krylov  wrote:

> On 24/02/2016 18:20, Vitaly Davidovich wrote:
>
>> If you want, run a poll to get a better feel for what others may think.
>> Doug did that recently for some CompletableFuture behavior change
>> proposals. Ultimately, I don't make the call on this anyway so I'm just
>> giving you my opinion as a would-be user of this API.
>>
> This discussion has been going on for nearly half of a year, everyone is
> saying "I don't care for the name, but..".
> Another poll will not do any good. I would rather have a Project Lead to
> make a final decision on the API, a harmless one either way IMO.
>
> Ivan
>
>


From ivan at azulsystems.com  Wed Feb 24 16:20:58 2016
From: ivan at azulsystems.com (Ivan Krylov)
Date: Wed, 24 Feb 2016 19:20:58 +0300
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <20160224075626.185057628eggemoggin.niobe.net>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
	
	<7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
	<20160224075626.185057628eggemoggin.niobe.net>
Message-ID: <56CDD86A.1050107@azulsystems.com>

On 24/02/2016 18:56, mark.reinhold at oracle.com wrote:
> 2016/2/23 22:22 -0800, Gil Tene :
>> ...
>>
>> Let's just go with it.
>>
>> Thread.onSpinWait() FTW!
> I agree.  The "on" prefix is not exclusively reserved for APIs that
> deliver event objects, and in this context it clarifies the meaning
> of the method.
>
> - Mark
I have updated the JEP: https://bugs.openjdk.java.net/browse/JDK-8147832


From Roger.Riggs at Oracle.com  Wed Feb 24 16:27:38 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 24 Feb 2016 11:27:38 -0500
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <416730A7-88EE-4855-81E3-99B7E603793F@oracle.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com> <56CC8A4A.9080303@gmail.com>
	
	<416730A7-88EE-4855-81E3-99B7E603793F@oracle.com>
Message-ID: <56CDD9FA.2030103@Oracle.com>

Hi,

Moving cleanNextPending to the CleanerImpl is ok but exposes more of the 
CleanerImpl
to the rest of java.base internals; loosing a bit of encapsulation.

As for the calls to Reference.tryHandlePending(false), the existing code 
in Bits to reserveMemory
is called on arbitrary threads (whoever was creating a Direct-X-Buffer).

The additional call in the cleanNextPending is in the same thread 
context and helps
cleaning and reference processing occur sooner.

Roger


On 2/23/2016 9:28 PM, Mandy Chung wrote:
>> On Feb 23, 2016, at 3:22 PM, Kim Barrett  wrote:
>>
>> I'm not sure what to do about any of this; I'm just feeling like maybe
>> the proposed API / implementation isn't quite right yet.
> I suggest restore the JavaLangRefAccess used by direct byte buffer code and revisit such explicit mechanism until there is a need.
>
> Another concern is that this method gets to run in any arbitrary thread rather than the dedicated reference handler thread / cleaner thread.
>
> Mandy



From adinn at redhat.com  Wed Feb 24 16:56:23 2016
From: adinn at redhat.com (Andrew Dinn)
Date: Wed, 24 Feb 2016 16:56:23 +0000
Subject: [concurrency-interest] Spin Loop Hint support: Draft JEP proposal
In-Reply-To: <20160224075626.185057628eggemoggin.niobe.net>
References: <5932C61D-C9D8-493D-9E2F-2C7965398C41@azulsystems.com>
	<015C733F-130A-47C0-8F68-1CBB11CC0C38@azulsystems.com>
	<56164C3C.4050800@cs.oswego.edu> <561E6F04.9030708@cs.oswego.edu>
	<54AE479E-9891-48FB-AD47-ECED8147DABE@azul.com>
	<561FC6F9.1080701@cs.oswego.edu>
	<25B8ED3D-4B35-48F4-BF24-32D48E85343E@azul.com>
	<562BC968.70603@cs.oswego.edu>
	<8917A25C-4A16-4291-B145-FA7180E09F78@azul.com>
	<4A8EF318-A674-4619-94A6-6321AA198AEF@azul.com>
	<20151207093440.607909@eggemoggin.niobe.net>
	
	<45A813DE-6059-475D-8D48-AFE4A3F36733@azul.com>
	
	<56CCF815.7010105@cs.oswego.edu>
	
	
	
	<7E021577-92C4-4676-BA1A-FFCCCAAE5222@azul.com>
	<20160224075626.185057628eggemoggin.niobe.net>
Message-ID: <56CDE0B7.6090308@redhat.com>

On 24/02/16 15:56, mark.reinhold at oracle.com wrote:
> 2016/2/23 22:22 -0800, Gil Tene :
>> ...
>>
>> Let's just go with it.
>>
>> Thread.onSpinWait() FTW!
> 
> I agree.  The "on" prefix is not exclusively reserved for APIs that
> deliver event objects, and in this context it clarifies the meaning
> of the method.

Hallelujah, now we can start painting the bike shed!

regards,


Andrew Dinn
-----------
Senior Principal Software Engineer
Red Hat UK Ltd
Registered in UK and Wales under Company Registration No. 3798903
Directors: Michael Cunningham (US), Michael O'Neill (Ireland), Paul
Argiry (US)


From thomas.stuefe at gmail.com  Wed Feb 24 17:30:07 2016
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Wed, 24 Feb 2016 18:30:07 +0100
Subject: RFR(s): 8150460: (linux|bsd|aix)_close.c: file descriptor table may
	become large or may not work at all
Message-ID: 

Hi all,

please take a look at this proposed fix.

The bug: https://bugs.openjdk.java.net/browse/JDK-8150460
The Webrev:
http://cr.openjdk.java.net/~stuefe/webrevs/8150460-linux_close-fdTable/webrev.00/webrev/

Basically, the file descriptor table implemented in linux_close.c may not
work for RLIMIT_NO_FILE=infinite or may grow very large (I saw a 50MB
table) for high values for RLIMIT_NO_FILE. Please see details in the bug
description.

The proposed solution is to implement the file descriptor table not as
plain array, but as a twodimensional sparse array, which grows on demand.
This keeps the memory footprint small and fixes the corner cases described
in the bug description.

Please note that the implemented solution is kept simple, at the cost of
somewhat higher (some kb) memory footprint for low values of RLIMIT_NO_FILE.
This can be optimized, if we even think it is worth the trouble.

Please also note that the proposed implementation now uses a mutex lock for
every call to getFdEntry() - I do not think this matters, as this is all in
preparation for an IO system call, which are usually way more expensive
than a pthread mutex. But again, this could be optimized.

This is an implementation proposal for Linux; the same code found its way
to BSD and AIX. Should you approve of this fix, I will modify those files
too.

Thank you and Kind Regards, Thomas


From Roger.Riggs at Oracle.com  Wed Feb 24 19:28:01 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Wed, 24 Feb 2016 14:28:01 -0500
Subject: RFR(s): 8150460: (linux|bsd|aix)_close.c: file descriptor table
	may become large or may not work at all
In-Reply-To: 
References: 
Message-ID: <56CE0441.6060308@Oracle.com>

Hi Thomas,

On 2/24/2016 12:30 PM, Thomas St?fe wrote:
> Hi all,
>
> please take a look at this proposed fix.
>
> The bug: https://bugs.openjdk.java.net/browse/JDK-8150460
> The Webrev:
> http://cr.openjdk.java.net/~stuefe/webrevs/8150460-linux_close-fdTable/webrev.00/webrev/
>
> Basically, the file descriptor table implemented in linux_close.c may not
> work for RLIMIT_NO_FILE=infinite or may grow very large (I saw a 50MB
> table) for high values for RLIMIT_NO_FILE. Please see details in the bug
> description.
>
> The proposed solution is to implement the file descriptor table not as
> plain array, but as a twodimensional sparse array, which grows on demand.
> This keeps the memory footprint small and fixes the corner cases described
> in the bug description.
>
> Please note that the implemented solution is kept simple, at the cost of
> somewhat higher (some kb) memory footprint for low values of RLIMIT_NO_FILE.
> This can be optimized, if we even think it is worth the trouble.
>
> Please also note that the proposed implementation now uses a mutex lock for
> every call to getFdEntry() - I do not think this matters, as this is all in
> preparation for an IO system call, which are usually way more expensive
> than a pthread mutex. But again, this could be optimized.
I would suggest preallocating the index[0] array and then skip the mutex 
for that case.
That would give the same as current performance.

And I'd suggest a different hi/low split of the indexes to reduce the 
size of pre-allocated memory.
Most processes are going to use a lot fewer than 16384 fd's.  How about 
2048?
I have my doubts about needing to cover fd's  up to the full range of 32 
bits.
Can the RLIMIT_NO_FILE be used too parametrize the allocation of the 
first level table?

Not specific to your change but it would nice to see consistency between 
libnio and libnet on
the name of the sigWakeup/INTERRUPT_SIGNAL constant.
>
> This is an implementation proposal for Linux; the same code found its way
> to BSD and AIX. Should you approve of this fix, I will modify those files
> too.
yes please.

$.02, Roger

>
> Thank you and Kind Regards, Thomas



From aleksej.efimov at oracle.com  Wed Feb 24 22:50:23 2016
From: aleksej.efimov at oracle.com (Aleksej Efimov)
Date: Thu, 25 Feb 2016 01:50:23 +0300
Subject: [9] RFR: 8150174: Update JAX-WS RI integration to latest version
	(2.3.0-SNAPSHOT)
Message-ID: <56CE33AF.3060504@oracle.com>

Hi,

Please, review the bulk update of JAX-WS/B from upstream projects:
http://cr.openjdk.java.net/~aefimov/jaxws-integrations/8150174/9/00/

Details (list of fixed issues) can be found in bug report:
https://bugs.openjdk.java.net/browse/JDK-8150174

The following test sets were executed over JDK9 with integrated changes:
jdk_other JTREG tests; JCK9 jaxws tests; JAXWS unit tests;

Thanks,
Aleksej


From amy.lu at oracle.com  Thu Feb 25 01:49:52 2016
From: amy.lu at oracle.com (Amy Lu)
Date: Thu, 25 Feb 2016 09:49:52 +0800
Subject: JDK 9 RFR of JDK-8150608: Mark BashStreams.java as intermittently
	failing and put to ProblemList
Message-ID: <56CE5DC0.2020906@oracle.com>

java/nio/charset/coders/BashStreams.java

This test is known to fail intermittently, this patch is to mark the 
test accordingly with keyword 'intermittent? and put it to 
ProblemList.txt for further investigation.

bug: https://bugs.openjdk.java.net/browse/JDK-8150608
webrev: http://cr.openjdk.java.net/~amlu/8150608/webrev.00/

Thanks,
Amy


From brian.burkhalter at oracle.com  Thu Feb 25 01:57:42 2016
From: brian.burkhalter at oracle.com (Brian Burkhalter)
Date: Wed, 24 Feb 2016 17:57:42 -0800
Subject: JDK 9 RFR of JDK-8150608: Mark BashStreams.java as intermittently
	failing and put to ProblemList
In-Reply-To: <56CE5DC0.2020906@oracle.com>
References: <56CE5DC0.2020906@oracle.com>
Message-ID: <0809E343-3FD6-4AAC-A1DB-A552F57A6AD8@oracle.com>

Hi Amy,

+1 for me but I am not a JDK 9 Reviewer.

Thanks,

Brian

On Feb 24, 2016, at 5:49 PM, Amy Lu  wrote:

> java/nio/charset/coders/BashStreams.java
> 
> This test is known to fail intermittently, this patch is to mark the test accordingly with keyword 'intermittent? and put it to ProblemList.txt for further investigation.
> 
> bug: https://bugs.openjdk.java.net/browse/JDK-8150608
> webrev: http://cr.openjdk.java.net/~amlu/8150608/webrev.00/
> 
> Thanks,
> Amy



From iris.clark at oracle.com  Thu Feb 25 05:59:20 2016
From: iris.clark at oracle.com (Iris Clark)
Date: Wed, 24 Feb 2016 21:59:20 -0800 (PST)
Subject: RFR: 8144062: Determine appropriate module for jdk.Version
In-Reply-To: 
References: <50765a19-34e4-4115-a32d-cea84c25394e@default>
	<38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com>
	<565628C5.1040905@oracle.com>
	<92b9301d-2418-4618-b877-b13c88216520@default>
	<7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default>
	<2f3ec2a8-4488-4c21-ae79-01c4599d5121@default>
	<56B1BFE3.1010201@oracle.com>
	
Message-ID: <86a4690e-7a83-4bc4-9c3a-9d617475f3bb@default>

Hi, Alan, Mandy, and Mark.

After exploring a few module options (jdk.Version, jdk.dev [0]), 
it looks like the best choice is to move jdk.Version to 
java.lang.Runtime.Version (a nested class of Runtime).  It supports 
the values returned by the java.{vm.}?version and 
java.{vm.}?specification.version system properties.  By making 
Version an SE API, it may be exported by the java.base module.

Please review the following changes to move jdk.Version to 
jdk.lang.Runtime.Version.  Note that beyond the package name and
class declaration (to static), the only other changes in Version
are related to indentation.

Bug

    8144062: Determine appropriate module for jdk.Version
    https://bugs.openjdk.java.net/browse/JDK-8144062

webrev

    http://cr.openjdk.java.net/~iris/verona/8144062/webrev.0/

Additionally, the following changes to JEP 223 [1] (modulo indentation)
will be made:

252,253c252,253
< A simple JDK-specific Java API to parse, validate, and compare version
< strings will be defined:
---
> A simple Java API to parse, validate, and compare version strings will be
> defined as a nested class of Runtime:
255c255
<     package jdk;
---
>     package java.lang;
258a259
>     public class Runtime {
283a285
>     }
309a312,315
>
> The existing specification for the above properties in
> `System.getProperties()` will be modified to indicate that the return values
> for these properties may be interpreted as a `java.lang.Runtime.Version`.

Thanks,
Iris

[0] jdk.dev renamed to jdk.jlink in jigsaw/jake.
[1] http://openjdk.java.net/jeps/223


From david.holmes at oracle.com  Thu Feb 25 06:58:20 2016
From: david.holmes at oracle.com (David Holmes)
Date: Thu, 25 Feb 2016 16:58:20 +1000
Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in
	executables with 32k tbss/tdata
In-Reply-To: <56C48EF2.3030403@oracle.com>
References: <56962632.7080404@oracle.com>
	
	<569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com>
	<56C48EF2.3030403@oracle.com>
Message-ID: <56CEA60C.5050605@oracle.com>

For the record this looks fine to me too.

Thanks,
David
-----

On 18/02/2016 1:17 AM, cheleswer sahu wrote:
> Hi,
> I have made changes in the property name
> (jdk.lang.processReaperUseDefaultStackSize) and code as suggested in the
> earlier reviews.
>
>   --- old/src/java.base/share/classes/java/lang/ProcessHandleImpl.java
> 2016-02-17 18:48:10.545639999 +0530
> +++ new/src/java.base/share/classes/java/lang/ProcessHandleImpl.java
> 2016-02-17 18:48:10.081639999 +0530
> @@ -81,9 +81,8 @@
>                   ThreadGroup systemThreadGroup = tg;
>
>                   ThreadFactory threadFactory = grimReaper -> {
> -                    // Our thread stack requirement is quite modest.
> -                    Thread t = new Thread(systemThreadGroup, grimReaper,
> -                            "process reaper", 32768);
> +                    long stackSize =
> Boolean.getBoolean("jdk.lang.processReaperUseDefaultStackSize") ? 0 : 32768;
> +                    Thread t = new Thread(systemThreadGroup,
> grimReaper, "process reaper", stackSize);
>                       t.setDaemon(true);
>                       // A small attempt (probably futile) to avoid
> priority inversion
>                       t.setPriority(Thread.MAX_PRIORITY);
>
> I would really like to thanks "Martin" for the patch
> (http://cr.openjdk.java.net/~martin/webrevs/openjdk9/tls-size-guarantee/)
> which he has provided. Since we support a wider range of glibc versions
> and platform using patch will
> require more testing and work. I think the use of system property for
> this issue will be safer at this point of time.
>
> Regards,
> Cheleswer
>
>
> On 1/19/2016 5:40 PM, David Holmes wrote:
>> On 19/01/2016 9:53 PM, Kevin Walls wrote:
>>> |
>>> Hi Cheleswer, I think Martin is suggesting something like:
>>> |
>>>
>>> // Use a modest stack size, unless requested otherwise:
>>> long stackSize =
>>> Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0 : 32768;
>>
>> Someone from core-libs needs to chime on what the appropriate
>> namespace for such a property would be.
>>
>> David
>> -----
>>
>>> Thread t = new Thread(systemThreadGroup, grimReaper, "process
>>> reaper", stackSize);
>>>
>>> |||
>>> If that tests OK for you, it may be the way we can go ahead with the
>>> point fix for this.
>>>
>>> Regards
>>> Kevin
>>> |
>>> On 18/01/2016 16:52, Martin Buchholz wrote:
>>>> Historical note - I never liked having a reaper thread for each
>>>> subprocess, but no other reliable implementation is known. Given the
>>>> potential for many reaper threads, I at least wanted to keep the
>>>> memory waste low.
>>>>
>>>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu
>>>>  wrote:
>>>>
>>>>> +                   Thread t = null;
>>>>> +                   if
>>>>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) {
>>>>> +                       t = new Thread(systemThreadGroup, grimReaper,
>>>>> "process reaper");
>>>>> +                    } else {
>>>>> +                       // Our thread stack requirement is quite
>>>>> modest.
>>>>> +                       t = new Thread(systemThreadGroup, grimReaper,
>>>>> "process reaper", 32768);
>>>>> +                    }
>>>> If we do end up using this strategy, cleaner would be to use
>>>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long-
>>>>
>>>> stackSize - the desired stack size for the new thread, or zero to
>>>> indicate that this parameter is to be ignored.
>>>
>


From christoph.langer at sap.com  Thu Feb 25 07:02:05 2016
From: christoph.langer at sap.com (Langer, Christoph)
Date: Thu, 25 Feb 2016 07:02:05 +0000
Subject: RFR 8150470: JCK: api/xsl/conf/copy/copy19 test failure
In-Reply-To: <56CDA3C8.3040507@oracle.com>
References: <7d93f07319f54ba084fdd94adfac5e88@derote13de11.global.corp.sap>
	<56CDA3C8.3040507@oracle.com>
Message-ID: <615890f220334932bf4c6e8bde92df76@derote13de12.global.corp.sap>

Thanks Aleksej

From: Aleksej Efimov [mailto:aleksej.efimov at oracle.com]
Sent: Mittwoch, 24. Februar 2016 13:36
To: Langer, Christoph 
Cc: Joe Wang ; core-libs-dev at openjdk.java.net
Subject: Re: RFR 8150470: JCK: api/xsl/conf/copy/copy19 test failure

Hi Christoph,
The changes looks good to me too. I will push them shortly today.

With Best Regards,
Aleksej

On 02/24/2016 02:49 AM, Langer, Christoph wrote:
Hi,

the changeset for 8149915 (enabling validate-annotations feature for xsd schema with annotation causes NPE) introduced a regression. Here is the fix.

Bugid: https://bugs.openjdk.java.net/browse/JDK-8150470
Webrev: http://cr.openjdk.java.net/~clanger/webrevs/8150470.1/

Please review and push this one liner.

Thanks & Best regards
Christoph




From peter.levart at gmail.com  Thu Feb 25 07:24:24 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 25 Feb 2016 08:24:24 +0100
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: 
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com> <56CC8A4A.9080303@gmail.com>
	
Message-ID: <56CEAC28.80802@gmail.com>

Hi Kim, Roger, Mandy,

Resending with correct From: field to reach the list too...

On 02/24/2016 12:22 AM, Kim Barrett wrote:
>> On Feb 23, 2016, at 11:35 AM, Peter Levart  wrote:
>>
>> Hi Roger, Mandy,
>>
>> Here's another webrev:
>>
>> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.05/
>>
>> I renamed the method and reworded the specification. Is this better now?
>>
>>
>> On 02/22/2016 10:56 PM, Roger Riggs wrote:
>>> Hi Mandy,
>>>
>>> On 2/22/2016 4:41 PM, Mandy Chung wrote:
>>>
>>>> The existing way to do that is to register phantom references in your own ReferenceQueue and then drain the queue at appropriate point.  Would you consider having a method to return ReferenceQueue maintained by the cleaner instead?
>>> If the queue is exposed, there is no assurance that the cleanable function would be called.
>>> Any caller could introduce a bug by not doing the proper cleaning.
>>>
>>> I was more concerned with the crossing of Reference.tryHandlePending with the cleaning thread.
>>> The method description does not mention anything related to the Reference processing thread
>>> though that is all implementation.  The @implNote might be a bit more concise and less informal.
>>>
>>> Roger
>> Yes, ReferenceHandler thread is just an implementation detail. The specification of java.lang.Reference subclasses doesn't even mention it. It talks about GC enqueueing Reference objects:
>>
>> "Suppose that the garbage collector determines at a certain point in time...
>> ...At the same time or at some later time it will enqueue those newly-cleared weak references that are registered with reference queues."
>>
>> So in this respect ReferenceHandler is just an extension of GC Reference discovery/enqueuing logic, delegated to a background thread on Java side. The Cleaner.cleanNextPending() method tries to be true to its specification - it tries to invoke next cleanup action for which GC has determined that its monitored object is phantom reachable without waiting for ReferenceHandler thread to transfer it from pending list to the queue.
>>
>> Since Reference.tryHandlePending is just transfering Reference objects from one list to another and does that using two locks (the Reference.lock and the ReferenceQueue.lock) but never holds them both together or calls any outside code while holding any of the locks, there's no danger of dead-locking, if that was your concern.
>>
>> Regards, Peter
> ------------------------------------------------------------------------------
> src/java.base/share/classes/java/lang/ref/Cleaner.java
>   242     public boolean cleanNextPending() {
>   243         while (!impl.cleanNextEnqueuedCleanable()) {
>   244             if (!Reference.tryHandlePending(false)) {
>   245                 return false;
>   246             }
>   247         }
>   248         return true;
>   249     }
>
> This can loop for an arbitrarily long time if there are many pending
> references that are unrelated to the invoking Cleaner.  It could even
> loop arbitrarily long if there are lots of pending references and only
> a few scattered ones are for the invoking Cleaner, because the
> Cleaner's thread might take care of each of them before the helping
> thread re-checks the queue.

That could only happen if the rate of Reference discovery by STW GC 
phases was greater than the processing in this loop so that the loop 
could never drain the Reference.pending list. But that's a good point.

>
> The Disposer class that was mentioned as a use-case for this new
> function doesn't try to do anything like that; it just loops until
> there aren't any more in the queue.  The caller there is only affected
> by the number of enqueued objects, not by the number of unrelated
> pending objects.
>
> OTOH, while not structured exactly the same, I think this has a
> similar effect to the existing Direct-X-Buffer support provided by
> Bits.  Helping the single reference processing thread could be useful,
> though contention for the pending list lock could be a problem.  And
> this isn't *quite* the same as what we have now, because the reference
> will be enqueued and the Cleaner will be notified, so that the helper
> and the Cleaner compete to process the queued cleanup; with
> sun.misc.Cleaner the helper just directly invokes the cleanup.
>
> I'm not sure what to do about any of this; I'm just feeling like maybe
> the proposed API / implementation isn't quite right yet.

sun.misc.Cleaner was executed directly by the ReferenceHandler thread or 
whichever thread was helping it by invoking 
Reference.tryHandlePending(). Such helping was beneficial in nio Bits 
for two reasons:
- the cleanup work was done by more threads. Deallocating native memory 
does take some time > 0 as I have found out.
- some work was done synchronously in the allocating thread, so 
re-attempts to reserve native memory were scheduled immediately after 
releasing some memory which increased fairness and chances of success in 
spite of multiple threads trying to reserve concurrently.

If we remove special casing for sun.misc.Cleaner, then ReferenceHandler 
thread is left with just swapping pointers - no actual cleanup work. So 
I thought, maybe we could manage without helping it and just help the 
new Cleaner thread dequeue and execute Cleanables. This mostly works for 
4 concurrent allocating threads on my 4-core i7 PC (using 
DirectBufferAllocTest) which is better than nothing 
(DirectBufferAllocTest reliably fails with 2 allocating threads when not 
helping the Cleaner thread). But when overcommitting the 4 cores with 8 
or more allocating threads and helping just the Cleaner thread, the test 
still fails.

So I created another variant where I used a combined approach:

http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.06/

I kept the public boolean Cleaner.cleanNextPending() method which now 
only deals with enqueued Cleanable(s). I think this method might still 
be beneficial for public use in situations where cleanup actions take 
relatively long time to execute so that the rate of cleanup falls behind 
the rate of registration of new cleanup actions.

I resurrected the JavaLangRefAccess interface to patch into the 
ReferenceHandler thread processing from nio Bits, but I redesigned the 
logic. The redesigned method Reference.enqueuePendingReferences() takes 
a snapshot of current pending references in one go and enqueues them all 
without holding or reobtaining the Reference.lock. In addition, it waits 
for concurrent threads that took a snapshot of pending references before 
that to finish enqueuing them. The method therefore guarantees that all 
Reference(s) discovered by the time the method was called, are enqueued 
before returning.

Combining this with a fair lock in Bits.reserveMemory() taken after 1st 
optimistic attempt to reserve memory fails, works quite good. There's no 
sleeping necessary in Bits.reserveMemory and the DirectBufferAllocTest 
passes no matter how many allocating threads are involved. The 
throughput of allocation+deallocation of direct buffers is at least as 
good as before or even better as shown in this JMH benchmark test:

http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/DirectBufferAllocBench06.java

So what do you think of this variant?

>
> ------------------------------------------------------------------------------
> src/java.base/share/classes/java/nio/Direct-X-Buffer.java
>    34 import jdk.internal.ref.CleanerFactory;
> &etc.
>
> It might make sense to give Direct-X-Buffer its own dedicated Cleaner,
> rather than sharing the jdk.internal Cleaner.
>
> ------------------------------------------------------------------------------
>

Yes, it might make sense because direct buffers can be a heavy user. 
Each Cleaner uses it's own background thread though, so the number of 
JDK threads would increase. I wonder if it is possible to redesign the 
internals of ReferenceQueue so that a single thread could remove 
elements from multiple queues in a fair (round-robin) fashion. In such 
case, multiple Cleaners could share a thread but still enable draining 
just a chosen queue from the outside...

Regards, Peter



From Alan.Bateman at oracle.com  Thu Feb 25 08:00:26 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 25 Feb 2016 08:00:26 +0000
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CEAC28.80802@gmail.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com> <56CC8A4A.9080303@gmail.com>
	
	<56CEAC28.80802@gmail.com>
Message-ID: <56CEB49A.4090000@oracle.com>



On 25/02/2016 07:24, Peter Levart wrote:
> :
>
> I kept the public boolean Cleaner.cleanNextPending() method which now 
> only deals with enqueued Cleanable(s). I think this method might still 
> be beneficial for public use in situations where cleanup actions take 
> relatively long time to execute so that the rate of cleanup falls 
> behind the rate of registration of new cleanup actions.
I think we need also need to look at the option where this is not 
public. I have concerns that it is exposing implementation to some 
extent and that may become an attractive nuisance in the future. This 
shouldn't be an issue for the NIO buffer usage, we can keep the usage 
via the shared secrets mechanism. I think this is what Mandy is 
suggesting too.

-Alan.


From Alan.Bateman at oracle.com  Thu Feb 25 08:06:43 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Thu, 25 Feb 2016 08:06:43 +0000
Subject: RFR: 8144062: Determine appropriate module for jdk.Version
In-Reply-To: <86a4690e-7a83-4bc4-9c3a-9d617475f3bb@default>
References: <50765a19-34e4-4115-a32d-cea84c25394e@default>
	<38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com>
	<565628C5.1040905@oracle.com>
	<92b9301d-2418-4618-b877-b13c88216520@default>
	<7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default>
	<2f3ec2a8-4488-4c21-ae79-01c4599d5121@default>
	<56B1BFE3.1010201@oracle.com>
	
	<86a4690e-7a83-4bc4-9c3a-9d617475f3bb@default>
Message-ID: <56CEB613.8060207@oracle.com>

On 25/02/2016 05:59, Iris Clark wrote:
> Hi, Alan, Mandy, and Mark.
>
> After exploring a few module options (jdk.Version, jdk.dev [0]),
> it looks like the best choice is to move jdk.Version to
> java.lang.Runtime.Version (a nested class of Runtime).  It supports
> the values returned by the java.{vm.}?version and
> java.{vm.}?specification.version system properties.  By making
> Version an SE API, it may be exported by the java.base module.
>
> Please review the following changes to move jdk.Version to
> jdk.lang.Runtime.Version.  Note that beyond the package name and
> class declaration (to static), the only other changes in Version
> are related to indentation.
>
This looks good. We should probably change the synopsis on the JIRA 
issue to make it clearer that it is promoting Version to a standard API.

Once this is in then the multi-release JAR file patch can be updated to 
make use of this.

-Alan.


From amaembo at gmail.com  Thu Feb 25 09:02:58 2016
From: amaembo at gmail.com (Tagir Valeev)
Date: Thu, 25 Feb 2016 15:02:58 +0600
Subject: Collectors.minBy/maxBy behavior for null value
In-Reply-To: <1552250860.1046795.1456258479619.JavaMail.zimbra@u-pem.fr>
References: 
	<56CB24B9.4010907@gmail.com>
	<1552250860.1046795.1456258479619.JavaMail.zimbra@u-pem.fr>
Message-ID: 

Hello!

I have not checked first, but actually Stream.reduce(BinaryOperator),
Stream.max(Comparator) and Stream.min(Comparator) behave like findFirst():
all of them throw NPE if the result is null. So I would agree with Peter
and Remi: Collectors.reducing(BinaryOperator) should be changed to throw
NPE if the result of reduction is null to be consistent with Stream.reduce.

Seems that the fix is very simple, one line change in Collectors.reducing
finisher (like a -> a.present ? Optional.of(a.value) : Optional.empty()
instead of a -> Optional.ofNullable(a.value)). However this changes the
published (though unspecified) behavior. Should I log an issue and make a
patch?

With best regards,
Tagir Valeev.

On Wed, Feb 24, 2016 at 2:14 AM, Remi Forax  wrote:

> I agree with Peter,
> nulls in stream are tolerated for backward compatibility with existing
> code,
> null is not something that should trigger a specific behavior apart a NPE.
>
> regards,
> R?mi
>
> ----- Mail original -----
> > De: "Peter Levart" 
> > ?: "Tagir Valeev" , "core-libs-dev" <
> core-libs-dev at openjdk.java.net>
> > Envoy?: Lundi 22 F?vrier 2016 16:09:45
> > Objet: Re: Collectors.minBy/maxBy behavior for null value
> >
> >
> >
> > On 02/22/2016 11:06 AM, Tagir Valeev wrote:
> > > Hello!
> > >
> > > Consider the following code:
> > >
> > > Comparator cmp =
> Comparator.nullsFirst(Comparator.naturalOrder());
> > > System.out.println(Stream.of("a", "b",
> > > null).collect(Collectors.minBy(cmp)));
> > >
> > > It prints Optional.empty, so the result is indistinguishable from empty
> > > Stream (siimlar for maxBy). This behavior is not consistent with
> > > Stream.findFirst() (which throws NPE if the resulting element is null).
> > > Currently minBy spec says:
> > >
> > > This produces a result equivalent to:
> > >       reducing(BinaryOperator.minBy(comparator))
> > >
> > > However reducing spec says nothing what will occur if the result of
> > > reduction is null. To my opinion at very least the spec of reducing
> should
> > > be updated to specify the behavior when reducing result is null. It
> > > probably would also be nice to add a note into minBy/maxBy spec about
> this.
> > >
> > > What do you think?
> >
> > My personal opinion is that it would be better to throw NPE in this case
> > (like with findFirst()). Silently converting null element to
> > Optional.empty() is probably not the desired behavior and I doubt anyone
> > is exploiting this implementation detail in such way.
> >
> > Regards, Peter
> >
> > >
> > > With best regards,
> > > Tagir Valeev.
> >
> >
>


From paul.sandoz at oracle.com  Thu Feb 25 09:14:43 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Thu, 25 Feb 2016 10:14:43 +0100
Subject: RFR: 8144062: Determine appropriate module for jdk.Version
In-Reply-To: <56CEB613.8060207@oracle.com>
References: <50765a19-34e4-4115-a32d-cea84c25394e@default>
	<38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com>
	<565628C5.1040905@oracle.com>
	<92b9301d-2418-4618-b877-b13c88216520@default>
	<7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default>
	<2f3ec2a8-4488-4c21-ae79-01c4599d5121@default>
	<56B1BFE3.1010201@oracle.com>
	
	<86a4690e-7a83-4bc4-9c3a-9d617475f3bb@default>
	<56CEB613.8060207@oracle.com>
Message-ID: 


> On 25 Feb 2016, at 09:06, Alan Bateman  wrote:
> 
> On 25/02/2016 05:59, Iris Clark wrote:
>> Hi, Alan, Mandy, and Mark.
>> 
>> After exploring a few module options (jdk.Version, jdk.dev [0]),
>> it looks like the best choice is to move jdk.Version to
>> java.lang.Runtime.Version (a nested class of Runtime).  It supports
>> the values returned by the java.{vm.}?version and
>> java.{vm.}?specification.version system properties.  By making
>> Version an SE API, it may be exported by the java.base module.
>> 
>> Please review the following changes to move jdk.Version to
>> jdk.lang.Runtime.Version.  Note that beyond the package name and
>> class declaration (to static), the only other changes in Version
>> are related to indentation.
>> 
> This looks good. We should probably change the synopsis on the JIRA issue to make it clearer that it is promoting Version to a standard API.
> 

Dunno if the following was discussed for the review of internal impl.

I think it would be better to internally maintain an int[] for the version numbers rather than List, that is more efficient in terms of space and comparing/equality (you can even use the new array comparison methods, thus the compareVersion becomes a one-liner).

It?s ok to take the hit for returning List a simple AbstractList wrapper (also implementing RandomAccess) will suffice [*].


> Once this is in then the multi-release JAR file patch can be updated to make use of this.
> 

And since this will now be part of SE it might be possible to revisit (as a follow on issue) the definition and use of the JarFile.Release enum by MR-JARs.

Paul.

[*]
public List version() {
    class VersionList extends AbstractList implements RandomAccess {
        @Override
        public Integer get(int index) {
            return version[index];
        }

        @Override
        public int size() {
            return version.length;
        }
    }
    return new VersionList();
}


From peter.levart at gmail.com  Thu Feb 25 09:17:25 2016
From: peter.levart at gmail.com (Peter Levart)
Date: Thu, 25 Feb 2016 10:17:25 +0100
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CEB49A.4090000@oracle.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com> <56CC8A4A.9080303@gmail.com>
	
	<56CEAC28.80802@gmail.com> <56CEB49A.4090000@oracle.com>
Message-ID: <56CEC6A5.3070202@gmail.com>

Hi Alan,

On 02/25/2016 09:00 AM, Alan Bateman wrote:
>
>
> On 25/02/2016 07:24, Peter Levart wrote:
>> :
>>
>> I kept the public boolean Cleaner.cleanNextPending() method which now 
>> only deals with enqueued Cleanable(s). I think this method might 
>> still be beneficial for public use in situations where cleanup 
>> actions take relatively long time to execute so that the rate of 
>> cleanup falls behind the rate of registration of new cleanup actions.
> I think we need also need to look at the option where this is not 
> public. I have concerns that it is exposing implementation to some 
> extent and that may become an attractive nuisance in the future. This 
> shouldn't be an issue for the NIO buffer usage, we can keep the usage 
> via the shared secrets mechanism. I think this is what Mandy is 
> suggesting too.
>
> -Alan.

Sure, no problem. Here's a variant that keeps the 
Cleaner.cleanNextPending() method private and exposed via SharedSecrets 
to nio Bits but is otherwise equivalent to webrev.06:

http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.06priv/

Regards, Peter



From daniel.fuchs at oracle.com  Thu Feb 25 14:30:44 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Thu, 25 Feb 2016 15:30:44 +0100
Subject: RFR: 8150533 - Test
	java/util/logging/LogManagerAppContextDeadlock.java times out
	intermittently.
Message-ID: <56CF1014.1070500@oracle.com>

Hi,

Please find below a simple fix for

8150533 - Test java/util/logging/LogManagerAppContextDeadlock.java times 
out intermittently.

This is a test bug.

The issue is that the logger created by thread t2 might get
garbage collected before the main thread calls
LogManager.getLogger().

If that happens when there's no security manager then
the test will wait forever - and be killed by jtreg
timeout.

The fix is quite simple:

http://cr.openjdk.java.net/~dfuchs/webrev_8150533/webrev.00/

best regards,

-- daniel


From Roger.Riggs at Oracle.com  Thu Feb 25 15:16:56 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Thu, 25 Feb 2016 10:16:56 -0500
Subject: JDK 9 RFR of JDK-8150608: Mark BashStreams.java as intermittently
	failing and put to ProblemList
In-Reply-To: <0809E343-3FD6-4AAC-A1DB-A552F57A6AD8@oracle.com>
References: <56CE5DC0.2020906@oracle.com>
	<0809E343-3FD6-4AAC-A1DB-A552F57A6AD8@oracle.com>
Message-ID: <56CF1AE8.80904@Oracle.com>

Looks fine to put it on the problem list

Reviewed.

On 2/24/2016 8:57 PM, Brian Burkhalter wrote:
> Hi Amy,
>
> +1 for me but I am not a JDK 9 Reviewer.
>
> Thanks,
>
> Brian
>
> On Feb 24, 2016, at 5:49 PM, Amy Lu  wrote:
>
>> java/nio/charset/coders/BashStreams.java
>>
>> This test is known to fail intermittently, this patch is to mark the test accordingly with keyword 'intermittent? and put it to ProblemList.txt for further investigation.
>>
>> bug: https://bugs.openjdk.java.net/browse/JDK-8150608
>> webrev: http://cr.openjdk.java.net/~amlu/8150608/webrev.00/
>>
>> Thanks,
>> Amy



From nadeesh.tv at oracle.com  Thu Feb 25 15:44:20 2016
From: nadeesh.tv at oracle.com (nadeesh tv)
Date: Thu, 25 Feb 2016 21:14:20 +0530
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: 
References: <56BCED89.7040007@oracle.com>
	
Message-ID: <56CF2154.6050503@oracle.com>


Hi all,
Please see the updated webrev 
http://cr.openjdk.java.net/~ntv/8032051/webrev.02/

Thanks and Regards,
Nadeesh
On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
> Thanks for the changes.
>
> In `DateTimeFormatter`, the code should be
>
> .parseLenient()
> .appendOffsetId()
> .parseStrict()
>
> and the same in the other case. This ensures that existing callers who
> then embed the formatter in another formatter (like the
> ZONED_DATE_TIME constant) are unaffected.
>
>
> The logic for lenient parsing does not look right as it only handles
> types 5 and 6. This table shows the mappings needed:
>
> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
> "+HHmm",  -> "+HHmmss",
> "+HH:mm",  -> "+HH:mm:ss",
> "+HHMM",  -> "+HHmmss",
> "+HH:MM",  -> "+HH:mm:ss",
> "+HHMMss",  -> "+HHmmss",
> "+HH:MM:ss",  -> "+HH:mm:ss",
> "+HHMMSS",  -> "+HHmmss",
> "+HH:MM:SS",  -> "+HH:mm:ss",
> "+HHmmss",
> "+HH:mm:ss",
>
> Note that the "+HH" pattern is a special case, as we don't know
> whether to use the colon or non-colon pattern. Whether to require
> colon or not is based on whether the next character after the HH is a
> colon or not.
>
> Proposed appendOffsetId() Javadoc:
>
> * Appends the zone offset, such as '+01:00', to the formatter.
> * 
      
> * This appends an instruction to format/parse the offset ID to the builder.
> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
> * See {@link #appendOffset(String, String)} for details on formatting
> and parsing.
>
> Proposed appendOffset(String, String) Javadoc:
>
> * During parsing, the offset...
>
> changed to:
>
> * When parsing in strict mode, the input must contain the mandatory
> and optional elements are defined by the specified pattern.
> * If the offset cannot be parsed then an exception is thrown unless
> the section of the formatter is optional.
> * 
      
> * When parsing in lenient mode, only the hours are mandatory - minutes
> and seconds are optional.
> * The colons are required if the specified pattern contains a colon.
> * If the specified pattern is "+HH", the presence of colons is
> determined by whether the character after the hour digits is a colon
> or not.
> * If the offset cannot be parsed then an exception is thrown unless
> the section of the formatter is optional.
>
> thanks and sorry for delay
> Stephen
>
>
>
> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>> Hi all,
>>
>> Please review a fix for
>>
>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>
>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>
>> --
>> Thanks and Regards,
>> Nadeesh TV
>>

-- 
Thanks and Regards,
Nadeesh TV



From martinrb at google.com  Thu Feb 25 16:30:50 2016
From: martinrb at google.com (Martin Buchholz)
Date: Thu, 25 Feb 2016 08:30:50 -0800
Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in
	executables with 32k tbss/tdata
In-Reply-To: <56C59BC3.6040207@oracle.com>
References: <56962632.7080404@oracle.com>
	
	<569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com>
	<56C48EF2.3030403@oracle.com> <56C59BC3.6040207@oracle.com>
Message-ID: 

My hope is that we eventually eliminate StackOverflowError by
implementing dynamically growable thread stacks.

Until then, my hope is that hotspot give the thread the memory it asks
for, for actually storing stack frames, and thread locals should not
steal space from that.

The system property introduced in this change is an ugly workaround for that.

On Thu, Feb 18, 2016 at 2:24 AM, Kevin Walls  wrote:
> Hi Cheleswer,
>
> Looks good to me.
>
> Thanks
> Kevin
> (Also, as one of the comments was that there may be no real cost to using
> default stack sizes here (on most systems...?), having
> jdk.lang.processReaperUseDefaultStackSize gives us a way to test that
> widely, and one day the 32k may be able to disappear.)
>
>
>
>
>
> On 17/02/2016 15:17, cheleswer sahu wrote:
>
> Hi,
> I have made changes in the property name
> (jdk.lang.processReaperUseDefaultStackSize) and code as suggested in the
> earlier reviews.
>
>  --- old/src/java.base/share/classes/java/lang/ProcessHandleImpl.java
> 2016-02-17 18:48:10.545639999 +0530
> +++ new/src/java.base/share/classes/java/lang/ProcessHandleImpl.java
> 2016-02-17 18:48:10.081639999 +0530
> @@ -81,9 +81,8 @@
>                  ThreadGroup systemThreadGroup = tg;
>
>                  ThreadFactory threadFactory = grimReaper -> {
> -                    // Our thread stack requirement is quite modest.
> -                    Thread t = new Thread(systemThreadGroup, grimReaper,
> -                            "process reaper", 32768);
> +                    long stackSize =
> Boolean.getBoolean("jdk.lang.processReaperUseDefaultStackSize") ? 0 : 32768;
> +                    Thread t = new Thread(systemThreadGroup, grimReaper,
> "process reaper", stackSize);
>                      t.setDaemon(true);
>                      // A small attempt (probably futile) to avoid priority
> inversion
>                      t.setPriority(Thread.MAX_PRIORITY);
>
> I would really like to thanks "Martin" for the patch
> (http://cr.openjdk.java.net/~martin/webrevs/openjdk9/tls-size-guarantee/)
> which he has provided. Since we support a wider range of glibc versions and
> platform using patch will
> require more testing and work. I think the use of system property for this
> issue will be safer at this point of time.
>
> Regards,
> Cheleswer
>
>
> On 1/19/2016 5:40 PM, David Holmes wrote:
>
> On 19/01/2016 9:53 PM, Kevin Walls wrote:
>
> |
> Hi Cheleswer, I think Martin is suggesting something like:
> |
>
> // Use a modest stack size, unless requested otherwise:
> long stackSize = Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0
> : 32768;
>
>
> Someone from core-libs needs to chime on what the appropriate namespace for
> such a property would be.
>
> David
> -----
>
> Thread t = new Thread(systemThreadGroup, grimReaper, "process reaper",
> stackSize);
>
> |||
> If that tests OK for you, it may be the way we can go ahead with the
> point fix for this.
>
> Regards
> Kevin
> |
> On 18/01/2016 16:52, Martin Buchholz wrote:
>
> Historical note - I never liked having a reaper thread for each
> subprocess, but no other reliable implementation is known.  Given the
> potential for many reaper threads, I at least wanted to keep the
> memory waste low.
>
> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu
>   wrote:
>
> +                   Thread t = null;
> +                   if
> (Boolean.getBoolean("processReaperUseDefaultStackSize")) {
> +                       t = new Thread(systemThreadGroup, grimReaper,
> "process reaper");
> +                    } else {
> +                       // Our thread stack requirement is quite modest.
> +                       t = new Thread(systemThreadGroup, grimReaper,
> "process reaper", 32768);
> +                    }
>
> If we do end up using this strategy, cleaner would be to use
> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long-
> stackSize - the desired stack size for the new thread, or zero to
> indicate that this parameter is to be ignored.
>
>
>
>


From aph at redhat.com  Thu Feb 25 16:33:42 2016
From: aph at redhat.com (Andrew Haley)
Date: Thu, 25 Feb 2016 16:33:42 +0000
Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in
	executables with 32k tbss/tdata
In-Reply-To: 
References: <56962632.7080404@oracle.com>
	
	<569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com>
	<56C48EF2.3030403@oracle.com> <56C59BC3.6040207@oracle.com>
	
Message-ID: <56CF2CE6.7000408@redhat.com>

On 02/25/2016 04:30 PM, Martin Buchholz wrote:
> My hope is that we eventually eliminate StackOverflowError by
> implementing dynamically growable thread stacks.

Well, we've got this anyway, at least in Linux.  Isn't this something
that naturally should be pushed down to the thread library?

Andrew.



From thomas.stuefe at gmail.com  Thu Feb 25 16:51:54 2016
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Thu, 25 Feb 2016 17:51:54 +0100
Subject: RFR(s): 8150460: (linux|bsd|aix)_close.c: file descriptor table
	may become large or may not work at all
In-Reply-To: <56CE0441.6060308@Oracle.com>
References: 
	<56CE0441.6060308@Oracle.com>
Message-ID: 

Hi Roger,

thank you for the review!

New webrev:
http://cr.openjdk.java.net/~stuefe/webrevs/8150460-linux_close-fdTable/webrev.01/webrev/

Please find my comments inline.

On Wed, Feb 24, 2016 at 8:28 PM, Roger Riggs  wrote:

> Hi Thomas,
>
> On 2/24/2016 12:30 PM, Thomas St?fe wrote:
>
>> Hi all,
>>
>> please take a look at this proposed fix.
>>
>> The bug: https://bugs.openjdk.java.net/browse/JDK-8150460
>> The Webrev:
>>
>> http://cr.openjdk.java.net/~stuefe/webrevs/8150460-linux_close-fdTable/webrev.00/webrev/
>>
>> Basically, the file descriptor table implemented in linux_close.c may not
>> work for RLIMIT_NO_FILE=infinite or may grow very large (I saw a 50MB
>> table) for high values for RLIMIT_NO_FILE. Please see details in the bug
>> description.
>>
>> The proposed solution is to implement the file descriptor table not as
>> plain array, but as a twodimensional sparse array, which grows on demand.
>> This keeps the memory footprint small and fixes the corner cases described
>> in the bug description.
>>
>> Please note that the implemented solution is kept simple, at the cost of
>> somewhat higher (some kb) memory footprint for low values of
>> RLIMIT_NO_FILE.
>> This can be optimized, if we even think it is worth the trouble.
>>
>> Please also note that the proposed implementation now uses a mutex lock
>> for
>> every call to getFdEntry() - I do not think this matters, as this is all
>> in
>> preparation for an IO system call, which are usually way more expensive
>> than a pthread mutex. But again, this could be optimized.
>>
> I would suggest preallocating the index[0] array and then skip the mutex
> for that case.
> That would give the same as current performance.
>
>
I did this.


> And I'd suggest a different hi/low split of the indexes to reduce the size
> of pre-allocated memory.
> Most processes are going to use a lot fewer than 16384 fd's.  How about
> 2048?
>

I did this too. Now I calculate the split point based on RLIMIT_NO_FILE.
For small values of RLIMIT_NO_FILE
(<64K), I effectivly fall back to a one-dimensional array by making the
first level table a size 1. For larger values,
multiple second level tables, each 64K size, will be allocated on demand
(save for the first one which is preallocated).


> I have my doubts about needing to cover fd's  up to the full range of 32
> bits.
> Can the RLIMIT_NO_FILE be used too parametrize the allocation of the first
> level table?
>
>
I did this.

Interesting note, I found nowhere in the Posix specs a mentioning that
socked descriptors have to be handed out
sequentially and therefore cannot be larger than RLIMIT_NO_FILE. But in
reality on all operating systems file descriptors
seem to be [0, RLIMIT_NO_FILE).


Not specific to your change but it would nice to see consistency between
> libnio and libnet on
> the name of the sigWakeup/INTERRUPT_SIGNAL constant.


I agree, but this is out of the scope of this bug fix.


>
>
>> This is an implementation proposal for Linux; the same code found its way
>> to BSD and AIX. Should you approve of this fix, I will modify those files
>> too.
>>
> yes please.
>
> $.02, Roger
>
>
>
Thanks, Thomas


>
>> Thank you and Kind Regards, Thomas
>>
>
>


From joe.darcy at oracle.com  Thu Feb 25 17:43:58 2016
From: joe.darcy at oracle.com (Joseph D. Darcy)
Date: Thu, 25 Feb 2016 09:43:58 -0800
Subject: RFR: 8150533 - Test
	java/util/logging/LogManagerAppContextDeadlock.java
	times out intermittently.
In-Reply-To: <56CF1014.1070500@oracle.com>
References: <56CF1014.1070500@oracle.com>
Message-ID: <56CF3D5E.9030503@oracle.com>

Hi Daniel,

Looks fine; thanks,

-Joe

On 2/25/2016 6:30 AM, Daniel Fuchs wrote:
> Hi,
>
> Please find below a simple fix for
>
> 8150533 - Test java/util/logging/LogManagerAppContextDeadlock.java 
> times out intermittently.
>
> This is a test bug.
>
> The issue is that the logger created by thread t2 might get
> garbage collected before the main thread calls
> LogManager.getLogger().
>
> If that happens when there's no security manager then
> the test will wait forever - and be killed by jtreg
> timeout.
>
> The fix is quite simple:
>
> http://cr.openjdk.java.net/~dfuchs/webrev_8150533/webrev.00/
>
> best regards,
>
> -- daniel



From mandy.chung at oracle.com  Thu Feb 25 18:49:48 2016
From: mandy.chung at oracle.com (Mandy Chung)
Date: Thu, 25 Feb 2016 10:49:48 -0800
Subject: RFR: 8144062: Determine appropriate module for jdk.Version
In-Reply-To: <86a4690e-7a83-4bc4-9c3a-9d617475f3bb@default>
References: <50765a19-34e4-4115-a32d-cea84c25394e@default>
	<38DD183B-1D89-4C6F-B759-3F4512656F77@oracle.com>
	<565628C5.1040905@oracle.com>
	<92b9301d-2418-4618-b877-b13c88216520@default>
	<7ca276d8-5e4f-4da2-a1a9-999a0cac9f57@default>
	<2f3ec2a8-4488-4c21-ae79-01c4599d5121@default>
	<56B1BFE3.1010201@oracle.com>
	
	<86a4690e-7a83-4bc4-9c3a-9d617475f3bb@default>
Message-ID: 


> On Feb 24, 2016, at 9:59 PM, Iris Clark  wrote:
> 
> Hi, Alan, Mandy, and Mark.
> 
> After exploring a few module options (jdk.Version, jdk.dev [0]), 
> it looks like the best choice is to move jdk.Version to 
> java.lang.Runtime.Version (a nested class of Runtime).  It supports 
> the values returned by the java.{vm.}?version and 
> java.{vm.}?specification.version system properties.  By making 
> Version an SE API, it may be exported by the java.base module.
> 
> Please review the following changes to move jdk.Version to 
> jdk.lang.Runtime.Version.  Note that beyond the package name and
> class declaration (to static), the only other changes in Version
> are related to indentation.
> 
> Bug
> 
>    8144062: Determine appropriate module for jdk.Version
>    https://bugs.openjdk.java.net/browse/JDK-8144062
> 
> webrev
> 
>    http://cr.openjdk.java.net/~iris/verona/8144062/webrev.0/

942      * A representation of the JDK version-string

version string for the runtime instead of JDK

One suggestion is to move the details about fourth and later elements to @implSpec or @implNote.

Have you considered moving the static Version::current method to Runtime::version?

Mandy



From tprintezis at twitter.com  Thu Feb 25 20:48:04 2016
From: tprintezis at twitter.com (Tony Printezis)
Date: Thu, 25 Feb 2016 12:48:04 -0800
Subject: Frequent allocations / promotions of
	StringCoding$String{Encoder,Decoder} objects
Message-ID: 

Hi all,

We recently noticed that in several of our production services there are very frequent allocations / promotions of coder objects from the StringCoding class (e.g. java.lang.StringCoding$String{Encoder,Decoder} instances). We are pretty sure that these are allocated to replace the coders cached in the StringCoding ThreadLocals (we also see SoftReferences being allocated / promoted along with the coder objects):

? ? /** The cached coders for each thread */
? ? private final static ThreadLocal> decoder =
? ? ? ? new ThreadLocal<>();
? ? private final static ThreadLocal> encoder =
? ? ? ? new ThreadLocal<>();

We seem to be doing encoding / decoding using at least two different charsets (we also see Encoder / Decoders for those two charsets being allocated / promoted) which is causing this churn (i.e., the coder allocations are not caused by the SoftReferences being cleared; they are frequently replaced when a coder for a different charset is required). Even though we observed this behavior with JDK 8 it should also exist in JDK 9 (StringCoding still has the same two ThreadLocals; we can confirm this very easily).

Has anyone identified this issue before? We believe that caching a small number of coders per thread (instead of just one) could avoid this unnecessary churn. We?ll be happy to contribute such a fix if there?s interest in getting it accepted.

Also, why are SoftReferences used here? In our case, if a thread does some encoding / decoding once it?s likely it will likely keep doing it for ever. So the use of SoftReferences here is not very helpful (it adds unnecessary overhead to the GC, not only due to the extra copying but also due to the extra work required during reference processing). Was there a specific reason for the use of SoftReferences here?

Finally, the StringCoding coder c'tor allocates a new Charset coder (Charset{Encoder,Decoder}) for the specific charset. But such Charset coders already seem to be cached in ThreadLocals in the sun.nio.cs.ThreadLocalCoders class. Any reason why we cannot re-use those? (Oh, and also note that this cache does not use SoftReferences, which makes their use by the StringCoding class even more perplexing.)

(a tip of the hat to my colleague Peter Beaman for discovering this issue)

Tony

-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com



From richard.warburton at gmail.com  Thu Feb 25 20:57:56 2016
From: richard.warburton at gmail.com (Richard Warburton)
Date: Thu, 25 Feb 2016 20:57:56 +0000
Subject: Frequent allocations / promotions of StringCoding$String{Encoder,
	Decoder} objects
In-Reply-To: 
References: 
Message-ID: 

Hi,

Finally, the StringCoding coder c'tor allocates a new Charset coder
> (Charset{Encoder,Decoder}) for the specific charset. But such Charset
> coders already seem to be cached in ThreadLocals in the
> sun.nio.cs.ThreadLocalCoders class. Any reason why we cannot re-use those?
> (Oh, and also note that this cache does not use SoftReferences, which makes
> their use by the StringCoding class even more perplexing.)
>

+1. I was confused by this behaviour when I submitted a String related
patch a while back but never got round to submitting a fix. It actually
means that in String decoding often passing the charset to use by String is
faster than passing it Charset object - counter-intuitive and less typesafe.

regards,

  Richard Warburton

  http://insightfullogic.com
  @RichardWarburto 


From aleksey.shipilev at oracle.com  Thu Feb 25 21:16:27 2016
From: aleksey.shipilev at oracle.com (Aleksey Shipilev)
Date: Fri, 26 Feb 2016 00:16:27 +0300
Subject: Frequent allocations / promotions of
	StringCoding$String{Encoder,Decoder} objects
In-Reply-To: 
References: 
Message-ID: <56CF6F2B.8010506@oracle.com>

On 02/25/2016 11:48 PM, Tony Printezis wrote:
> Has anyone identified this issue before? 

Hm, there is a blast from the past:
 https://bugs.openjdk.java.net/browse/JDK-4806753

In comments there, Mark suggests a patch that apparently handles
multiple coders. I wonder where did that go. Also, Martin Buchholz is
still an assignee there :)

> We believe that caching a small number of coders per thread (instead
> of just one) could avoid this unnecessary churn. We?ll be happy to
> contribute such a fix if there?s interest in getting it accepted.

Yes, this seems important.

Thanks,
-Aleksey



From forax at univ-mlv.fr  Thu Feb 25 21:22:07 2016
From: forax at univ-mlv.fr (Remi Forax)
Date: Thu, 25 Feb 2016 22:22:07 +0100 (CET)
Subject: Frequent allocations / promotions of
	StringCoding$String{Encoder,	Decoder} objects
In-Reply-To: 
References: 
	
Message-ID: <273934007.9254851.1456435327155.JavaMail.zimbra@u-pem.fr>

Hi Tony, hi Richard,
i think it's a security feature,
otherwise you can create a rogue encoder that will be used for a Charset like UTF-8.

regards,
R?mi 


----- Mail original -----
> De: "Richard Warburton" 
> ?: "Tony Printezis" 
> Cc: "core-libs-dev" 
> Envoy?: Jeudi 25 F?vrier 2016 21:57:56
> Objet: Re: Frequent allocations / promotions of StringCoding$String{Encoder,	Decoder} objects
> 
> Hi,
> 
> Finally, the StringCoding coder c'tor allocates a new Charset coder
> > (Charset{Encoder,Decoder}) for the specific charset. But such Charset
> > coders already seem to be cached in ThreadLocals in the
> > sun.nio.cs.ThreadLocalCoders class. Any reason why we cannot re-use those?
> > (Oh, and also note that this cache does not use SoftReferences, which makes
> > their use by the StringCoding class even more perplexing.)
> >
> 
> +1. I was confused by this behaviour when I submitted a String related
> patch a while back but never got round to submitting a fix. It actually
> means that in String decoding often passing the charset to use by String is
> faster than passing it Charset object - counter-intuitive and less typesafe.
> 
> regards,
> 
>   Richard Warburton
> 
>   http://insightfullogic.com
>   @RichardWarburto 
> 


From xueming.shen at oracle.com  Thu Feb 25 21:21:28 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Thu, 25 Feb 2016 13:21:28 -0800
Subject: Frequent allocations / promotions of StringCoding$String{Encoder, 
	Decoder} objects
In-Reply-To: 
References: 
	
Message-ID: <56CF7058.1030009@oracle.com>

On 02/25/2016 12:57 PM, Richard Warburton wrote:
> Hi,
>
> Finally, the StringCoding coder c'tor allocates a new Charset coder
>> (Charset{Encoder,Decoder}) for the specific charset. But such Charset
>> coders already seem to be cached in ThreadLocals in the
>> sun.nio.cs.ThreadLocalCoders class. Any reason why we cannot re-use those?
>> (Oh, and also note that this cache does not use SoftReferences, which makes
>> their use by the StringCoding class even more perplexing.)
>>
> +1. I was confused by this behaviour when I submitted a String related
> patch a while back but never got round to submitting a fix. It actually
> means that in String decoding often passing the charset to use by String is
> faster than passing it Charset object - counter-intuitive and less typesafe.
>

We can't cache the "coder" from a passing in charset for security reason.

sherman




From richard.warburton at gmail.com  Thu Feb 25 21:29:42 2016
From: richard.warburton at gmail.com (Richard Warburton)
Date: Thu, 25 Feb 2016 21:29:42 +0000
Subject: Frequent allocations / promotions of StringCoding$String{Encoder, 
	Decoder} objects
In-Reply-To: <56CF7058.1030009@oracle.com>
References: 
	
	<56CF7058.1030009@oracle.com>
Message-ID: 

Hi,

+1. I was confused by this behaviour when I submitted a String related
>>>
>> patch a while back but never got round to submitting a fix. It actually
>> means that in String decoding often passing the charset to use by String
>> is
>> faster than passing it Charset object - counter-intuitive and less
>> typesafe.
>>
>>
> We can't cache the "coder" from a passing in charset for security reason.
>

Thanks for reminding me of this, apologies I had forgotten the issue.

Elsewhere in the string encoding/decoding code there is (or at least was
the last time I looked) an assumption that certain charset implementations
are "trusted" - basically ones by Oracle. It used to be ones on the
bootclasspath, I don't know about what happens in the modular world.
Wouldn't it be reasonable to trust those same charsets with this
optimisation as well? They are the most commonly used.

regards,

  Richard Warburton

  http://insightfullogic.com
  @RichardWarburto 


From xueming.shen at oracle.com  Thu Feb 25 21:38:34 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Thu, 25 Feb 2016 13:38:34 -0800
Subject: Frequent allocations / promotions of StringCoding$String{Encoder, 
	Decoder} objects
In-Reply-To: 
References: 		<56CF7058.1030009@oracle.com>
	
Message-ID: <56CF745A.10706@oracle.com>

On 02/25/2016 01:29 PM, Richard Warburton wrote:
> Hi,
>
>             +1. I was confused by this behaviour when I submitted a String related
>
>         patch a while back but never got round to submitting a fix. It actually
>         means that in String decoding often passing the charset to use by String is
>         faster than passing it Charset object - counter-intuitive and less typesafe.
>
>
>     We can't cache the "coder" from a passing in charset for security reason.
>
>
> Thanks for reminding me of this, apologies I had forgotten the issue.
>
> Elsewhere in the string encoding/decoding code there is (or at least was the last time I looked) an assumption that certain charset implementations are "trusted" - basically ones by Oracle. It used to be ones on the bootclasspath, I don't know about what happens in the modular world. Wouldn't it be reasonable to trust those same charsets with this optimisation as well? They are the most commonly used.
>
>

It's a good point. I think we probably can/should trust those charsets.

sherman


From xueming.shen at oracle.com  Thu Feb 25 21:49:06 2016
From: xueming.shen at oracle.com (Xueming Shen)
Date: Thu, 25 Feb 2016 13:49:06 -0800
Subject: Frequent allocations / promotions of StringCoding$String{Encoder,
	Decoder} objects
In-Reply-To: 
References: 
Message-ID: <56CF76D2.50209@oracle.com>

On 02/25/2016 12:48 PM, Tony Printezis wrote:
> Hi all,
>
> We recently noticed that in several of our production services there are very frequent allocations / promotions of coder objects from the StringCoding class (e.g. java.lang.StringCoding$String{Encoder,Decoder} instances). We are pretty sure that these are allocated to replace the coders cached in the StringCoding ThreadLocals (we also see SoftReferences being allocated / promoted along with the coder objects):
>
>      /** The cached coders for each thread */
>      private final static ThreadLocal>  decoder =
>          new ThreadLocal<>();
>      private final static ThreadLocal>  encoder =
>          new ThreadLocal<>();
>
> We seem to be doing encoding / decoding using at least two different charsets (we also see Encoder / Decoders for those two charsets being allocated / promoted) which is causing this churn (i.e., the coder allocations are not caused by the SoftReferences being cleared; they are frequently replaced when a coder for a different charset is required). Even though we observed this behavior with JDK 8 it should also exist in JDK 9 (StringCoding still has the same two ThreadLocals; we can confirm this very easily).
>
> Has anyone identified this issue before? We believe that caching a small number of coders per thread (instead of just one) could avoid this unnecessary churn. We?ll be happy to contribute such a fix if there?s interest in getting it accepted.
>

The assumption here is that "most programs to use one Charset repeatedly, especially
in one thread...". Obviously the assumption is wrong in your use case. It might be possible
to add multiple-entry/level cache for >= 1 coders, as we do in Charset class (it's not
threadlocal in Charset though).

sherman


From kevin.walls at oracle.com  Fri Feb 26 10:08:20 2016
From: kevin.walls at oracle.com (Kevin Walls)
Date: Fri, 26 Feb 2016 10:08:20 +0000
Subject: RFR[9u-dev] 8130425: libjvm crash due to stack overflow in
	executables with 32k tbss/tdata
In-Reply-To: 
References: <56962632.7080404@oracle.com>
	
	<569E239D.1020701@oracle.com> <569E27A2.4070207@oracle.com>
	<56C48EF2.3030403@oracle.com> <56C59BC3.6040207@oracle.com>
	
Message-ID: <56D02414.9090808@oracle.com>


Thanks Martin - I think we're going to proceed with the arguably 
somewhat ugly change as it does solve a real problem, although somewhat 
unusual to have such large thread local sizes for it to matter.  And we 
may have to deal with a thread library that takes away stack space for 
thread local data for a while.

Thanks
Kevin


On 25/02/2016 16:30, Martin Buchholz wrote:
> My hope is that we eventually eliminate StackOverflowError by
> implementing dynamically growable thread stacks.
>
> Until then, my hope is that hotspot give the thread the memory it asks
> for, for actually storing stack frames, and thread locals should not
> steal space from that.
>
> The system property introduced in this change is an ugly workaround for that.
>
> On Thu, Feb 18, 2016 at 2:24 AM, Kevin Walls  wrote:
>> Hi Cheleswer,
>>
>> Looks good to me.
>>
>> Thanks
>> Kevin
>> (Also, as one of the comments was that there may be no real cost to using
>> default stack sizes here (on most systems...?), having
>> jdk.lang.processReaperUseDefaultStackSize gives us a way to test that
>> widely, and one day the 32k may be able to disappear.)
>>
>>
>>
>>
>>
>> On 17/02/2016 15:17, cheleswer sahu wrote:
>>
>> Hi,
>> I have made changes in the property name
>> (jdk.lang.processReaperUseDefaultStackSize) and code as suggested in the
>> earlier reviews.
>>
>>   --- old/src/java.base/share/classes/java/lang/ProcessHandleImpl.java
>> 2016-02-17 18:48:10.545639999 +0530
>> +++ new/src/java.base/share/classes/java/lang/ProcessHandleImpl.java
>> 2016-02-17 18:48:10.081639999 +0530
>> @@ -81,9 +81,8 @@
>>                   ThreadGroup systemThreadGroup = tg;
>>
>>                   ThreadFactory threadFactory = grimReaper -> {
>> -                    // Our thread stack requirement is quite modest.
>> -                    Thread t = new Thread(systemThreadGroup, grimReaper,
>> -                            "process reaper", 32768);
>> +                    long stackSize =
>> Boolean.getBoolean("jdk.lang.processReaperUseDefaultStackSize") ? 0 : 32768;
>> +                    Thread t = new Thread(systemThreadGroup, grimReaper,
>> "process reaper", stackSize);
>>                       t.setDaemon(true);
>>                       // A small attempt (probably futile) to avoid priority
>> inversion
>>                       t.setPriority(Thread.MAX_PRIORITY);
>>
>> I would really like to thanks "Martin" for the patch
>> (http://cr.openjdk.java.net/~martin/webrevs/openjdk9/tls-size-guarantee/)
>> which he has provided. Since we support a wider range of glibc versions and
>> platform using patch will
>> require more testing and work. I think the use of system property for this
>> issue will be safer at this point of time.
>>
>> Regards,
>> Cheleswer
>>
>>
>> On 1/19/2016 5:40 PM, David Holmes wrote:
>>
>> On 19/01/2016 9:53 PM, Kevin Walls wrote:
>>
>> |
>> Hi Cheleswer, I think Martin is suggesting something like:
>> |
>>
>> // Use a modest stack size, unless requested otherwise:
>> long stackSize = Boolean.getBoolean("processReaperUseDefaultStackSize") ? 0
>> : 32768;
>>
>>
>> Someone from core-libs needs to chime on what the appropriate namespace for
>> such a property would be.
>>
>> David
>> -----
>>
>> Thread t = new Thread(systemThreadGroup, grimReaper, "process reaper",
>> stackSize);
>>
>> |||
>> If that tests OK for you, it may be the way we can go ahead with the
>> point fix for this.
>>
>> Regards
>> Kevin
>> |
>> On 18/01/2016 16:52, Martin Buchholz wrote:
>>
>> Historical note - I never liked having a reaper thread for each
>> subprocess, but no other reliable implementation is known.  Given the
>> potential for many reaper threads, I at least wanted to keep the
>> memory waste low.
>>
>> On Wed, Jan 13, 2016 at 2:25 AM, cheleswer sahu
>>   wrote:
>>
>> +                   Thread t = null;
>> +                   if
>> (Boolean.getBoolean("processReaperUseDefaultStackSize")) {
>> +                       t = new Thread(systemThreadGroup, grimReaper,
>> "process reaper");
>> +                    } else {
>> +                       // Our thread stack requirement is quite modest.
>> +                       t = new Thread(systemThreadGroup, grimReaper,
>> "process reaper", 32768);
>> +                    }
>>
>> If we do end up using this strategy, cleaner would be to use
>> https://docs.oracle.com/javase/8/docs/api/java/lang/Thread.html#Thread-java.lang.ThreadGroup-java.lang.Runnable-java.lang.String-long-
>> stackSize - the desired stack size for the new thread, or zero to
>> indicate that this parameter is to be ignored.
>>
>>
>>
>>



From scolebourne at joda.org  Fri Feb 26 11:16:45 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Fri, 26 Feb 2016 11:16:45 +0000
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: <56CF2154.6050503@oracle.com>
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
Message-ID: 

I don't think this is quite right.

if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
   parseType = 10;
}

This code will *always* select type 10 (colons) if a colon is found at
position+3. Whereas the spec now says that it should only do this if
the pattern is "HH". For other patterns, the colon/no-colon choice is
defined to be based on the pattern.

That said, I'm thinking it is better to make the spec more lenient to
match the behaviour as implemented:


When parsing in lenient mode, only the hours are mandatory - minutes
and seconds are optional. If the character after the hour digits is a colon
then the parser will parse using the pattern "HH:mm:ss", otherwise the
parser will parse using the pattern "HHmmss".


Additional TCKDateTimeFormatterBuilder tests will be needed to
demonstrate the above. There should also be a test for data following
the lenient parse. The following should all succeed:

DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
"+01:00Europe/London"
"+0100Europe/London"

DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
"+01:Europe/London"

Note this special case, where the colon affects the parse type, but is
not ultimately part of the offset, thus it is left to match the
appendLiteral(":")

You may want to think of some additional nasty edge cases!

Stephen

On 25 February 2016 at 15:44, nadeesh tv  wrote:
>
> Hi all,
> Please see the updated webrev
> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>
> Thanks and Regards,
> Nadeesh
>
> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>
>> Thanks for the changes.
>>
>> In `DateTimeFormatter`, the code should be
>>
>> .parseLenient()
>> .appendOffsetId()
>> .parseStrict()
>>
>> and the same in the other case. This ensures that existing callers who
>> then embed the formatter in another formatter (like the
>> ZONED_DATE_TIME constant) are unaffected.
>>
>>
>> The logic for lenient parsing does not look right as it only handles
>> types 5 and 6. This table shows the mappings needed:
>>
>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>> "+HHmm",  -> "+HHmmss",
>> "+HH:mm",  -> "+HH:mm:ss",
>> "+HHMM",  -> "+HHmmss",
>> "+HH:MM",  -> "+HH:mm:ss",
>> "+HHMMss",  -> "+HHmmss",
>> "+HH:MM:ss",  -> "+HH:mm:ss",
>> "+HHMMSS",  -> "+HHmmss",
>> "+HH:MM:SS",  -> "+HH:mm:ss",
>> "+HHmmss",
>> "+HH:mm:ss",
>>
>> Note that the "+HH" pattern is a special case, as we don't know
>> whether to use the colon or non-colon pattern. Whether to require
>> colon or not is based on whether the next character after the HH is a
>> colon or not.
>>
>> Proposed appendOffsetId() Javadoc:
>>
>> * Appends the zone offset, such as '+01:00', to the formatter.
>> * 
      
>> * This appends an instruction to format/parse the offset ID to the
>> builder.
>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>> * See {@link #appendOffset(String, String)} for details on formatting
>> and parsing.
>>
>> Proposed appendOffset(String, String) Javadoc:
>>
>> * During parsing, the offset...
>>
>> changed to:
>>
>> * When parsing in strict mode, the input must contain the mandatory
>> and optional elements are defined by the specified pattern.
>> * If the offset cannot be parsed then an exception is thrown unless
>> the section of the formatter is optional.
>> * 
      
>> * When parsing in lenient mode, only the hours are mandatory - minutes
>> and seconds are optional.
>> * The colons are required if the specified pattern contains a colon.
>> * If the specified pattern is "+HH", the presence of colons is
>> determined by whether the character after the hour digits is a colon
>> or not.
>> * If the offset cannot be parsed then an exception is thrown unless
>> the section of the formatter is optional.
>>
>> thanks and sorry for delay
>> Stephen
>>
>>
>>
>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>>
>>> Hi all,
>>>
>>> Please review a fix for
>>>
>>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>>
>>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>>
>>> --
>>> Thanks and Regards,
>>> Nadeesh TV
>>>
>
> --
> Thanks and Regards,
> Nadeesh TV
>


From michael.haupt at oracle.com  Fri Feb 26 13:01:48 2016
From: michael.haupt at oracle.com (Michael Haupt)
Date: Fri, 26 Feb 2016 14:01:48 +0100
Subject: RFR(M): 8150635: j.l.i.MethodHandles.loop(...) throws
	IndexOutOfBoundsException
Message-ID: 

Dear all,

please review this fix.
Bug: https://bugs.openjdk.java.net/browse/JDK-8150635
Webrev: http://cr.openjdk.java.net/~mhaupt/8150635/webrev.00

There were two issues to address that led to IOOBE being thrown.

1. Lack of support for deriving the loop signature from given parameter type lists in case no init function is given at all.

2. The loop combinator reporting a wrong exception in case a parameter type list is longer than the common parameter sequence. In this case, an error about non-effectively identical parameter type lists should be signalled.

See the JIRA issue for details and examples.

Thanks,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603

Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
 	Oracle is committed to developing practices and products that help protect the environment



From Roger.Riggs at Oracle.com  Fri Feb 26 14:15:42 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 26 Feb 2016 09:15:42 -0500
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: 
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
Message-ID: <56D05E0E.6030003@Oracle.com>

HI Stephen,

How lenient is lenient supposed to be? Looking at the offset test cases, 
it seems to allow  minutes
and seconds digits to be parsed even if the pattern did not include them.

+ @DataProvider(name="lenientOffsetParseData")
+ Object[][] data_lenient_offset_parse() {
+ return new Object[][] {
+ {"+HH", "+01", 3600},
+ {"+HH", "+0101", 3660},
+ {"+HH", "+010101", 3661},
+ {"+HH", "+01", 3600},
+ {"+HH", "+01:01", 3660},
+ {"+HH", "+01:01:01", 3661},

Thanks, Roger


On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
> I don't think this is quite right.
>
> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>     parseType = 10;
> }
>
> This code will *always* select type 10 (colons) if a colon is found at
> position+3. Whereas the spec now says that it should only do this if
> the pattern is "HH". For other patterns, the colon/no-colon choice is
> defined to be based on the pattern.
>
> That said, I'm thinking it is better to make the spec more lenient to
> match the behaviour as implemented:
>
>
> When parsing in lenient mode, only the hours are mandatory - minutes
> and seconds are optional. If the character after the hour digits is a colon
> then the parser will parse using the pattern "HH:mm:ss", otherwise the
> parser will parse using the pattern "HHmmss".
>
>
> Additional TCKDateTimeFormatterBuilder tests will be needed to
> demonstrate the above. There should also be a test for data following
> the lenient parse. The following should all succeed:
>
> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
> "+01:00Europe/London"
> "+0100Europe/London"
>
> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
> "+01:Europe/London"
>
> Note this special case, where the colon affects the parse type, but is
> not ultimately part of the offset, thus it is left to match the
> appendLiteral(":")
>
> You may want to think of some additional nasty edge cases!
>
> Stephen
>
> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>> Hi all,
>> Please see the updated webrev
>> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>>
>> Thanks and Regards,
>> Nadeesh
>>
>> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>> Thanks for the changes.
>>>
>>> In `DateTimeFormatter`, the code should be
>>>
>>> .parseLenient()
>>> .appendOffsetId()
>>> .parseStrict()
>>>
>>> and the same in the other case. This ensures that existing callers who
>>> then embed the formatter in another formatter (like the
>>> ZONED_DATE_TIME constant) are unaffected.
>>>
>>>
>>> The logic for lenient parsing does not look right as it only handles
>>> types 5 and 6. This table shows the mappings needed:
>>>
>>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>>> "+HHmm",  -> "+HHmmss",
>>> "+HH:mm",  -> "+HH:mm:ss",
>>> "+HHMM",  -> "+HHmmss",
>>> "+HH:MM",  -> "+HH:mm:ss",
>>> "+HHMMss",  -> "+HHmmss",
>>> "+HH:MM:ss",  -> "+HH:mm:ss",
>>> "+HHMMSS",  -> "+HHmmss",
>>> "+HH:MM:SS",  -> "+HH:mm:ss",
>>> "+HHmmss",
>>> "+HH:mm:ss",
>>>
>>> Note that the "+HH" pattern is a special case, as we don't know
>>> whether to use the colon or non-colon pattern. Whether to require
>>> colon or not is based on whether the next character after the HH is a
>>> colon or not.
>>>
>>> Proposed appendOffsetId() Javadoc:
>>>
>>> * Appends the zone offset, such as '+01:00', to the formatter.
>>> * 
      
>>> * This appends an instruction to format/parse the offset ID to the
>>> builder.
>>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>>> * See {@link #appendOffset(String, String)} for details on formatting
>>> and parsing.
>>>
>>> Proposed appendOffset(String, String) Javadoc:
>>>
>>> * During parsing, the offset...
>>>
>>> changed to:
>>>
>>> * When parsing in strict mode, the input must contain the mandatory
>>> and optional elements are defined by the specified pattern.
>>> * If the offset cannot be parsed then an exception is thrown unless
>>> the section of the formatter is optional.
>>> * 
      
>>> * When parsing in lenient mode, only the hours are mandatory - minutes
>>> and seconds are optional.
>>> * The colons are required if the specified pattern contains a colon.
>>> * If the specified pattern is "+HH", the presence of colons is
>>> determined by whether the character after the hour digits is a colon
>>> or not.
>>> * If the offset cannot be parsed then an exception is thrown unless
>>> the section of the formatter is optional.
>>>
>>> thanks and sorry for delay
>>> Stephen
>>>
>>>
>>>
>>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>>> Hi all,
>>>>
>>>> Please review a fix for
>>>>
>>>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>>>
>>>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>>>
>>>> --
>>>> Thanks and Regards,
>>>> Nadeesh TV
>>>>
>> --
>> Thanks and Regards,
>> Nadeesh TV
>>



From scolebourne at joda.org  Fri Feb 26 14:42:53 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Fri, 26 Feb 2016 14:42:53 +0000
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: <56D05E0E.6030003@Oracle.com>
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
	<56D05E0E.6030003@Oracle.com>
Message-ID: 

Lenient can be however lenient we define it to be. Allowing minutes
and seconds to be parsed when not specified in the pattern is the key
part of the change. Whether the parser copes with both colons and
no-colons is the choice at hand here. It seems to me that since the
parser can easily handle figuring out whether the colon is present or
not, we should just allow the parser to be fully lenient.

Stephen


On 26 February 2016 at 14:15, Roger Riggs  wrote:
> HI Stephen,
>
> How lenient is lenient supposed to be? Looking at the offset test cases, it
> seems to allow  minutes
> and seconds digits to be parsed even if the pattern did not include them.
>
> + @DataProvider(name="lenientOffsetParseData")
> + Object[][] data_lenient_offset_parse() {
> + return new Object[][] {
> + {"+HH", "+01", 3600},
> + {"+HH", "+0101", 3660},
> + {"+HH", "+010101", 3661},
> + {"+HH", "+01", 3600},
> + {"+HH", "+01:01", 3660},
> + {"+HH", "+01:01:01", 3661},
>
> Thanks, Roger
>
>
>
> On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
>>
>> I don't think this is quite right.
>>
>> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>>     parseType = 10;
>> }
>>
>> This code will *always* select type 10 (colons) if a colon is found at
>> position+3. Whereas the spec now says that it should only do this if
>> the pattern is "HH". For other patterns, the colon/no-colon choice is
>> defined to be based on the pattern.
>>
>> That said, I'm thinking it is better to make the spec more lenient to
>> match the behaviour as implemented:
>>
>>
>> When parsing in lenient mode, only the hours are mandatory - minutes
>> and seconds are optional. If the character after the hour digits is a
>> colon
>> then the parser will parse using the pattern "HH:mm:ss", otherwise the
>> parser will parse using the pattern "HHmmss".
>>
>>
>> Additional TCKDateTimeFormatterBuilder tests will be needed to
>> demonstrate the above. There should also be a test for data following
>> the lenient parse. The following should all succeed:
>>
>>
>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
>> "+01:00Europe/London"
>> "+0100Europe/London"
>>
>>
>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
>> "+01:Europe/London"
>>
>> Note this special case, where the colon affects the parse type, but is
>> not ultimately part of the offset, thus it is left to match the
>> appendLiteral(":")
>>
>> You may want to think of some additional nasty edge cases!
>>
>> Stephen
>>
>> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>>>
>>> Hi all,
>>> Please see the updated webrev
>>> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>>>
>>> Thanks and Regards,
>>> Nadeesh
>>>
>>> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>>>
>>>> Thanks for the changes.
>>>>
>>>> In `DateTimeFormatter`, the code should be
>>>>
>>>> .parseLenient()
>>>> .appendOffsetId()
>>>> .parseStrict()
>>>>
>>>> and the same in the other case. This ensures that existing callers who
>>>> then embed the formatter in another formatter (like the
>>>> ZONED_DATE_TIME constant) are unaffected.
>>>>
>>>>
>>>> The logic for lenient parsing does not look right as it only handles
>>>> types 5 and 6. This table shows the mappings needed:
>>>>
>>>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>>>> "+HHmm",  -> "+HHmmss",
>>>> "+HH:mm",  -> "+HH:mm:ss",
>>>> "+HHMM",  -> "+HHmmss",
>>>> "+HH:MM",  -> "+HH:mm:ss",
>>>> "+HHMMss",  -> "+HHmmss",
>>>> "+HH:MM:ss",  -> "+HH:mm:ss",
>>>> "+HHMMSS",  -> "+HHmmss",
>>>> "+HH:MM:SS",  -> "+HH:mm:ss",
>>>> "+HHmmss",
>>>> "+HH:mm:ss",
>>>>
>>>> Note that the "+HH" pattern is a special case, as we don't know
>>>> whether to use the colon or non-colon pattern. Whether to require
>>>> colon or not is based on whether the next character after the HH is a
>>>> colon or not.
>>>>
>>>> Proposed appendOffsetId() Javadoc:
>>>>
>>>> * Appends the zone offset, such as '+01:00', to the formatter.
>>>> * 
      
>>>> * This appends an instruction to format/parse the offset ID to the
>>>> builder.
>>>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>>>> * See {@link #appendOffset(String, String)} for details on formatting
>>>> and parsing.
>>>>
>>>> Proposed appendOffset(String, String) Javadoc:
>>>>
>>>> * During parsing, the offset...
>>>>
>>>> changed to:
>>>>
>>>> * When parsing in strict mode, the input must contain the mandatory
>>>> and optional elements are defined by the specified pattern.
>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>> the section of the formatter is optional.
>>>> * 
      
>>>> * When parsing in lenient mode, only the hours are mandatory - minutes
>>>> and seconds are optional.
>>>> * The colons are required if the specified pattern contains a colon.
>>>> * If the specified pattern is "+HH", the presence of colons is
>>>> determined by whether the character after the hour digits is a colon
>>>> or not.
>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>> the section of the formatter is optional.
>>>>
>>>> thanks and sorry for delay
>>>> Stephen
>>>>
>>>>
>>>>
>>>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Please review a fix for
>>>>>
>>>>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>>>>
>>>>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>> Nadeesh TV
>>>>>
>>> --
>>> Thanks and Regards,
>>> Nadeesh TV
>>>
>


From Roger.Riggs at Oracle.com  Fri Feb 26 15:00:06 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 26 Feb 2016 10:00:06 -0500
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: 
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
	<56D05E0E.6030003@Oracle.com>
	
Message-ID: <56D06876.4020000@Oracle.com>

Hi Stephen,

It does not seem natural to me with a pattern of HHMM for it to parse 
more than 4 digits.
I can see lenient modifying the behavior as it it were HHmm, but there 
is no indication in the pattern
that seconds would be considered.  How it would it be implied from the spec?

In the original issue, appendOffsetId is defined as using the +HH:MM:ss 
pattern and
specific to ISO the MM should be allowed to be optional.   There is no 
question of parsing
extra digits not included in the requested pattern.

Separately, this is specifying the new lenient behavior of 
appendOffset(pattern, noffsetText).
In that case, I don't think it will be understood that patterns 
'shorter' than the input will
gobble up extra digits and ':'s.

Roger




On 2/26/2016 9:42 AM, Stephen Colebourne wrote:
> Lenient can be however lenient we define it to be. Allowing minutes
> and seconds to be parsed when not specified in the pattern is the key
> part of the change. Whether the parser copes with both colons and
> no-colons is the choice at hand here. It seems to me that since the
> parser can easily handle figuring out whether the colon is present or
> not, we should just allow the parser to be fully lenient.
>
> Stephen
>
>
> On 26 February 2016 at 14:15, Roger Riggs  wrote:
>> HI Stephen,
>>
>> How lenient is lenient supposed to be? Looking at the offset test cases, it
>> seems to allow  minutes
>> and seconds digits to be parsed even if the pattern did not include them.
>>
>> + @DataProvider(name="lenientOffsetParseData")
>> + Object[][] data_lenient_offset_parse() {
>> + return new Object[][] {
>> + {"+HH", "+01", 3600},
>> + {"+HH", "+0101", 3660},
>> + {"+HH", "+010101", 3661},
>> + {"+HH", "+01", 3600},
>> + {"+HH", "+01:01", 3660},
>> + {"+HH", "+01:01:01", 3661},
>>
>> Thanks, Roger
>>
>>
>>
>> On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
>>> I don't think this is quite right.
>>>
>>> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>>>      parseType = 10;
>>> }
>>>
>>> This code will *always* select type 10 (colons) if a colon is found at
>>> position+3. Whereas the spec now says that it should only do this if
>>> the pattern is "HH". For other patterns, the colon/no-colon choice is
>>> defined to be based on the pattern.
>>>
>>> That said, I'm thinking it is better to make the spec more lenient to
>>> match the behaviour as implemented:
>>>
>>>
>>> When parsing in lenient mode, only the hours are mandatory - minutes
>>> and seconds are optional. If the character after the hour digits is a
>>> colon
>>> then the parser will parse using the pattern "HH:mm:ss", otherwise the
>>> parser will parse using the pattern "HHmmss".
>>>
>>>
>>> Additional TCKDateTimeFormatterBuilder tests will be needed to
>>> demonstrate the above. There should also be a test for data following
>>> the lenient parse. The following should all succeed:
>>>
>>>
>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
>>> "+01:00Europe/London"
>>> "+0100Europe/London"
>>>
>>>
>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
>>> "+01:Europe/London"
>>>
>>> Note this special case, where the colon affects the parse type, but is
>>> not ultimately part of the offset, thus it is left to match the
>>> appendLiteral(":")
>>>
>>> You may want to think of some additional nasty edge cases!
>>>
>>> Stephen
>>>
>>> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>>>> Hi all,
>>>> Please see the updated webrev
>>>> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>>>>
>>>> Thanks and Regards,
>>>> Nadeesh
>>>>
>>>> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>>>> Thanks for the changes.
>>>>>
>>>>> In `DateTimeFormatter`, the code should be
>>>>>
>>>>> .parseLenient()
>>>>> .appendOffsetId()
>>>>> .parseStrict()
>>>>>
>>>>> and the same in the other case. This ensures that existing callers who
>>>>> then embed the formatter in another formatter (like the
>>>>> ZONED_DATE_TIME constant) are unaffected.
>>>>>
>>>>>
>>>>> The logic for lenient parsing does not look right as it only handles
>>>>> types 5 and 6. This table shows the mappings needed:
>>>>>
>>>>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>>>>> "+HHmm",  -> "+HHmmss",
>>>>> "+HH:mm",  -> "+HH:mm:ss",
>>>>> "+HHMM",  -> "+HHmmss",
>>>>> "+HH:MM",  -> "+HH:mm:ss",
>>>>> "+HHMMss",  -> "+HHmmss",
>>>>> "+HH:MM:ss",  -> "+HH:mm:ss",
>>>>> "+HHMMSS",  -> "+HHmmss",
>>>>> "+HH:MM:SS",  -> "+HH:mm:ss",
>>>>> "+HHmmss",
>>>>> "+HH:mm:ss",
>>>>>
>>>>> Note that the "+HH" pattern is a special case, as we don't know
>>>>> whether to use the colon or non-colon pattern. Whether to require
>>>>> colon or not is based on whether the next character after the HH is a
>>>>> colon or not.
>>>>>
>>>>> Proposed appendOffsetId() Javadoc:
>>>>>
>>>>> * Appends the zone offset, such as '+01:00', to the formatter.
>>>>> * 
      
>>>>> * This appends an instruction to format/parse the offset ID to the
>>>>> builder.
>>>>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>>>>> * See {@link #appendOffset(String, String)} for details on formatting
>>>>> and parsing.
>>>>>
>>>>> Proposed appendOffset(String, String) Javadoc:
>>>>>
>>>>> * During parsing, the offset...
>>>>>
>>>>> changed to:
>>>>>
>>>>> * When parsing in strict mode, the input must contain the mandatory
>>>>> and optional elements are defined by the specified pattern.
>>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>>> the section of the formatter is optional.
>>>>> * 
      
>>>>> * When parsing in lenient mode, only the hours are mandatory - minutes
>>>>> and seconds are optional.
>>>>> * The colons are required if the specified pattern contains a colon.
>>>>> * If the specified pattern is "+HH", the presence of colons is
>>>>> determined by whether the character after the hour digits is a colon
>>>>> or not.
>>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>>> the section of the formatter is optional.
>>>>>
>>>>> thanks and sorry for delay
>>>>> Stephen
>>>>>
>>>>>
>>>>>
>>>>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> Please review a fix for
>>>>>>
>>>>>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>>>>>
>>>>>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>>>>>
>>>>>> --
>>>>>> Thanks and Regards,
>>>>>> Nadeesh TV
>>>>>>
>>>> --
>>>> Thanks and Regards,
>>>> Nadeesh TV
>>>>



From christoph.langer at sap.com  Fri Feb 26 15:01:54 2016
From: christoph.langer at sap.com (Langer, Christoph)
Date: Fri, 26 Feb 2016 15:01:54 +0000
Subject: RFR: JDK-8150704 XALAN: ERROR: 'No more DTM IDs are available' when
	transforming with lots of temporary result trees
Message-ID: <8d2c365d5ade48e59d939bf1ac624ae7@DEWDFE13DE11.global.corp.sap>

Hi,

I've created a fix proposal for the issue I have reported in this bug:
https://bugs.openjdk.java.net/browse/JDK-8150704

The webrev can be found here:
http://cr.openjdk.java.net/~clanger/webrevs/8150704.1/

The Xalan parser would eventually run out of DTM IDs if xsl transformations involve lots of temporary result trees. Those are never released although they could. A testcase is included for this. I've also done some cleanups in the Xalan code and in the tests.

Thanks in advance for looking at this :)

Best regards
Christoph



From Roger.Riggs at Oracle.com  Fri Feb 26 15:02:23 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 26 Feb 2016 10:02:23 -0500
Subject: RFR 9: 8150346 : java/lang/ProcessHandle/Infotest.java failed:
	startTime after process spawn completed
In-Reply-To: <56CCDD8F.3070807@Oracle.com>
References: <56CCDD8F.3070807@Oracle.com>
Message-ID: <56D068FF.20504@Oracle.com>

Ping...

On 2/23/2016 5:30 PM, Roger Riggs wrote:
> Please review a test fix to resolve a timing race with the start time 
> of a spawned process.
>
> Webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-start-8150346/
>
> Jira:
>    https://bugs.openjdk.java.net/browse/JDK-8150346
>
> Thanks, Roger
>



From claes.redestad at oracle.com  Fri Feb 26 15:11:51 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Fri, 26 Feb 2016 16:11:51 +0100
Subject: RFR 9: 8150346 : java/lang/ProcessHandle/Infotest.java failed:
	startTime after process spawn completed
In-Reply-To: <56D068FF.20504@Oracle.com>
References: <56CCDD8F.3070807@Oracle.com> <56D068FF.20504@Oracle.com>
Message-ID: <56D06B37.9010901@oracle.com>

Looks good to me.

/Claes

On 2016-02-26 16:02, Roger Riggs wrote:
> Ping...
>
> On 2/23/2016 5:30 PM, Roger Riggs wrote:
>> Please review a test fix to resolve a timing race with the start time 
>> of a spawned process.
>>
>> Webrev:
>>   http://cr.openjdk.java.net/~rriggs/webrev-start-8150346/
>>
>> Jira:
>>    https://bugs.openjdk.java.net/browse/JDK-8150346
>>
>> Thanks, Roger
>>
>



From vlashch at microsoft.com  Tue Feb 23 22:12:21 2016
From: vlashch at microsoft.com (Vladimir Shcherbakov)
Date: Tue, 23 Feb 2016 22:12:21 +0000
Subject: RFR 8124977 cmdline encoding challenges on Windows
In-Reply-To: <56CCA93C.6020807@oracle.com>
References: 
	
	<558C017A.2020708@oracle.com>
	
	<5651E9AA.9060908@oracle.com>
	
	
	<56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com>
	
	
	<56CB3099.4030108@oracle.com> <56CCA93C.6020807@oracle.com>
Message-ID: 

Hi Naoto,

1) The system locale determines which code page is used on the system by default on operating systems that use Unicode as their native encoding (all OSes  from Windows 2000 to Windows 10) to convert text data from Unicode to code page whenever dealing with legacy non-Unicode applications. Only applications that do not use Unicode as their default character-encoding mechanism are affected by this setting; therefore, applications that are already Unicode-encoded can safely ignore the value and functionality of this setting.

2) The fundamental representation of text in Windows NT-based operating systems is UTF-16, and the WCHAR data type is a UTF-16 code unit. Java launcher, from the other side, uses CHAR as a code unit - so to use UNICODE charset with Java launcher we had to encode entire command line with UTF-8 (convert from UTF-16 to UTF-8). After that step we can state that Java launcher is Unicode-encoded and can safely ignore the value and functionality of the system locale. To let JVM know that we use UTF-8 as a default UNICODE encoding for platform string  - we assign the value to sprops.sun_jnu_encoding property (mac osx does the same) instead of reading system locale code page. 

The main idea of the fix was to change the way of how java and javac works with so called platform string on Windows. Before the fix the platform string was read as ANSI encoded - that's why the system locale code page was very important. The sun.jnu.encoding property is responsible for storing the platform string encoding. On Windows the property could be set with the system locale but the system locale doesn't support (by design) UTF-8 or with -Dsun.jnu.encoding switch, but the switch only works with java not with javac, and the switch was useless for ANSI encoded platform string.

Thanks,
Vladimir.

-----Original Message-----
From: Naoto Sato [mailto:naoto.sato at oracle.com] 
Sent: Tuesday, February 23, 2016 10:47 AM
To: Kumar Srinivasan ; Vladimir Shcherbakov ; SHEN,XUEMING 
Cc: Martin Sawicki ; core-libs-dev Libs 
Subject: Re: RFR 8124977 cmdline encoding challenges on Windows

Hello,

Sorry if this has already been discussed, but this is my first time looking at the fix. In java_props_md.c, sprops.sun_jnu_encoding is now always "UTF-8". Is it always the case? What if the system admin switches the locale for "non-Unicode" applications in the Windows control panel?

Naoto

On 2/22/16 8:00 AM, Kumar Srinivasan wrote:
>
> Hi Naoto, Sherman,  can you please take a look.
> I tested with the jprt build and test all tests pass.
>
> Hi Vladimir, et. al.,
>
> It appears that there has been more simplifications from the previous 
> webrev.04. :-)
>
> It would've helped if you highlight the changes you have made from the 
> previous revision, unfortunately this is one of the deficiencies of 
> webrev.
>
> There are some inconsistencies in the coding conventions:
>
> parse_manifest.c
> + if (q == 0) return -1;
>
> we expect the return to be on the next line.
>
> similarly main.c
>
> if (0 == q)
> {
>
> I can fix those up. If I were to push this change, who should I 
> attribute the changes to ? ie. in the Contributed-by: line of the 
> commit info ?
> Please note these have to be email addresses of the contributors.
>
> Thanks
> Kumar
>
>> Hi Kumar,
>>
>> We posted another web review here:
>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.open
>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.05%2f&data=01%7C01%7Cvlashc
>> h%40microsoft.com%7Cf33316507f214e013a4008d33c81c785%7C72f988bf86f141
>> af91ab2d7cd011db47%7C1&sdata=%2fTQaWH0KGurgvZcdCQRZHSyaftjlMsW5FVc%2f
>> 14Wc5fA%3d
>>
>> The patch was successfully tested.
>>
>> Test details:
>> * Regression tests folder: jdk/test/tools/launcher/
>> * Builds were used: windows-x86_64-normal-server-fastdebug,
>> windows-x86_64-normal-server-release, 
>> windows-x86-normal-server-release;
>> * Platforms were used:  Windows 7(64 bit), Windows 8.1, Windows 
>> Server
>> 2012 R2 DC, Windows 10 ;
>> * System locales were used: English (United States), Persian, 
>> Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia);
>>
>> Thanks,
>> Vladimir.
>>
>> -----Original Message-----
>> From: Martin Sawicki
>> Sent: Thursday, January 14, 2016 11:34 AM
>> To: Kumar Srinivasan ; Vladimir 
>> Shcherbakov 
>> Cc: core-libs-dev Libs ; Naoto Sato 
>> 
>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Thanks for the feedback.
>> Investigating the regression failure.
>> We'll get back as soon as we figure this out.  (and yes, we'll run 
>> this through some localized Windows VMs)
>>
>> Cheers
>>
>> -----Original Message-----
>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>> Sent: Tuesday, January 12, 2016 2:35 PM
>> To: Martin Sawicki ; Vladimir Shcherbakov 
>> 
>> Cc: core-libs-dev Libs ; Naoto Sato 
>> 
>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Hi Martin, Vladimir,
>>
>> It was suggested that this patch be tested on localized Windows 
>> machines and/or trying with the various Windows native encodings, 
>> appreciate if you can verify this as well.
>>
>> Thanks
>> Kumar
>>
>> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote:
>>> Hi,
>>>
>>> Was on vacation, I started to prepare the patch from webrev.04 for 
>>> integration. Please note: made some adjustments to your patch to 
>>> pass jcheck, ie. usage of tabs and space at line endings, and 
>>> modifications to Copyright dates.
>>>
>>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE.
>>> I have attached a patch to for your reference.
>>>
>>> However, there is a regression test failure on Windows, 
>>> jdk/test/tools/launcher/I18NTest.java
>>>
>>> ---Test info----
>>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java
>>>
>>> ++++Test Output++++
>>> javac: file not found: i18nH?lloWorld.java ----End test info-----
>>>
>>> Have you run all the launcher regression tests with this changeset ?
>>>
>>> Thanks
>>> Kumar
>>>
>>>> Hi Kumar, just wondering if there are any updates on processing 
>>>> this submission.
>>>> Thanks!
>>>>
>>>> -----Original Message-----
>>>> From: Vladimir Shcherbakov
>>>> Sent: Wednesday, November 25, 2015 2:38 PM
>>>> To: Kumar Srinivasan ; Martin 
>>>> Sawicki 
>>>> Cc: Kirk Shoop ; core-libs-dev Libs 
>>>> 
>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>> Hi Kumar,
>>>>
>>>> Please find updated webreview here:
>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op
>>>> en 
>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cmarc
>>>> in
>>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86f1
>>>> 41 
>>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjpNf
>>>> wn
>>>> bIw%3d
>>>>
>>>> Thanks,
>>>> Vladimir.
>>>>
>>>> -----Original Message-----
>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>> Sent: Sunday, November 22, 2015 8:14 AM
>>>> To: Martin Sawicki 
>>>> Cc: Kirk Shoop ; Vladimir Shcherbakov 
>>>> ; core-libs-dev Libs 
>>>> 
>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>>
>>>> Hi Martin, et. al.,
>>>>
>>>> Sorry for not getting back earlier, I am very busy right now with 
>>>> my other large commitments for JDK9.
>>>>
>>>> I will sponsor this "enhancement/bug fix" sometime in the new year, 
>>>> meanwhile, there is the changeset  [1] which is likely to cause 
>>>> merge conflicts, and perhaps logic issues.
>>>>
>>>> Thanks
>>>> Kumar
>>>>
>>>> [1]
>>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.
>>>> op
>>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c01
>>>> %7 
>>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f98
>>>> 8b 
>>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNOJk
>>>> 7M
>>>> g4lt2P0sdzLS%2fT9Q%3d
>>>>> Hi all
>>>>> Here's an updated webrev attempting to take into account the 
>>>>> various pieces of feedback we have received:
>>>>>
>>>>> Issue:
>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs.
>>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%40m
>>>>> ic
>>>>> ro
>>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91ab
>>>>> 2d
>>>>> 7c
>>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsUIq
>>>>> 9E
>>>>> %3
>>>>> d
>>>>> Webrev:
>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.o
>>>>> pe
>>>>> nj
>>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvlas
>>>>> hc
>>>>> h%
>>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f14
>>>>> 1a
>>>>> f9
>>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN667B
>>>>> Jn
>>>>> WY
>>>>> 0%3d
>>>>>
>>>>> (Vladimir Shcherbakov is now working on this from our side)
>>>>>
>>>>> Looking forward to any other feedback.
>>>>> Thanks
>>>>>
>>>>> -----Original Message-----
>>>>> From: core-libs-dev 
>>>>> [mailto:core-libs-dev-bounces at openjdk.java.net]
>>>>> On Behalf Of Kumar Srinivasan
>>>>> Sent: Thursday, June 25, 2015 6:26 AM
>>>>> To: Kirk Shoop (MS OPEN TECH) 
>>>>> Cc: Valery Kopylov (Akvelon) ; 
>>>>> core-libs-dev Libs 
>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>
>>>>> Hi Kirk,
>>>>>
>>>>> Thanks for proposing this change.
>>>>>
>>>>> If you notice all the posix calls are wrapped in JLI_* this gives 
>>>>> us the ability to use "W" functions.  I almost got it done, 
>>>>> several years ago, but we upgraded to VS2010 and my work based on 
>>>>> VS2003 keeled over, meanwhile my focus was  "shifted" to something else.
>>>>>
>>>>> main.c: is really envisioned to be a stub  compiled by the tool 
>>>>> launchers, like java, javac, javah, jar etc. I prefer to see all 
>>>>> the heavy logic in this file moved to the platform specific file
>>>>> windows/java_md.*
>>>>>
>>>>> For the reason specified above we need to move fprintf or any 
>>>>> naked posix calls to JLI_* indirections.
>>>>>
>>>>> I don't see any tests ? The tests must be written in java and 
>>>>> placed in jdk/test/tools/launcher, there is a helper framework 
>>>>> TestHelper.java.
>>>>>
>>>>> There are other changes in nio, charsets etc, this will be 
>>>>> reviewed by my colleague specializing in that area (Sherman) cc'ed.
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>> Kumar
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Issue:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2f
>>>>>> bu
>>>>>> gs
>>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%4
>>>>>> 0m
>>>>>> ic
>>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af9
>>>>>> 1a
>>>>>> b2
>>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2f
>>>>>> sU
>>>>>> Iq
>>>>>> 9E%3d
>>>>>>
>>>>>> Webrev:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>> op
>>>>>> en
>>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40micr
>>>>>> os
>>>>>> of
>>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab2d
>>>>>> 7c
>>>>>> d0
>>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11B0
>>>>>> M%
>>>>>> 3d
>>>>>>
>>>>>> This webrev intends to address interaction between Windows 
>>>>>> console and java apps.
>>>>>>
>>>>>> Two switches were added that change the behavior of the launcher.
>>>>>> The defaults do not change the launcher behavior.
>>>>>>
>>>>>>       -Dwindows.UnicodeConsole=true - switches on Unicode support 
>>>>>> in the Windows console. This optional switch causes the launcher 
>>>>>> to call GetCommandLineW() and parse the arguments in unicode. It 
>>>>>> also modifies how the codepage for console output is selected.
>>>>>>
>>>>>>       -Dfile.encoding.unicode="UTF-8" - identifies Unicode 
>>>>>> charset to use; If not specified, UTF-8 is used by default. 
>>>>>> Ignored when windows.UnicodeConsole is not set to true. When the 
>>>>>> first switch is used, this optional switch allows the codepage 
>>>>>> for console output to be controlled.
>>>>>>
>>>>>> I would like to get feedback on the approach here and any 
>>>>>> additional work that is required solve these particular Unicode 
>>>>>> issues on Windows.
>>>>>>
>>>>>> Kirk
>

From vlashch at microsoft.com  Wed Feb 24 05:52:18 2016
From: vlashch at microsoft.com (Vladimir Shcherbakov)
Date: Wed, 24 Feb 2016 05:52:18 +0000
Subject: RFR 8124977 cmdline encoding challenges on Windows
In-Reply-To: <56CD3754.1050106@oracle.com>
References: 
	
	<558C017A.2020708@oracle.com>
	
	<5651E9AA.9060908@oracle.com>
	
	
	<56941A34.8090608@oracle.com> <56957F88.6060708@oracle.com>
	
	
	<56CB3099.4030108@oracle.com> <56CCA93C.6020807@oracle.com>
	
	<56CCFA7F.705@oracle.com> <56CD3754.1050106@oracle.com>
Message-ID: 

Hi Sherman,

1) If you can point out the regression test cases that are compromised by the fix - it would be very helpful;
2) From my understanding you can change default encoding by starting java with -Dsun.jnu.encoding=UTF-8 - this is well known feature that never caused problems (javac doesn't have such a switch );
3) If you state that java is non-Unicode on Windows by nature - the issue  JDK-8124977 is a feature not a bug  :)

Thanks,
Vladimir.

-----Original Message-----
From: Xueming Shen [mailto:xueming.shen at oracle.com] 
Sent: Tuesday, February 23, 2016 8:54 PM
To: Vladimir Shcherbakov 
Cc: Naoto Sato ; Kumar Srinivasan ; Martin Sawicki ; core-libs-dev Libs 
Subject: Re: RFR 8124977 cmdline encoding challenges on Windows

Vladimir,

sun.jnu.encoding is used by
JNU_NewStringPlatform/JNU_GetStringPlatformChars. The JNU_ pair is "widely" used by the various native library code to convert between the jstring and native char*, with the assumption that the "platform encoding" for the native char* is the "default" encoding used by the underlying platform/os APIs that takes char* parameters or return char* values, in case of Windows, it's the code page decided by the system locale. We have migrated certain areas completely to use the "W" version/WChar APIs, such as the https://na01.safelinks.protection.outlook.com/?url=java.io&data=01%7c01%7cvlashch%40microsoft.com%7c635061d867af4ad4105008d33cd679e7%7c72f988bf86f141af91ab2d7cd011db47%7c1&sdata=l4G1yzKKhniPRYJvBsGxchsBZvuWliVq8tILa0pLoY8%3d, the system properties initialization, but I'm think lots of areas still work on the "A" apis, especially I think the "char*" interface between the jvm and the libraries is still the the "ansi" codepage, not the utf8. Those work on utf8 have their names explicitly named as "xyzUTF" or similar.

For example, the "java_home_dir" path used in libjava/TimeZone.c/getSystemTimeZoneID/
TimeZone_md.c/findJavaTZ_md is encoded frm jstring java_home to char* via JNU_GetSTringPlatformChars.
Simply change/hardcode the jnu_sun.encoding to utf8 probably will cause the timezone code stop to work if the java_home_dir path has some non-ascii characters in it (the jdk/jre is installed in a Japanese/Chinese directory, for example).

A quick "grep" indicates java.desktop/windows/native/libawt/windows
package has a heavily
usage of the JNU_ pair as well. I'm not sure if this awt implementation is still being used though :-)

Before we clear all these internal "StringPlatform" use cases (I'm not sure if they are also used by external), I don't think we can simply set the sun.jnu.encoding to utf8, though it's very attractive.

Thanks,
-Sherman

On 2/23/16 4:34 PM, Naoto Sato wrote:
> Hi Vladimir,
>
> I think it would work fine with the Java launcher, but what about 
> other areas, which may rely on the native encodings? Java runtime is 
> in itself a "non-Unicode" application, so still there may be the area 
> affected by hardcoding "UTF-8" as the native encoding. Have you 
> checked in such cases? Sherman, will you comment on this too?
>
> Naoto
>
> On 2/23/16 2:12 PM, Vladimir Shcherbakov wrote:
>> Hi Naoto,
>>
>> 1) The system locale determines which code page is used on the system 
>> by default on operating systems that use Unicode as their native 
>> encoding (all OSes  from Windows 2000 to Windows 10) to convert text 
>> data from Unicode to code page whenever dealing with legacy 
>> non-Unicode applications. Only applications that do not use Unicode 
>> as their default character-encoding mechanism are affected by this 
>> setting; therefore, applications that are already Unicode-encoded can 
>> safely ignore the value and functionality of this setting.
>>
>> 2) The fundamental representation of text in Windows NT-based 
>> operating systems is UTF-16, and the WCHAR data type is a UTF-16 code 
>> unit. Java launcher, from the other side, uses CHAR as a code unit - 
>> so to use UNICODE charset with Java launcher we had to encode entire 
>> command line with UTF-8 (convert from UTF-16 to UTF-8). After that 
>> step we can state that Java launcher is Unicode-encoded and can 
>> safely ignore the value and functionality of the system locale. To 
>> let JVM know that we use UTF-8 as a default UNICODE encoding for 
>> platform string  - we assign the value to sprops.sun_jnu_encoding 
>> property (mac osx does the same) instead of reading system locale 
>> code page.
>>
>> The main idea of the fix was to change the way of how java and javac 
>> works with so called platform string on Windows. Before the fix the 
>> platform string was read as ANSI encoded - that's why the system 
>> locale code page was very important. The sun.jnu.encoding property is 
>> responsible for storing the platform string encoding. On Windows the 
>> property could be set with the system locale but the system locale 
>> doesn't support (by design) UTF-8 or with -Dsun.jnu.encoding switch, 
>> but the switch only works with java not with javac, and the switch 
>> was useless for ANSI encoded platform string.
>>
>> Thanks,
>> Vladimir.
>>
>> -----Original Message-----
>> From: Naoto Sato [mailto:naoto.sato at oracle.com]
>> Sent: Tuesday, February 23, 2016 10:47 AM
>> To: Kumar Srinivasan ; Vladimir 
>> Shcherbakov ; SHEN,XUEMING 
>> 
>> Cc: Martin Sawicki ; core-libs-dev Libs 
>> 
>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>
>> Hello,
>>
>> Sorry if this has already been discussed, but this is my first time 
>> looking at the fix. In java_props_md.c, sprops.sun_jnu_encoding is 
>> now always "UTF-8". Is it always the case? What if the system admin 
>> switches the locale for "non-Unicode" applications in the Windows 
>> control panel?
>>
>> Naoto
>>
>> On 2/22/16 8:00 AM, Kumar Srinivasan wrote:
>>>
>>> Hi Naoto, Sherman,  can you please take a look.
>>> I tested with the jprt build and test all tests pass.
>>>
>>> Hi Vladimir, et. al.,
>>>
>>> It appears that there has been more simplifications from the 
>>> previous webrev.04. :-)
>>>
>>> It would've helped if you highlight the changes you have made from 
>>> the previous revision, unfortunately this is one of the deficiencies 
>>> of webrev.
>>>
>>> There are some inconsistencies in the coding conventions:
>>>
>>> parse_manifest.c
>>> + if (q == 0) return -1;
>>>
>>> we expect the return to be on the next line.
>>>
>>> similarly main.c
>>>
>>> if (0 == q)
>>> {
>>>
>>> I can fix those up. If I were to push this change, who should I 
>>> attribute the changes to ? ie. in the Contributed-by: line of the 
>>> commit info ?
>>> Please note these have to be email addresses of the contributors.
>>>
>>> Thanks
>>> Kumar
>>>
>>>> Hi Kumar,
>>>>
>>>> We posted another web review here:
>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.op
>>>> en 
>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.05%2f&data=01%7C01%7Cvlas
>>>> hc
>>>> h%40microsoft.com%7Cf33316507f214e013a4008d33c81c785%7C72f988bf86f1
>>>> 41 
>>>> af91ab2d7cd011db47%7C1&sdata=%2fTQaWH0KGurgvZcdCQRZHSyaftjlMsW5FVc%
>>>> 2f
>>>> 14Wc5fA%3d
>>>>
>>>> The patch was successfully tested.
>>>>
>>>> Test details:
>>>> * Regression tests folder: jdk/test/tools/launcher/
>>>> * Builds were used: windows-x86_64-normal-server-fastdebug,
>>>> windows-x86_64-normal-server-release,
>>>> windows-x86-normal-server-release;
>>>> * Platforms were used:  Windows 7(64 bit), Windows 8.1, Windows 
>>>> Server
>>>> 2012 R2 DC, Windows 10 ;
>>>> * System locales were used: English (United States), Persian, 
>>>> Japanese (Japan), Chinese (Traditional, Taiwan), Russian (Russia);
>>>>
>>>> Thanks,
>>>> Vladimir.
>>>>
>>>> -----Original Message-----
>>>> From: Martin Sawicki
>>>> Sent: Thursday, January 14, 2016 11:34 AM
>>>> To: Kumar Srinivasan ; Vladimir 
>>>> Shcherbakov 
>>>> Cc: core-libs-dev Libs ; Naoto Sato 
>>>> 
>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>> Thanks for the feedback.
>>>> Investigating the regression failure.
>>>> We'll get back as soon as we figure this out.  (and yes, we'll run 
>>>> this through some localized Windows VMs)
>>>>
>>>> Cheers
>>>>
>>>> -----Original Message-----
>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>> Sent: Tuesday, January 12, 2016 2:35 PM
>>>> To: Martin Sawicki ; Vladimir Shcherbakov 
>>>> 
>>>> Cc: core-libs-dev Libs ; Naoto Sato 
>>>> 
>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>
>>>> Hi Martin, Vladimir,
>>>>
>>>> It was suggested that this patch be tested on localized Windows 
>>>> machines and/or trying with the various Windows native encodings, 
>>>> appreciate if you can verify this as well.
>>>>
>>>> Thanks
>>>> Kumar
>>>>
>>>> On 1/11/2016 1:10 PM, Kumar Srinivasan wrote:
>>>>> Hi,
>>>>>
>>>>> Was on vacation, I started to prepare the patch from webrev.04 for 
>>>>> integration. Please note: made some adjustments to your patch to 
>>>>> pass jcheck, ie. usage of tabs and space at line endings, and 
>>>>> modifications to Copyright dates.
>>>>>
>>>>> Also fixed a minor bug on unix replaced JLI_TRUE with JNI_TRUE.
>>>>> I have attached a patch to for your reference.
>>>>>
>>>>> However, there is a regression test failure on Windows, 
>>>>> jdk/test/tools/launcher/I18NTest.java
>>>>>
>>>>> ---Test info----
>>>>> Executed command: C:\mmm\jdk\bin\javac.exe i18nH?lloWorld.java
>>>>>
>>>>> ++++Test Output++++
>>>>> javac: file not found: i18nH?lloWorld.java ----End test info-----
>>>>>
>>>>> Have you run all the launcher regression tests with this changeset ?
>>>>>
>>>>> Thanks
>>>>> Kumar
>>>>>
>>>>>> Hi Kumar, just wondering if there are any updates on processing 
>>>>>> this submission.
>>>>>> Thanks!
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Vladimir Shcherbakov
>>>>>> Sent: Wednesday, November 25, 2015 2:38 PM
>>>>>> To: Kumar Srinivasan ; Martin 
>>>>>> Sawicki 
>>>>>> Cc: Kirk Shoop ; core-libs-dev Libs 
>>>>>> 
>>>>>> Subject: RE: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>
>>>>>> Hi Kumar,
>>>>>>
>>>>>> Please find updated webreview here:
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>> op
>>>>>> en
>>>>>> jdk.java.net%2f~kshoop%2f8124977%2fwebrev.04%2f&data=01%7C01%7Cma
>>>>>> rc
>>>>>> in
>>>>>> s%40microsoft.com%7C13ff309b775c4c019fc308d31ba0c43c%7C72f988bf86
>>>>>> f1
>>>>>> 41
>>>>>> af91ab2d7cd011db47%7C1&sdata=3hhbO5mNPyTvtrTb4kCR42zsWGPGzDhqnmjp
>>>>>> Nf
>>>>>> wn
>>>>>> bIw%3d
>>>>>>
>>>>>> Thanks,
>>>>>> Vladimir.
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com]
>>>>>> Sent: Sunday, November 22, 2015 8:14 AM
>>>>>> To: Martin Sawicki 
>>>>>> Cc: Kirk Shoop ; Vladimir Shcherbakov 
>>>>>> ; core-libs-dev Libs 
>>>>>> 
>>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>
>>>>>>
>>>>>> Hi Martin, et. al.,
>>>>>>
>>>>>> Sorry for not getting back earlier, I am very busy right now with 
>>>>>> my other large commitments for JDK9.
>>>>>>
>>>>>> I will sponsor this "enhancement/bug fix" sometime in the new 
>>>>>> year, meanwhile, there is the changeset  [1] which is likely to 
>>>>>> cause merge conflicts, and perhaps logic issues.
>>>>>>
>>>>>> Thanks
>>>>>> Kumar
>>>>>>
>>>>>> [1]
>>>>>> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2fhg.
>>>>>> op
>>>>>> enjdk.java.net%2fjdk9%2fdev%2fjdk%2frev%2f3b201a9ef918&data=01%7c
>>>>>> 01
>>>>>> %7
>>>>>> cvlashch%40microsoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f
>>>>>> 98
>>>>>> 8b
>>>>>> f86f141af91ab2d7cd011db47%7c1&sdata=I2FKvBn82%2fxhW3D%2fi%2bRWaNO
>>>>>> Jk
>>>>>> 7M
>>>>>> g4lt2P0sdzLS%2fT9Q%3d
>>>>>>> Hi all
>>>>>>> Here's an updated webrev attempting to take into account the 
>>>>>>> various pieces of feedback we have received:
>>>>>>>
>>>>>>> Issue:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fbugs. 
>>>>>>>
>>>>>>> openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch%4
>>>>>>> 0m
>>>>>>> ic
>>>>>>> ro
>>>>>>> soft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141af91
>>>>>>> ab
>>>>>>> 2d
>>>>>>> 7c
>>>>>>> d011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%2fsU
>>>>>>> Iq
>>>>>>> 9E
>>>>>>> %3
>>>>>>> d
>>>>>>> Webrev:
>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr
>>>>>>> .o
>>>>>>> pe
>>>>>>> nj
>>>>>>> dk.java.net%2f~kshoop%2f8124977%2fwebrev.03%2f&data=01%7C01%7Cvl
>>>>>>> as
>>>>>>> hc
>>>>>>> h%
>>>>>>> 40microsoft.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f
>>>>>>> 14
>>>>>>> 1a
>>>>>>> f9
>>>>>>> 1ab2d7cd011db47%7C1&sdata=101HBPar2AZ63GJWyubWH0DiKmNI%2bOxknN66
>>>>>>> 7B
>>>>>>> Jn
>>>>>>> WY
>>>>>>> 0%3d
>>>>>>>
>>>>>>> (Vladimir Shcherbakov is now working on this from our side)
>>>>>>>
>>>>>>> Looking forward to any other feedback.
>>>>>>> Thanks
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: core-libs-dev
>>>>>>> [mailto:core-libs-dev-bounces at openjdk.java.net]
>>>>>>> On Behalf Of Kumar Srinivasan
>>>>>>> Sent: Thursday, June 25, 2015 6:26 AM
>>>>>>> To: Kirk Shoop (MS OPEN TECH) 
>>>>>>> Cc: Valery Kopylov (Akvelon) ; 
>>>>>>> core-libs-dev Libs 
>>>>>>> Subject: Re: RFR 8124977 cmdline encoding challenges on Windows
>>>>>>>
>>>>>>> Hi Kirk,
>>>>>>>
>>>>>>> Thanks for proposing this change.
>>>>>>>
>>>>>>> If you notice all the posix calls are wrapped in JLI_* this 
>>>>>>> gives us the ability to use "W" functions.  I almost got it 
>>>>>>> done, several years ago, but we upgraded to VS2010 and my work 
>>>>>>> based on
>>>>>>> VS2003 keeled over, meanwhile my focus was  "shifted" to 
>>>>>>> something else.
>>>>>>>
>>>>>>> main.c: is really envisioned to be a stub  compiled by the tool 
>>>>>>> launchers, like java, javac, javah, jar etc. I prefer to see all 
>>>>>>> the heavy logic in this file moved to the platform specific file
>>>>>>> windows/java_md.*
>>>>>>>
>>>>>>> For the reason specified above we need to move fprintf or any 
>>>>>>> naked posix calls to JLI_* indirections.
>>>>>>>
>>>>>>> I don't see any tests ? The tests must be written in java and 
>>>>>>> placed in jdk/test/tools/launcher, there is a helper framework 
>>>>>>> TestHelper.java.
>>>>>>>
>>>>>>> There are other changes in nio, charsets etc, this will be 
>>>>>>> reviewed by my colleague specializing in that area (Sherman) cc'ed.
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>> Kumar
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 6/22/2015 2:01 PM, Kirk Shoop (MS OPEN TECH) wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Issue:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%
>>>>>>>> 2f
>>>>>>>> bu
>>>>>>>> gs
>>>>>>>> .openjdk.java.net%2fbrowse%2fJDK-8124977&data=01%7c01%7cvlashch
>>>>>>>> %4
>>>>>>>> 0m
>>>>>>>> ic
>>>>>>>> rosoft.com%7c4d49ae546dba4d29b7be08d2f3589ee1%7c72f988bf86f141a
>>>>>>>> f9
>>>>>>>> 1a
>>>>>>>> b2
>>>>>>>> d7cd011db47%7c1&sdata=FjmfM%2fnPbWB%2fMsUU8uDzAUo3aPu3zOELVsJO%
>>>>>>>> 2f
>>>>>>>> sU
>>>>>>>> Iq
>>>>>>>> 9E%3d
>>>>>>>>
>>>>>>>> Webrev:
>>>>>>>> https://na01.safelinks.protection.outlook.com/?url=http:%2f%2fcr.
>>>>>>>> op
>>>>>>>> en
>>>>>>>> jdk.java.net%2f~kshoop%2f8124977%2f&data=01%7C01%7Cvlashch%40mi
>>>>>>>> cr
>>>>>>>> os
>>>>>>>> of
>>>>>>>> t.com%7C4d49ae546dba4d29b7be08d2f3589ee1%7C72f988bf86f141af91ab
>>>>>>>> 2d
>>>>>>>> 7c
>>>>>>>> d0
>>>>>>>> 11db47%7C1&sdata=RAA%2b5aIzKtrk5X85oLXKlPzbpSk%2bgJZRI%2b0QSI11
>>>>>>>> B0
>>>>>>>> M%
>>>>>>>> 3d
>>>>>>>>
>>>>>>>> This webrev intends to address interaction between Windows 
>>>>>>>> console and java apps.
>>>>>>>>
>>>>>>>> Two switches were added that change the behavior of the launcher.
>>>>>>>> The defaults do not change the launcher behavior.
>>>>>>>>
>>>>>>>>        -Dwindows.UnicodeConsole=true - switches on Unicode 
>>>>>>>> support in the Windows console. This optional switch causes the 
>>>>>>>> launcher to call GetCommandLineW() and parse the arguments in 
>>>>>>>> unicode. It also modifies how the codepage for console output is selected.
>>>>>>>>
>>>>>>>>        -Dfile.encoding.unicode="UTF-8" - identifies Unicode 
>>>>>>>> charset to use; If not specified, UTF-8 is used by default.
>>>>>>>> Ignored when windows.UnicodeConsole is not set to true. When 
>>>>>>>> the first switch is used, this optional switch allows the 
>>>>>>>> codepage for console output to be controlled.
>>>>>>>>
>>>>>>>> I would like to get feedback on the approach here and any 
>>>>>>>> additional work that is required solve these particular Unicode 
>>>>>>>> issues on Windows.
>>>>>>>>
>>>>>>>> Kirk
>>>


From scolebourne at joda.org  Fri Feb 26 17:09:42 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Fri, 26 Feb 2016 17:09:42 +0000
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: <56D06876.4020000@Oracle.com>
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
	<56D05E0E.6030003@Oracle.com>
	
	<56D06876.4020000@Oracle.com>
Message-ID: 

On 26 February 2016 at 15:00, Roger Riggs  wrote:
> Hi Stephen,
>
> It does not seem natural to me with a pattern of HHMM for it to parse more
> than 4 digits.
> I can see lenient modifying the behavior as it it were HHmm, but there is no
> indication in the pattern
> that seconds would be considered.  How it would it be implied from the spec?

The spec is being expanded to define what happens. Previously it
didn't define it at all, and would throw an error.

Lenient parsing typically accepts much more than the strict parsing.

When parsing numbers, you may set the SignStyle to NEVER, but the sign
will still be parsed in lenient mode

When parsing text, you may select the short output format, but any
length of text will be parsed in lenient mode.

As such, it is very much in line with the behavour of the API to parse
a much broader format than the one requested in lenient mode. (None of
this affects strict mode).

Stephen


> In the original issue, appendOffsetId is defined as using the +HH:MM:ss
> pattern and
> specific to ISO the MM should be allowed to be optional.   There is no
> question of parsing
> extra digits not included in the requested pattern.
>
> Separately, this is specifying the new lenient behavior of
> appendOffset(pattern, noffsetText).
> In that case, I don't think it will be understood that patterns 'shorter'
> than the input will
> gobble up extra digits and ':'s.
>
> Roger
>
>
>
>
>
> On 2/26/2016 9:42 AM, Stephen Colebourne wrote:
>
> Lenient can be however lenient we define it to be. Allowing minutes
> and seconds to be parsed when not specified in the pattern is the key
> part of the change. Whether the parser copes with both colons and
> no-colons is the choice at hand here. It seems to me that since the
> parser can easily handle figuring out whether the colon is present or
> not, we should just allow the parser to be fully lenient.
>
> Stephen
>
>
> On 26 February 2016 at 14:15, Roger Riggs  wrote:
>
> HI Stephen,
>
> How lenient is lenient supposed to be? Looking at the offset test cases, it
> seems to allow  minutes
> and seconds digits to be parsed even if the pattern did not include them.
>
> + @DataProvider(name="lenientOffsetParseData")
> + Object[][] data_lenient_offset_parse() {
> + return new Object[][] {
> + {"+HH", "+01", 3600},
> + {"+HH", "+0101", 3660},
> + {"+HH", "+010101", 3661},
> + {"+HH", "+01", 3600},
> + {"+HH", "+01:01", 3660},
> + {"+HH", "+01:01:01", 3661},
>
> Thanks, Roger
>
>
>
> On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
>
> I don't think this is quite right.
>
> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>     parseType = 10;
> }
>
> This code will *always* select type 10 (colons) if a colon is found at
> position+3. Whereas the spec now says that it should only do this if
> the pattern is "HH". For other patterns, the colon/no-colon choice is
> defined to be based on the pattern.
>
> That said, I'm thinking it is better to make the spec more lenient to
> match the behaviour as implemented:
>
>
> When parsing in lenient mode, only the hours are mandatory - minutes
> and seconds are optional. If the character after the hour digits is a
> colon
> then the parser will parse using the pattern "HH:mm:ss", otherwise the
> parser will parse using the pattern "HHmmss".
>
>
> Additional TCKDateTimeFormatterBuilder tests will be needed to
> demonstrate the above. There should also be a test for data following
> the lenient parse. The following should all succeed:
>
>
> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
> "+01:00Europe/London"
> "+0100Europe/London"
>
>
> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
> "+01:Europe/London"
>
> Note this special case, where the colon affects the parse type, but is
> not ultimately part of the offset, thus it is left to match the
> appendLiteral(":")
>
> You may want to think of some additional nasty edge cases!
>
> Stephen
>
> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>
> Hi all,
> Please see the updated webrev
> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>
> Thanks and Regards,
> Nadeesh
>
> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>
> Thanks for the changes.
>
> In `DateTimeFormatter`, the code should be
>
> .parseLenient()
> .appendOffsetId()
> .parseStrict()
>
> and the same in the other case. This ensures that existing callers who
> then embed the formatter in another formatter (like the
> ZONED_DATE_TIME constant) are unaffected.
>
>
> The logic for lenient parsing does not look right as it only handles
> types 5 and 6. This table shows the mappings needed:
>
> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
> "+HHmm",  -> "+HHmmss",
> "+HH:mm",  -> "+HH:mm:ss",
> "+HHMM",  -> "+HHmmss",
> "+HH:MM",  -> "+HH:mm:ss",
> "+HHMMss",  -> "+HHmmss",
> "+HH:MM:ss",  -> "+HH:mm:ss",
> "+HHMMSS",  -> "+HHmmss",
> "+HH:MM:SS",  -> "+HH:mm:ss",
> "+HHmmss",
> "+HH:mm:ss",
>
> Note that the "+HH" pattern is a special case, as we don't know
> whether to use the colon or non-colon pattern. Whether to require
> colon or not is based on whether the next character after the HH is a
> colon or not.
>
> Proposed appendOffsetId() Javadoc:
>
> * Appends the zone offset, such as '+01:00', to the formatter.
> * 
      
> * This appends an instruction to format/parse the offset ID to the
> builder.
> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
> * See {@link #appendOffset(String, String)} for details on formatting
> and parsing.
>
> Proposed appendOffset(String, String) Javadoc:
>
> * During parsing, the offset...
>
> changed to:
>
> * When parsing in strict mode, the input must contain the mandatory
> and optional elements are defined by the specified pattern.
> * If the offset cannot be parsed then an exception is thrown unless
> the section of the formatter is optional.
> * 
      
> * When parsing in lenient mode, only the hours are mandatory - minutes
> and seconds are optional.
> * The colons are required if the specified pattern contains a colon.
> * If the specified pattern is "+HH", the presence of colons is
> determined by whether the character after the hour digits is a colon
> or not.
> * If the offset cannot be parsed then an exception is thrown unless
> the section of the formatter is optional.
>
> thanks and sorry for delay
> Stephen
>
>
>
> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>
> Hi all,
>
> Please review a fix for
>
> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>
> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>
> --
> Thanks and Regards,
> Nadeesh TV
>
> --
> Thanks and Regards,
> Nadeesh TV
>
>


From Roger.Riggs at Oracle.com  Fri Feb 26 17:38:45 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 26 Feb 2016 12:38:45 -0500
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: 
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
	<56D05E0E.6030003@Oracle.com>
	
	<56D06876.4020000@Oracle.com>
	
Message-ID: <56D08DA5.5030700@Oracle.com>

Hi Stephen,

Even in lenient mode the parser needs to stick to the fields provided in 
the pattern.
If the caller intends to parse seconds, the pattern should include seconds.
Otherwise the caller has not been able to specify their intent.
That's consistent with lenient mode used in the other fields.
Otherwise, the pattern is irrelevant except for whether it contains a 
":" and makes
the spec nearly useless.

Roger


On 2/26/2016 12:09 PM, Stephen Colebourne wrote:
> On 26 February 2016 at 15:00, Roger Riggs  wrote:
>> Hi Stephen,
>>
>> It does not seem natural to me with a pattern of HHMM for it to parse more
>> than 4 digits.
>> I can see lenient modifying the behavior as it it were HHmm, but there is no
>> indication in the pattern
>> that seconds would be considered.  How it would it be implied from the spec?
> The spec is being expanded to define what happens. Previously it
> didn't define it at all, and would throw an error.
>
> Lenient parsing typically accepts much more than the strict parsing.
>
> When parsing numbers, you may set the SignStyle to NEVER, but the sign
> will still be parsed in lenient mode
>
> When parsing text, you may select the short output format, but any
> length of text will be parsed in lenient mode.
>
> As such, it is very much in line with the behavour of the API to parse
> a much broader format than the one requested in lenient mode. (None of
> this affects strict mode).
>
> Stephen
>
>
>> In the original issue, appendOffsetId is defined as using the +HH:MM:ss
>> pattern and
>> specific to ISO the MM should be allowed to be optional.   There is no
>> question of parsing
>> extra digits not included in the requested pattern.
>>
>> Separately, this is specifying the new lenient behavior of
>> appendOffset(pattern, noffsetText).
>> In that case, I don't think it will be understood that patterns 'shorter'
>> than the input will
>> gobble up extra digits and ':'s.
>>
>> Roger
>>
>>
>>
>>
>>
>> On 2/26/2016 9:42 AM, Stephen Colebourne wrote:
>>
>> Lenient can be however lenient we define it to be. Allowing minutes
>> and seconds to be parsed when not specified in the pattern is the key
>> part of the change. Whether the parser copes with both colons and
>> no-colons is the choice at hand here. It seems to me that since the
>> parser can easily handle figuring out whether the colon is present or
>> not, we should just allow the parser to be fully lenient.
>>
>> Stephen
>>
>>
>> On 26 February 2016 at 14:15, Roger Riggs  wrote:
>>
>> HI Stephen,
>>
>> How lenient is lenient supposed to be? Looking at the offset test cases, it
>> seems to allow  minutes
>> and seconds digits to be parsed even if the pattern did not include them.
>>
>> + @DataProvider(name="lenientOffsetParseData")
>> + Object[][] data_lenient_offset_parse() {
>> + return new Object[][] {
>> + {"+HH", "+01", 3600},
>> + {"+HH", "+0101", 3660},
>> + {"+HH", "+010101", 3661},
>> + {"+HH", "+01", 3600},
>> + {"+HH", "+01:01", 3660},
>> + {"+HH", "+01:01:01", 3661},
>>
>> Thanks, Roger
>>
>>
>>
>> On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
>>
>> I don't think this is quite right.
>>
>> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>>      parseType = 10;
>> }
>>
>> This code will *always* select type 10 (colons) if a colon is found at
>> position+3. Whereas the spec now says that it should only do this if
>> the pattern is "HH". For other patterns, the colon/no-colon choice is
>> defined to be based on the pattern.
>>
>> That said, I'm thinking it is better to make the spec more lenient to
>> match the behaviour as implemented:
>>
>>
>> When parsing in lenient mode, only the hours are mandatory - minutes
>> and seconds are optional. If the character after the hour digits is a
>> colon
>> then the parser will parse using the pattern "HH:mm:ss", otherwise the
>> parser will parse using the pattern "HHmmss".
>>
>>
>> Additional TCKDateTimeFormatterBuilder tests will be needed to
>> demonstrate the above. There should also be a test for data following
>> the lenient parse. The following should all succeed:
>>
>>
>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
>> "+01:00Europe/London"
>> "+0100Europe/London"
>>
>>
>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
>> "+01:Europe/London"
>>
>> Note this special case, where the colon affects the parse type, but is
>> not ultimately part of the offset, thus it is left to match the
>> appendLiteral(":")
>>
>> You may want to think of some additional nasty edge cases!
>>
>> Stephen
>>
>> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>>
>> Hi all,
>> Please see the updated webrev
>> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>>
>> Thanks and Regards,
>> Nadeesh
>>
>> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>
>> Thanks for the changes.
>>
>> In `DateTimeFormatter`, the code should be
>>
>> .parseLenient()
>> .appendOffsetId()
>> .parseStrict()
>>
>> and the same in the other case. This ensures that existing callers who
>> then embed the formatter in another formatter (like the
>> ZONED_DATE_TIME constant) are unaffected.
>>
>>
>> The logic for lenient parsing does not look right as it only handles
>> types 5 and 6. This table shows the mappings needed:
>>
>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>> "+HHmm",  -> "+HHmmss",
>> "+HH:mm",  -> "+HH:mm:ss",
>> "+HHMM",  -> "+HHmmss",
>> "+HH:MM",  -> "+HH:mm:ss",
>> "+HHMMss",  -> "+HHmmss",
>> "+HH:MM:ss",  -> "+HH:mm:ss",
>> "+HHMMSS",  -> "+HHmmss",
>> "+HH:MM:SS",  -> "+HH:mm:ss",
>> "+HHmmss",
>> "+HH:mm:ss",
>>
>> Note that the "+HH" pattern is a special case, as we don't know
>> whether to use the colon or non-colon pattern. Whether to require
>> colon or not is based on whether the next character after the HH is a
>> colon or not.
>>
>> Proposed appendOffsetId() Javadoc:
>>
>> * Appends the zone offset, such as '+01:00', to the formatter.
>> * 
      
>> * This appends an instruction to format/parse the offset ID to the
>> builder.
>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>> * See {@link #appendOffset(String, String)} for details on formatting
>> and parsing.
>>
>> Proposed appendOffset(String, String) Javadoc:
>>
>> * During parsing, the offset...
>>
>> changed to:
>>
>> * When parsing in strict mode, the input must contain the mandatory
>> and optional elements are defined by the specified pattern.
>> * If the offset cannot be parsed then an exception is thrown unless
>> the section of the formatter is optional.
>> * 
      
>> * When parsing in lenient mode, only the hours are mandatory - minutes
>> and seconds are optional.
>> * The colons are required if the specified pattern contains a colon.
>> * If the specified pattern is "+HH", the presence of colons is
>> determined by whether the character after the hour digits is a colon
>> or not.
>> * If the offset cannot be parsed then an exception is thrown unless
>> the section of the formatter is optional.
>>
>> thanks and sorry for delay
>> Stephen
>>
>>
>>
>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>
>> Hi all,
>>
>> Please review a fix for
>>
>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>
>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>
>> --
>> Thanks and Regards,
>> Nadeesh TV
>>
>> --
>> Thanks and Regards,
>> Nadeesh TV
>>
>>



From scolebourne at joda.org  Fri Feb 26 18:10:30 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Fri, 26 Feb 2016 18:10:30 +0000
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: <56D08DA5.5030700@Oracle.com>
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
	<56D05E0E.6030003@Oracle.com>
	
	<56D06876.4020000@Oracle.com>
	
	<56D08DA5.5030700@Oracle.com>
Message-ID: 

It is important to also consider the case where the user wants to
format using HH:MM but parse seconds if they are provided.

As I said above, this is no different to SignStyle, where the user
requests something specific on format, but accepts anything on input.

The pattern is still used for formatting and strict parsing under
these changes. It is effectively ignored in lenient parsing (which is
the very definition of leniency).

Another way to look at it:

using a pattern of HH:MM and strict:
+02 - disallowed
+02:00 - allowed
+02:00:00 - disallowed

using a pattern of HH:mm and strict:
+02 - allowed
+02:00 - allowed
+02:00:00 - disallowed

using any pattern and lenient:
+02 - allowed
+02:00 - allowed
+02:00:00 - allowed

This covers pretty much anything a user needs when parsing.

Stephen


On 26 February 2016 at 17:38, Roger Riggs  wrote:
> Hi Stephen,
>
> Even in lenient mode the parser needs to stick to the fields provided in the
> pattern.
> If the caller intends to parse seconds, the pattern should include seconds.
> Otherwise the caller has not been able to specify their intent.
> That's consistent with lenient mode used in the other fields.
> Otherwise, the pattern is irrelevant except for whether it contains a ":"
> and makes
> the spec nearly useless.
>
> Roger
>
>
>
> On 2/26/2016 12:09 PM, Stephen Colebourne wrote:
>>
>> On 26 February 2016 at 15:00, Roger Riggs  wrote:
>>>
>>> Hi Stephen,
>>>
>>> It does not seem natural to me with a pattern of HHMM for it to parse
>>> more
>>> than 4 digits.
>>> I can see lenient modifying the behavior as it it were HHmm, but there is
>>> no
>>> indication in the pattern
>>> that seconds would be considered.  How it would it be implied from the
>>> spec?
>>
>> The spec is being expanded to define what happens. Previously it
>> didn't define it at all, and would throw an error.
>>
>> Lenient parsing typically accepts much more than the strict parsing.
>>
>> When parsing numbers, you may set the SignStyle to NEVER, but the sign
>> will still be parsed in lenient mode
>>
>> When parsing text, you may select the short output format, but any
>> length of text will be parsed in lenient mode.
>>
>> As such, it is very much in line with the behavour of the API to parse
>> a much broader format than the one requested in lenient mode. (None of
>> this affects strict mode).
>>
>> Stephen
>>
>>
>>> In the original issue, appendOffsetId is defined as using the +HH:MM:ss
>>> pattern and
>>> specific to ISO the MM should be allowed to be optional.   There is no
>>> question of parsing
>>> extra digits not included in the requested pattern.
>>>
>>> Separately, this is specifying the new lenient behavior of
>>> appendOffset(pattern, noffsetText).
>>> In that case, I don't think it will be understood that patterns 'shorter'
>>> than the input will
>>> gobble up extra digits and ':'s.
>>>
>>> Roger
>>>
>>>
>>>
>>>
>>>
>>> On 2/26/2016 9:42 AM, Stephen Colebourne wrote:
>>>
>>> Lenient can be however lenient we define it to be. Allowing minutes
>>> and seconds to be parsed when not specified in the pattern is the key
>>> part of the change. Whether the parser copes with both colons and
>>> no-colons is the choice at hand here. It seems to me that since the
>>> parser can easily handle figuring out whether the colon is present or
>>> not, we should just allow the parser to be fully lenient.
>>>
>>> Stephen
>>>
>>>
>>> On 26 February 2016 at 14:15, Roger Riggs  wrote:
>>>
>>> HI Stephen,
>>>
>>> How lenient is lenient supposed to be? Looking at the offset test cases,
>>> it
>>> seems to allow  minutes
>>> and seconds digits to be parsed even if the pattern did not include them.
>>>
>>> + @DataProvider(name="lenientOffsetParseData")
>>> + Object[][] data_lenient_offset_parse() {
>>> + return new Object[][] {
>>> + {"+HH", "+01", 3600},
>>> + {"+HH", "+0101", 3660},
>>> + {"+HH", "+010101", 3661},
>>> + {"+HH", "+01", 3600},
>>> + {"+HH", "+01:01", 3660},
>>> + {"+HH", "+01:01:01", 3661},
>>>
>>> Thanks, Roger
>>>
>>>
>>>
>>> On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
>>>
>>> I don't think this is quite right.
>>>
>>> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>>>      parseType = 10;
>>> }
>>>
>>> This code will *always* select type 10 (colons) if a colon is found at
>>> position+3. Whereas the spec now says that it should only do this if
>>> the pattern is "HH". For other patterns, the colon/no-colon choice is
>>> defined to be based on the pattern.
>>>
>>> That said, I'm thinking it is better to make the spec more lenient to
>>> match the behaviour as implemented:
>>>
>>>
>>> When parsing in lenient mode, only the hours are mandatory - minutes
>>> and seconds are optional. If the character after the hour digits is a
>>> colon
>>> then the parser will parse using the pattern "HH:mm:ss", otherwise the
>>> parser will parse using the pattern "HHmmss".
>>>
>>>
>>> Additional TCKDateTimeFormatterBuilder tests will be needed to
>>> demonstrate the above. There should also be a test for data following
>>> the lenient parse. The following should all succeed:
>>>
>>>
>>>
>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
>>> "+01:00Europe/London"
>>> "+0100Europe/London"
>>>
>>>
>>>
>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
>>> "+01:Europe/London"
>>>
>>> Note this special case, where the colon affects the parse type, but is
>>> not ultimately part of the offset, thus it is left to match the
>>> appendLiteral(":")
>>>
>>> You may want to think of some additional nasty edge cases!
>>>
>>> Stephen
>>>
>>> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>>>
>>> Hi all,
>>> Please see the updated webrev
>>> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>>>
>>> Thanks and Regards,
>>> Nadeesh
>>>
>>> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>>
>>> Thanks for the changes.
>>>
>>> In `DateTimeFormatter`, the code should be
>>>
>>> .parseLenient()
>>> .appendOffsetId()
>>> .parseStrict()
>>>
>>> and the same in the other case. This ensures that existing callers who
>>> then embed the formatter in another formatter (like the
>>> ZONED_DATE_TIME constant) are unaffected.
>>>
>>>
>>> The logic for lenient parsing does not look right as it only handles
>>> types 5 and 6. This table shows the mappings needed:
>>>
>>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>>> "+HHmm",  -> "+HHmmss",
>>> "+HH:mm",  -> "+HH:mm:ss",
>>> "+HHMM",  -> "+HHmmss",
>>> "+HH:MM",  -> "+HH:mm:ss",
>>> "+HHMMss",  -> "+HHmmss",
>>> "+HH:MM:ss",  -> "+HH:mm:ss",
>>> "+HHMMSS",  -> "+HHmmss",
>>> "+HH:MM:SS",  -> "+HH:mm:ss",
>>> "+HHmmss",
>>> "+HH:mm:ss",
>>>
>>> Note that the "+HH" pattern is a special case, as we don't know
>>> whether to use the colon or non-colon pattern. Whether to require
>>> colon or not is based on whether the next character after the HH is a
>>> colon or not.
>>>
>>> Proposed appendOffsetId() Javadoc:
>>>
>>> * Appends the zone offset, such as '+01:00', to the formatter.
>>> * 
      
>>> * This appends an instruction to format/parse the offset ID to the
>>> builder.
>>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>>> * See {@link #appendOffset(String, String)} for details on formatting
>>> and parsing.
>>>
>>> Proposed appendOffset(String, String) Javadoc:
>>>
>>> * During parsing, the offset...
>>>
>>> changed to:
>>>
>>> * When parsing in strict mode, the input must contain the mandatory
>>> and optional elements are defined by the specified pattern.
>>> * If the offset cannot be parsed then an exception is thrown unless
>>> the section of the formatter is optional.
>>> * 
      
>>> * When parsing in lenient mode, only the hours are mandatory - minutes
>>> and seconds are optional.
>>> * The colons are required if the specified pattern contains a colon.
>>> * If the specified pattern is "+HH", the presence of colons is
>>> determined by whether the character after the hour digits is a colon
>>> or not.
>>> * If the offset cannot be parsed then an exception is thrown unless
>>> the section of the formatter is optional.
>>>
>>> thanks and sorry for delay
>>> Stephen
>>>
>>>
>>>
>>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>>
>>> Hi all,
>>>
>>> Please review a fix for
>>>
>>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>>
>>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>>
>>> --
>>> Thanks and Regards,
>>> Nadeesh TV
>>>
>>> --
>>> Thanks and Regards,
>>> Nadeesh TV
>>>
>>>
>


From volker.simonis at gmail.com  Fri Feb 26 19:00:16 2016
From: volker.simonis at gmail.com (Volker Simonis)
Date: Fri, 26 Feb 2016 20:00:16 +0100
Subject: RFR(s): 8148425: strerror() function is not thread-safe
In-Reply-To: 
References: 
Message-ID: 

Hi Thomas,

it's good that somebody finally addresses this long standing issue :)

However I wonder if it would be possible to align this effort with the
core libraries group (CC'ed). They already fix this issue with:

8133249: Occasional SIGSEGV: non thread-safe use of strerr in getLastErrorString
https://bugs.openjdk.java.net/browse/JDK-8133249

I would be nice if we could use the same version in hotspot and the
core libraries.

Regards,
Volker


On Fri, Feb 26, 2016 at 5:05 PM, Thomas St?fe  wrote:
> Hi,
>
> please take a look at this proposed fix:
>
> Bug: https://bugs.openjdk.java.net/browse/JDK-8148425
> Webrev:
> http://cr.openjdk.java.net/~stuefe/webrevs/8141425-strerror-replacement/webrev.00/webrev/
>
> This adds a replacement function os::strerror() as a drop-in for
> strerror(), which has a number of issues.
>
> strerror() is unsafe to use and may cause (and has caused) crashes in
> multithreaded scenarios. It is also not ideal for log files because of the
> implicit localization of the error messages.
>
> For details please see the discussion under the bug report.
>
> Please note that I did not yet change any call sites, although all call
> sites in the os namespace should already use the new function. I wanted to
> see whether there would be any general objections.
>
> Kind Regards, Thomas


From Roger.Riggs at Oracle.com  Fri Feb 26 21:39:01 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Fri, 26 Feb 2016 16:39:01 -0500
Subject: RFR: JDK-8149925 We don't need jdk.internal.ref.Cleaner any more
In-Reply-To: <56CEC6A5.3070202@gmail.com>
References: <56B72242.7050102@gmail.com>
	
	<56B7C328.3060800@gmail.com> <56B83553.3020202@oracle.com>
	
	<56B874DA.80001@gmail.com>
	<56B9EB17.7020303@oracle.com> <56C1E765.7080603@oracle.com>
	<56C1FE37.9010507@oracle.com>
	<015201d16813$333650c0$99a2f240$@apache.org>
	
	
	<56C34B1B.8050001@gmail.com>
	
	<56C43817.7060805@gmail.com>
	<7BA56B2F-C1C6-4EAF-B900-A825C6B602EF@oracle.com>
	<56CA080F.6010308@gmail.com>
	
	<56CB83FF.4010808@Oracle.com> <56CC8A4A.9080303@gmail.com>
	
	<56CEAC28.80802@gmail.com>
	<56CEB49A.4090000@oracle.com> <56CEC6A5.3070202@gmail.com>
Message-ID: <56D0C5F5.7060509@Oracle.com>

Hi Peter,

I think this cleans up all the points raised earlier.
The optimization for enqueuing from the reference queue seems ok to me 
and should be
more efficient than the previous implementation but I think Mandy or 
Alan should look at it also.

Thanks, Roger


On 2/25/2016 4:17 AM, Peter Levart wrote:
> Hi Alan,
>
> On 02/25/2016 09:00 AM, Alan Bateman wrote:
>>
>>
>> On 25/02/2016 07:24, Peter Levart wrote:
>>> :
>>>
>>> I kept the public boolean Cleaner.cleanNextPending() method which 
>>> now only deals with enqueued Cleanable(s). I think this method might 
>>> still be beneficial for public use in situations where cleanup 
>>> actions take relatively long time to execute so that the rate of 
>>> cleanup falls behind the rate of registration of new cleanup actions.
>> I think we need also need to look at the option where this is not 
>> public. I have concerns that it is exposing implementation to some 
>> extent and that may become an attractive nuisance in the future. This 
>> shouldn't be an issue for the NIO buffer usage, we can keep the usage 
>> via the shared secrets mechanism. I think this is what Mandy is 
>> suggesting too.
>>
>> -Alan.
>
> Sure, no problem. Here's a variant that keeps the 
> Cleaner.cleanNextPending() method private and exposed via 
> SharedSecrets to nio Bits but is otherwise equivalent to webrev.06:
>
> http://cr.openjdk.java.net/~plevart/jdk9-dev/removeInternalCleaner/webrev.06priv/ 
>
>
> Regards, Peter
>



From david.holmes at oracle.com  Sat Feb 27 01:20:58 2016
From: david.holmes at oracle.com (David Holmes)
Date: Sat, 27 Feb 2016 11:20:58 +1000
Subject: RFR(s): 8148425: strerror() function is not thread-safe
In-Reply-To: 
References: 
	
Message-ID: <56D0F9FA.6030004@oracle.com>

On 27/02/2016 5:00 AM, Volker Simonis wrote:
> Hi Thomas,
>
> it's good that somebody finally addresses this long standing issue :)
>
> However I wonder if it would be possible to align this effort with the
> core libraries group (CC'ed). They already fix this issue with:
>
> 8133249: Occasional SIGSEGV: non thread-safe use of strerr in getLastErrorString
> https://bugs.openjdk.java.net/browse/JDK-8133249
>
> I would be nice if we could use the same version in hotspot and the
> core libraries.

I don't find this:

+#if defined(LINUX) && (defined(_GNU_SOURCE) || \
+         (defined(_POSIX_C_SOURCE) && _POSIX_C_SOURCE < 200112L \
+             && defined(_XOPEN_SOURCE) && _XOPEN_SOURCE < 600))
+extern int __xpg_strerror_r(int, char *, size_t);
+#define strerror_r(a, b, c) __xpg_strerror_r((a), (b), (c))
+#endif

particularly appealing at all - you can't even decode it without having 
a POSIX and XOpen version history in front of you :(

And a version that requires a buffer to be passed in is problematic in a 
number of usage contexts in hotspot - see the comments in the bug 
report. A simplified version that converts symbolic error values to 
their string equivalent is much more appealing - albeit fixing an issue 
that "should" be handled at the library level.

David
-----

> Regards,
> Volker
>
>
> On Fri, Feb 26, 2016 at 5:05 PM, Thomas St?fe  wrote:
>> Hi,
>>
>> please take a look at this proposed fix:
>>
>> Bug: https://bugs.openjdk.java.net/browse/JDK-8148425
>> Webrev:
>> http://cr.openjdk.java.net/~stuefe/webrevs/8141425-strerror-replacement/webrev.00/webrev/
>>
>> This adds a replacement function os::strerror() as a drop-in for
>> strerror(), which has a number of issues.
>>
>> strerror() is unsafe to use and may cause (and has caused) crashes in
>> multithreaded scenarios. It is also not ideal for log files because of the
>> implicit localization of the error messages.
>>
>> For details please see the discussion under the bug report.
>>
>> Please note that I did not yet change any call sites, although all call
>> sites in the os namespace should already use the new function. I wanted to
>> see whether there would be any general objections.
>>
>> Kind Regards, Thomas


From stuart.marks at oracle.com  Sat Feb 27 02:05:36 2016
From: stuart.marks at oracle.com (Stuart Marks)
Date: Fri, 26 Feb 2016 18:05:36 -0800
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
	Quicksort
In-Reply-To: 
References: <4AAA27EA.8090902@Sun.COM>
	
Message-ID: <56D10470.9090509@oracle.com>

Wow, is this a reply to a nearly seven-year-old email? [1]

I don't know if Vladimir Yaroslavskiy is still on core-libs-dev. I dug through 
tthe archives and found that he had posted a couple messages somewhat later [2] 
[3] using different email addresses:

   Vladimir Iaroslavski 
   Vladimir Iaroslavski 

You might try to contact him at one of these addresses. Note however that the 
more recent one is still over five years old.

He's also on LinkedIn. His profile says he works for Oracle, but as far as I can 
see he no longer does.

Good luck,

s'marks


[1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-September/002630.html

[2] http://mail.openjdk.java.net/pipermail/core-libs-dev/2010-April/004133.html

[3] http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-February/005821.html



On 2/23/16 6:05 PM, Jason C. McDonald wrote:
> I think this is the best place to contact the original authors.
>
> The link to Vladimir Yaroslavskiy's original whitepaper describing the
> algorithm and its proofs was, unfortunately, broken. Using Archive.org's
> Wayback Machine, I was able to get the last known revision. I reformatted
> the document in LibreOffice for ease of reading, and fixed some minor
> grammatical errors.
>
> I also implemented the algorithm in an open-source (MIT License) C++
> library, which I hope to release in the coming few months.
>
> In order to make this algorithm and its proofs more easily accessible, I
> would like to make the revised whitepaper publicly and freely available from
> my own web servers, but I wanted to check with the original author(s) first.
> Furthermore, I wanted to find out if there have been any revisions since the
> 22 September 2009 version of the whitepaper I acquired.
>
> Thank you in advance!
>


From thomas.stuefe at gmail.com  Sat Feb 27 07:15:40 2016
From: thomas.stuefe at gmail.com (=?UTF-8?Q?Thomas_St=C3=BCfe?=)
Date: Sat, 27 Feb 2016 08:15:40 +0100
Subject: RFR(s): 8148425: strerror() function is not thread-safe
In-Reply-To: 
References: 
	
Message-ID: 

Hi Volker,

On Fri, Feb 26, 2016 at 8:00 PM, Volker Simonis 
wrote:

> Hi Thomas,
>
> it's good that somebody finally addresses this long standing issue :)
>
>
 Thanks :)


> However I wonder if it would be possible to align this effort with the
> core libraries group (CC'ed). They already fix this issue with:
>
> 8133249: Occasional SIGSEGV: non thread-safe use of strerr in
> getLastErrorString
> https://bugs.openjdk.java.net/browse/JDK-8133249
>
> I would be nice if we could use the same version in hotspot and the
> core libraries.
>
>
David already gave a good answer to this. For details see the comments in
the bug report.

In addition, there is the localization strerror(_s) provides: it is useless
and actually counterproductive in log files intended for the VM developers
themselves; but if those error messages get stuffed into Java Exception
texts and presented to the user, they might make more sense.

Kind Regards, Thomas


> Regards,
> Volker
>
>
> On Fri, Feb 26, 2016 at 5:05 PM, Thomas St?fe 
> wrote:
> > Hi,
> >
> > please take a look at this proposed fix:
> >
> > Bug: https://bugs.openjdk.java.net/browse/JDK-8148425
> > Webrev:
> >
> http://cr.openjdk.java.net/~stuefe/webrevs/8141425-strerror-replacement/webrev.00/webrev/
> >
> > This adds a replacement function os::strerror() as a drop-in for
> > strerror(), which has a number of issues.
> >
> > strerror() is unsafe to use and may cause (and has caused) crashes in
> > multithreaded scenarios. It is also not ideal for log files because of
> the
> > implicit localization of the error messages.
> >
> > For details please see the discussion under the bug report.
> >
> > Please note that I did not yet change any call sites, although all call
> > sites in the os namespace should already use the new function. I wanted
> to
> > see whether there would be any general objections.
> >
> > Kind Regards, Thomas
>


From chris.hegarty at oracle.com  Mon Feb 29 09:04:54 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Mon, 29 Feb 2016 09:04:54 +0000
Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package
In-Reply-To: 
References: 
	<5b1a5288-3c75-4437-aa78-b63169615db7@default>
	
	<66f6308b-60e1-4874-ae21-25c19f822f26@default>
	<90AE881D-5940-4BFF-91E6-7637AEB7D4EE@oracle.com>
	<431be29a-f350-4790-81bd-13365b505b0e@default>
	
Message-ID: <60EF1BB4-095F-4881-A37D-C508D570F2AF@oracle.com>

Updated webrv:
  http://cr.openjdk.java.net/~chegar/8150162.01/

The *{Major,Minor,...}Version(), and supporting, methods have been removed. As well
as the native methods that support their implementation.  Further, additional, clean up,
on the hotspot side, can be done as a follow up.

-Chris.

Note: test/sun/misc/Version/Version.java is no longer interesting as it asserts the values
returned by the *{Major,Minor,...}Version() methods, which no longer exist.

On 19 Feb 2016, at 19:05, Mandy Chung  wrote:

> 
>> On Feb 19, 2016, at 10:59 AM, Iris Clark  wrote:
>> 
>> Hi, Mandy.
>> 
>>> Before Verona, sun.misc.Version was generated at build time and was intended to set up the system properties about ?java.version?, ?java.runtime.version?, ?java.runtime.name?.  Did Verona change the logic such that this class is no longer needed?
>> 
>> Verona did not change logic for VM initialization, we just updated it to reflect differences in naming.  The class is still needed to handle VM initialization and to print out version information (see lines 67-124 [0]). 
> 
> Thanks for confirming that.
> 
>> 
>> Recommendations for deprecation/removal were in reference to the static methods, *{Major,Minor,...}Version() beginning at new line 127 which appear to simply retrieve portions of the system properties.    
>> 
> 
> Good.  Those methods were added for internal use for HS express era (supporting VM of different versions that is gone for long).  It?s time to remove them.
> 
> Mandy
> 
> 
>> Thanks,
>> Iris
>> 
>> [0] http://cr.openjdk.java.net/~chegar/8150162/jdk/src/java.base/share/classes/java/lang/Version.java.template.html
>> 
>> -----Original Message-----
>> From: Mandy Chung 
>> Sent: Friday, February 19, 2016 10:48 AM
>> To: Iris Clark
>> Cc: Mandy Chung; Chris Hegarty; core-libs-dev
>> Subject: Re: RFR [9] 8150162: Move sun.misc.Version to a truly internal package
>> 
>> 
>>> On Feb 19, 2016, at 9:52 AM, Iris Clark  wrote:
>>> 
>>> Hi, Chris.
>>> 
>>>>> I'm not the code maintainer, but I'd consider deprecating them.
>>> 
>>>> Or just removing them.
>>> 
>>> Agreed.
>> 
>> Before Verona, sun.misc.Version was generated at build time and was intended to set up the system properties about ?java.version?, ?java.runtime.version?, ?java.runtime.name?.  Did Verona change the logic such that this class is no longer needed?
>> 
>> On the other hand, I agree that the static methods getting the version numbers should be removed as they are replaced by the new Version API.
>> 
>> As for the webrev, moving to java.lang as a package-private type is fine.  I suggest to rename it to avoid any confusion with the new Version API such as VersionProps (or something better)?
>> 
>> Mandy
>> 
> 



From paul.sandoz at oracle.com  Mon Feb 29 11:23:59 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 29 Feb 2016 12:23:59 +0100
Subject: RFR(M): 8150635: j.l.i.MethodHandles.loop(...) throws
	IndexOutOfBoundsException
In-Reply-To: 
References: 
Message-ID: <8D2F1F62-A639-432B-8E13-29245E467BBA@oracle.com>


> On 26 Feb 2016, at 14:01, Michael Haupt  wrote:
> 
> Dear all,
> 
> please review this fix.
> Bug: https://bugs.openjdk.java.net/browse/JDK-8150635
> Webrev: http://cr.openjdk.java.net/~mhaupt/8150635/webrev.00
> 

MethodHandles.java
?

4087     private static List> buildCommonSuffix(List init, List step, List pred, List fini, int cpSize) {
4088         final List> empty = new ArrayList<>();
4089         final List nonNullInits = init.stream().filter(Objects::nonNull).collect(Collectors.toList());
4090         if (nonNullInits.isEmpty()) {
4091             final List> longest = Stream.of(step, pred, fini).flatMap(List::stream).filter(Objects::nonNull).
4092                     // take only those that can contribute to a common suffix because they are longer than the prefix
4093                     map(MethodHandle::type).filter(t -> t.parameterCount() > cpSize).map(MethodType::parameterList).
4094                     reduce((p, q) -> p.size() >= q.size() ? p : q).orElse(empty);
4095             return longest.size() == 0 ? empty : longest.subList(cpSize, longest.size());
4096         } else {
4097             return nonNullInits.stream().map(MethodHandle::type).map(MethodType::parameterList).
4098                     reduce((p, q) -> p.size() >= q.size() ? p : q).orElse(empty);
4099         }
4100     }

You can use List.of() (or before that method was added Collections.emptyList()) for the empty list as the returned lists will be unmodifiable (that returned from MethodType.parameterList is).

We also know in the second case that the stream can never be empty so arguably a get is sufficient, but that is just a trivial comment (really we want to assert, and the second best thing there would be a orElseThrow, but it looks odd in this context perhaps).


> * @apiNote Example:
> * 
      {@code
> * // iterative implementation of the factorial function as a loop handle
> * static int one(int k) { return 1; }
> * int inc(int i, int acc, int k) { return i + 1; }
> * int mult(int i, int acc, int k) { return i * acc; }
> * boolean pred(int i, int acc, int k) { return i < k; }
> * int fin(int i, int acc, int k) { return acc; }
> * // assume MH_one, MH_inc, MH_mult, MH_pred, and MH_fin are handles to the above methods
> * // null initializer for counter, should initialize to 0
> * MethodHandle[] counterClause = new MethodHandle[]{null, MH_inc};
> * MethodHandle[] accumulatorClause = new MethodHandle[]{MH_one, MH_mult, MH_pred, MH_fin};
> * MethodHandle loop = MethodHandles.loop(counterClause, accumulatorClause);
> * assertEquals(120, loop.invoke(5));
> * }
      



Do you need to update this example since it is referring to virtual methods? since you need to permute the arguments.


T8139885.java
?

Could we rename that test to say ?LoopCombinatorTest? ?

Paul.



> There were two issues to address that led to IOOBE being thrown.
> 
> 1. Lack of support for deriving the loop signature from given parameter type lists in case no init function is given at all.
> 
> 2. The loop combinator reporting a wrong exception in case a parameter type list is longer than the common parameter sequence. In this case, an error about non-effectively identical parameter type lists should be signalled.
> 
> See the JIRA issue for details and examples.
> 
> Thanks,
> 
> Michael
> 
> --
> 
> 
> Dr. Michael Haupt | Principal Member of Technical Staff
> Phone: +49 331 200 7277 | Fax: +49 331 200 7561
> Oracle Java Platform Group | LangTools Team | Nashorn
> Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany
> 
> ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
> Registergericht: Amtsgericht M?nchen, HRA 95603
> 
> Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
> Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
> Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
> 	Oracle is committed to developing practices and products that help protect the environment
> 



From lance.andersen at oracle.com  Mon Feb 29 11:43:46 2016
From: lance.andersen at oracle.com (Lance Andersen)
Date: Mon, 29 Feb 2016 06:43:46 -0500
Subject: [9] RFR: 8150174: Update JAX-WS RI integration to latest version
	(2.3.0-SNAPSHOT)
In-Reply-To: <56CE33AF.3060504@oracle.com>
References: <56CE33AF.3060504@oracle.com>
Message-ID: <3539B35D-4BB7-41E4-83B2-B01B4125A47A@oracle.com>

Hi Alejsej

This looks fine

Best
Lance
On Feb 24, 2016, at 5:50 PM, Aleksej Efimov  wrote:

> Hi,
> 
> Please, review the bulk update of JAX-WS/B from upstream projects:
> http://cr.openjdk.java.net/~aefimov/jaxws-integrations/8150174/9/00/
> 
> Details (list of fixed issues) can be found in bug report:
> https://bugs.openjdk.java.net/browse/JDK-8150174
> 
> The following test sets were executed over JDK9 with integrated changes:
> jdk_other JTREG tests; JCK9 jaxws tests; JAXWS unit tests;
> 
> Thanks,
> Aleksej



Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
Oracle Java Engineering 
1 Network Drive 
Burlington, MA 01803
Lance.Andersen at oracle.com





From michael.haupt at oracle.com  Mon Feb 29 12:56:41 2016
From: michael.haupt at oracle.com (Michael Haupt)
Date: Mon, 29 Feb 2016 13:56:41 +0100
Subject: RFR(S): 8150825: MethodHandles.tryFinally throws
	IndexOutOfBoundsException for non-conforming parameter lists
Message-ID: 

Dear all,

please review this fix.
Bug: https://bugs.openjdk.java.net/browse/JDK-8150825
Webrev: http://cr.openjdk.java.net/~mhaupt/8150825/webrev.00

The logic for checking conformance of tryFinally arguments did not take a too short target parameter type list into account.

Thanks,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603

Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
 	Oracle is committed to developing practices and products that help protect the environment



From claes.redestad at oracle.com  Mon Feb 29 12:59:24 2016
From: claes.redestad at oracle.com (Claes Redestad)
Date: Mon, 29 Feb 2016 13:59:24 +0100
Subject: RFR(S): 8150825: MethodHandles.tryFinally throws
	IndexOutOfBoundsException for non-conforming parameter lists
In-Reply-To: 
References: 
Message-ID: <56D440AC.3050701@oracle.com>

Looks good to me!

(Slightly off-topic: would it be possible to rename the test to 
something more descriptive, or is such things frowned upon?)

/Claes

On 2016-02-29 13:56, Michael Haupt wrote:
> Dear all,
>
> please review this fix.
> Bug: https://bugs.openjdk.java.net/browse/JDK-8150825
> Webrev: http://cr.openjdk.java.net/~mhaupt/8150825/webrev.00
>
> The logic for checking conformance of tryFinally arguments did not take a too short target parameter type list into account.
>
> Thanks,
>
> Michael
>



From paul.sandoz at oracle.com  Mon Feb 29 13:16:49 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 29 Feb 2016 14:16:49 +0100
Subject: RFR(S): 8150825: MethodHandles.tryFinally throws
	IndexOutOfBoundsException for non-conforming parameter lists
In-Reply-To: <56D440AC.3050701@oracle.com>
References: 
	<56D440AC.3050701@oracle.com>
Message-ID: <4F1F3ABE-3004-46ED-82C0-A10FF2D75713@oracle.com>


> On 29 Feb 2016, at 13:59, Claes Redestad  wrote:
> 
> Looks good to me!
> 
> (Slightly off-topic: would it be possible to rename the test to something more descriptive, or is such things frowned upon?)
> 

I support renaming too. Using the initial integration bug id for a test is likely to become confusing and less relevant over time (e.g. as new issues are added to the test itself).

The test name is important as it flows through the whole test process and when i see ??/invoke/T8139885.java failed? i don?t know really known what that means. If i saw ??/invoke/TryFinallyCombinatorTest.java failed? then i would have more of a clue :-)

My suggestion, as a follow on issue, is to split the test out into separate areas of high-level functionality and each test named according to that functionality.

Paul.


From michael.haupt at oracle.com  Mon Feb 29 13:17:12 2016
From: michael.haupt at oracle.com (Michael Haupt)
Date: Mon, 29 Feb 2016 14:17:12 +0100
Subject: RFR(S): 8150825: MethodHandles.tryFinally throws
	IndexOutOfBoundsException for non-conforming parameter lists
In-Reply-To: <56D440AC.3050701@oracle.com>
References: 
	<56D440AC.3050701@oracle.com>
Message-ID: <504C8D58-8B21-4B84-BE99-D704FEFB2B52@oracle.com>

Hi Claes,

> Am 29.02.2016 um 13:59 schrieb Claes Redestad :
> Looks good to me!

thank you!

> (Slightly off-topic: would it be possible to rename the test to something more descriptive, or is such things frowned upon?)

This being the second Reviewer request, and the naming having nagged me for a while, I'll do it at some point in the near future (probably by piggybacking the renaming onto something else). :-)

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603

Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
 	Oracle is committed to developing practices and products that help protect the environment



From michael.haupt at oracle.com  Mon Feb 29 13:39:57 2016
From: michael.haupt at oracle.com (Michael Haupt)
Date: Mon, 29 Feb 2016 14:39:57 +0100
Subject: RFR(M): 8150635: j.l.i.MethodHandles.loop(...) throws
	IndexOutOfBoundsException
In-Reply-To: <8D2F1F62-A639-432B-8E13-29245E467BBA@oracle.com>
References: 
	<8D2F1F62-A639-432B-8E13-29245E467BBA@oracle.com>
Message-ID: 

Hi Paul,

thank you for your comments. BTW I filed a CCC request in the meantime, as this change also affects the details of when exceptions are thrown. Answers below ...

> Am 29.02.2016 um 12:23 schrieb Paul Sandoz :
> MethodHandles.java
> ?
> 
> ...
> You can use List.of() (or before that method was added Collections.emptyList()) for the empty list as the returned lists will be unmodifiable (that returned from MethodType.parameterList is).

Right, thanks.

> We also know in the second case that the stream can never be empty so arguably a get is sufficient, but that is just a trivial comment (really we want to assert, and the second best thing there would be a orElseThrow, but it looks odd in this context perhaps).

Indeed the list cannot be empty, and the stream will always produce a result, so get() it is. Thanks!

>> * @apiNote Example:
>> * 
      {@code
>> * // iterative implementation of the factorial function as a loop handle
>> * static int one(int k) { return 1; }
>> * int inc(int i, int acc, int k) { return i + 1; }
>> ...
> 
> Do you need to update this example since it is referring to virtual methods? since you need to permute the arguments.

I've updated it to use static methods only, which was the intent, and which is what JavaDocExamplesTest covers. A test for using virtual methods was added anyway.

> T8139885.java
> ?
> 
> Could we rename that test to say ?LoopCombinatorTest? ?

With more and more bugs being covered by this test, that clearly makes sense. :-) I like the suggestion to split the test in several ones you made in http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-February/039138.html - I've filed the RFE here: https://bugs.openjdk.java.net/browse/JDK-8150832

A new webrev with the above changes (save the renaming) is at http://cr.openjdk.java.net/~mhaupt/8150635/webrev.01

Best,

Michael

-- 

 
Dr. Michael Haupt | Principal Member of Technical Staff
Phone: +49 331 200 7277 | Fax: +49 331 200 7561
Oracle Java Platform Group | LangTools Team | Nashorn
Oracle Deutschland B.V. & Co. KG | Schiffbauergasse 14 | 14467 Potsdam, Germany

ORACLE Deutschland B.V. & Co. KG | Hauptverwaltung: Riesstra?e 25, D-80992 M?nchen
Registergericht: Amtsgericht M?nchen, HRA 95603

Komplement?rin: ORACLE Deutschland Verwaltung B.V. | Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Gesch?ftsf?hrer: Alexander van der Ven, Jan Schultheiss, Val Maher
 	Oracle is committed to developing practices and products that help protect the environment



From paul.sandoz at oracle.com  Mon Feb 29 13:46:31 2016
From: paul.sandoz at oracle.com (Paul Sandoz)
Date: Mon, 29 Feb 2016 14:46:31 +0100
Subject: RFR(M): 8150635: j.l.i.MethodHandles.loop(...) throws
	IndexOutOfBoundsException
In-Reply-To: 
References: 
	<8D2F1F62-A639-432B-8E13-29245E467BBA@oracle.com>
	
Message-ID: 


> On 29 Feb 2016, at 14:39, Michael Haupt  wrote:
> 
>>> * @apiNote Example:
>>> * 
      {@code
>>> * // iterative implementation of the factorial function as a loop handle
>>> * static int one(int k) { return 1; }
>>> * int inc(int i, int acc, int k) { return i + 1; }
>>> ...
>> 
>> Do you need to update this example since it is referring to virtual methods? since you need to permute the arguments.
> 
> I've updated it to use static methods only, which was the intent, and which is what JavaDocExamplesTest covers. A test for using virtual methods was added anyway.
> 

Ok.


>> T8139885.java
>> ?
>> 
>> Could we rename that test to say ?LoopCombinatorTest? ?
> 
> With more and more bugs being covered by this test, that clearly makes sense. :-) I like the suggestion to split the test in several ones you made in http://mail.openjdk.java.net/pipermail/core-libs-dev/2016-February/039138.html - I've filed the RFE here: https://bugs.openjdk.java.net/browse/JDK-8150832
> 
> A new webrev with the above changes (save the renaming) is at http://cr.openjdk.java.net/~mhaupt/8150635/webrev.01
> 

+1

Paul.


From rob.mckenna at oracle.com  Mon Feb 29 14:40:14 2016
From: rob.mckenna at oracle.com (Rob McKenna)
Date: Mon, 29 Feb 2016 14:40:14 +0000
Subject: RFR(s): 8148425: strerror() function is not thread-safe
In-Reply-To: <56D0F9FA.6030004@oracle.com>
References: 
	
	<56D0F9FA.6030004@oracle.com>
Message-ID: <20160229144014.GB2814@vimes>

On 27/02/16 11:20, David Holmes wrote:
> On 27/02/2016 5:00 AM, Volker Simonis wrote:
> >Hi Thomas,
> >
> >it's good that somebody finally addresses this long standing issue :)
> >
> >However I wonder if it would be possible to align this effort with the
> >core libraries group (CC'ed). They already fix this issue with:
> >
> >8133249: Occasional SIGSEGV: non thread-safe use of strerr in getLastErrorString
> >https://bugs.openjdk.java.net/browse/JDK-8133249
> >
> >I would be nice if we could use the same version in hotspot and the
> >core libraries.
> 
> I don't find this:
> 
> +#if defined(LINUX) && (defined(_GNU_SOURCE) || \
> +         (defined(_POSIX_C_SOURCE) && _POSIX_C_SOURCE < 200112L \
> +             && defined(_XOPEN_SOURCE) && _XOPEN_SOURCE < 600))
> +extern int __xpg_strerror_r(int, char *, size_t);
> +#define strerror_r(a, b, c) __xpg_strerror_r((a), (b), (c))
> +#endif
> 
> particularly appealing at all - you can't even decode it without having a
> POSIX and XOpen version history in front of you :(
> 
> And a version that requires a buffer to be passed in is problematic in a
> number of usage contexts in hotspot - see the comments in the bug report. A
> simplified version that converts symbolic error values to their string
> equivalent is much more appealing - albeit fixing an issue that "should" be
> handled at the library level.

Agreed, its horrific. Unfortunately any fix for strerror thread safety is effectively a kludge on top of an ugly library situation. This fix seems to be going down the route of reimplementing sys_errlist (which iirc Solaris has removed and Linux has deprecated) I didn't feel confident taking that path with my fix but this may be a case of what works for hotspot may be different to what works for corelibs.

    -Rob

> 
> David
> -----
> 
> >Regards,
> >Volker
> >
> >
> >On Fri, Feb 26, 2016 at 5:05 PM, Thomas St?fe  wrote:
> >>Hi,
> >>
> >>please take a look at this proposed fix:
> >>
> >>Bug: https://bugs.openjdk.java.net/browse/JDK-8148425
> >>Webrev:
> >>http://cr.openjdk.java.net/~stuefe/webrevs/8141425-strerror-replacement/webrev.00/webrev/
> >>
> >>This adds a replacement function os::strerror() as a drop-in for
> >>strerror(), which has a number of issues.
> >>
> >>strerror() is unsafe to use and may cause (and has caused) crashes in
> >>multithreaded scenarios. It is also not ideal for log files because of the
> >>implicit localization of the error messages.
> >>
> >>For details please see the discussion under the bug report.
> >>
> >>Please note that I did not yet change any call sites, although all call
> >>sites in the os namespace should already use the new function. I wanted to
> >>see whether there would be any general objections.
> >>
> >>Kind Regards, Thomas


From chris.hegarty at oracle.com  Mon Feb 29 14:45:34 2016
From: chris.hegarty at oracle.com (Chris Hegarty)
Date: Mon, 29 Feb 2016 14:45:34 +0000
Subject: RFR [9] 8149122: Move sun.misc.URLClassPath and Resource to an
	internal package
Message-ID: <56D4598E.6050503@oracle.com>

sun.misc.URLClassPath, and its accompanying Resource, are not
"Critical APIs", as defined by JEP 260, so they should be moved
out of sun.misc and placed into a more appropriate package where
they can be encapsulated.

http://cr.openjdk.java.net/~chegar/8149122/
https://bugs.openjdk.java.net/browse/JDK-8149122

-Chris.


From Roger.Riggs at Oracle.com  Mon Feb 29 15:52:11 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 29 Feb 2016 10:52:11 -0500
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: 
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
	<56D05E0E.6030003@Oracle.com>
	
	<56D06876.4020000@Oracle.com>
	
	<56D08DA5.5030700@Oracle.com>
	
Message-ID: <56D4692B.1030402@Oracle.com>

Hi Stephen,

As a fix for the original issue[1], not correctly parsing a ISO defined 
offset, the use of lenient
was a convenient implementation technique (hack).  But with the expanded 
definition of lenient,
it will allow many forms of the offset that are not allowed by the ISO 
specification
and should not be accepted forDateTimeFormatter. ISO_OFFSET_DATE_TIME.
In particular, ISO requires the ":" to separate the minutes.
I'm not sure how to correctly fix the original issue with the new 
specification of lenient offset
parsing without introducing some more specific implementation information.


WRT the lenient parsing mode for appendOffset:

I was considering that the subfields of the offset were to be treated 
leniently but it seems
you were treating the entire offset field and text as the unit to be 
treated leniently.
The spec for lenient parsing would be clearer if it were specified as 
allowing any
of the patterns of appendOffset.  The current wording around the 
character after the hour
may be confusing.

In the specification of appendOffset(pattern, noOffsetText) how about:

"When parsing in lenient mode, the longest valid pattern that matches 
the input is used. Only the hours are mandatory, minutes  and seconds are optional."

Roger


[1] https://bugs.openjdk.java.net/browse/JDK-8032051




On 2/26/2016 1:10 PM, Stephen Colebourne wrote:
> It is important to also consider the case where the user wants to
> format using HH:MM but parse seconds if they are provided.
>
> As I said above, this is no different to SignStyle, where the user
> requests something specific on format, but accepts anything on input.
>
> The pattern is still used for formatting and strict parsing under
> these changes. It is effectively ignored in lenient parsing (which is
> the very definition of leniency).
>
> Another way to look at it:
>
> using a pattern of HH:MM and strict:
> +02 - disallowed
> +02:00 - allowed
> +02:00:00 - disallowed
>
> using a pattern of HH:mm and strict:
> +02 - allowed
> +02:00 - allowed
> +02:00:00 - disallowed
>
> using any pattern and lenient:
> +02 - allowed
> +02:00 - allowed
> +02:00:00 - allowed
>
> This covers pretty much anything a user needs when parsing.
>
> Stephen
>
>
> On 26 February 2016 at 17:38, Roger Riggs  wrote:
>> Hi Stephen,
>>
>> Even in lenient mode the parser needs to stick to the fields provided in the
>> pattern.
>> If the caller intends to parse seconds, the pattern should include seconds.
>> Otherwise the caller has not been able to specify their intent.
>> That's consistent with lenient mode used in the other fields.
>> Otherwise, the pattern is irrelevant except for whether it contains a ":"
>> and makes
>> the spec nearly useless.
>>
>> Roger
>>
>>
>>
>> On 2/26/2016 12:09 PM, Stephen Colebourne wrote:
>>> On 26 February 2016 at 15:00, Roger Riggs  wrote:
>>>> Hi Stephen,
>>>>
>>>> It does not seem natural to me with a pattern of HHMM for it to parse
>>>> more
>>>> than 4 digits.
>>>> I can see lenient modifying the behavior as it it were HHmm, but there is
>>>> no
>>>> indication in the pattern
>>>> that seconds would be considered.  How it would it be implied from the
>>>> spec?
>>> The spec is being expanded to define what happens. Previously it
>>> didn't define it at all, and would throw an error.
>>>
>>> Lenient parsing typically accepts much more than the strict parsing.
>>>
>>> When parsing numbers, you may set the SignStyle to NEVER, but the sign
>>> will still be parsed in lenient mode
>>>
>>> When parsing text, you may select the short output format, but any
>>> length of text will be parsed in lenient mode.
>>>
>>> As such, it is very much in line with the behavour of the API to parse
>>> a much broader format than the one requested in lenient mode. (None of
>>> this affects strict mode).
>>>
>>> Stephen
>>>
>>>
>>>> In the original issue, appendOffsetId is defined as using the +HH:MM:ss
>>>> pattern and
>>>> specific to ISO the MM should be allowed to be optional.   There is no
>>>> question of parsing
>>>> extra digits not included in the requested pattern.
>>>>
>>>> Separately, this is specifying the new lenient behavior of
>>>> appendOffset(pattern, noffsetText).
>>>> In that case, I don't think it will be understood that patterns 'shorter'
>>>> than the input will
>>>> gobble up extra digits and ':'s.
>>>>
>>>> Roger
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 2/26/2016 9:42 AM, Stephen Colebourne wrote:
>>>>
>>>> Lenient can be however lenient we define it to be. Allowing minutes
>>>> and seconds to be parsed when not specified in the pattern is the key
>>>> part of the change. Whether the parser copes with both colons and
>>>> no-colons is the choice at hand here. It seems to me that since the
>>>> parser can easily handle figuring out whether the colon is present or
>>>> not, we should just allow the parser to be fully lenient.
>>>>
>>>> Stephen
>>>>
>>>>
>>>> On 26 February 2016 at 14:15, Roger Riggs  wrote:
>>>>
>>>> HI Stephen,
>>>>
>>>> How lenient is lenient supposed to be? Looking at the offset test cases,
>>>> it
>>>> seems to allow  minutes
>>>> and seconds digits to be parsed even if the pattern did not include them.
>>>>
>>>> + @DataProvider(name="lenientOffsetParseData")
>>>> + Object[][] data_lenient_offset_parse() {
>>>> + return new Object[][] {
>>>> + {"+HH", "+01", 3600},
>>>> + {"+HH", "+0101", 3660},
>>>> + {"+HH", "+010101", 3661},
>>>> + {"+HH", "+01", 3600},
>>>> + {"+HH", "+01:01", 3660},
>>>> + {"+HH", "+01:01:01", 3661},
>>>>
>>>> Thanks, Roger
>>>>
>>>>
>>>>
>>>> On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
>>>>
>>>> I don't think this is quite right.
>>>>
>>>> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>>>>       parseType = 10;
>>>> }
>>>>
>>>> This code will *always* select type 10 (colons) if a colon is found at
>>>> position+3. Whereas the spec now says that it should only do this if
>>>> the pattern is "HH". For other patterns, the colon/no-colon choice is
>>>> defined to be based on the pattern.
>>>>
>>>> That said, I'm thinking it is better to make the spec more lenient to
>>>> match the behaviour as implemented:
>>>>
>>>>
>>>> When parsing in lenient mode, only the hours are mandatory - minutes
>>>> and seconds are optional. If the character after the hour digits is a
>>>> colon
>>>> then the parser will parse using the pattern "HH:mm:ss", otherwise the
>>>> parser will parse using the pattern "HHmmss".
>>>>
>>>>
>>>> Additional TCKDateTimeFormatterBuilder tests will be needed to
>>>> demonstrate the above. There should also be a test for data following
>>>> the lenient parse. The following should all succeed:
>>>>
>>>>
>>>>
>>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
>>>> "+01:00Europe/London"
>>>> "+0100Europe/London"
>>>>
>>>>
>>>>
>>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
>>>> "+01:Europe/London"
>>>>
>>>> Note this special case, where the colon affects the parse type, but is
>>>> not ultimately part of the offset, thus it is left to match the
>>>> appendLiteral(":")
>>>>
>>>> You may want to think of some additional nasty edge cases!
>>>>
>>>> Stephen
>>>>
>>>> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>>>>
>>>> Hi all,
>>>> Please see the updated webrev
>>>> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>>>>
>>>> Thanks and Regards,
>>>> Nadeesh
>>>>
>>>> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>>>
>>>> Thanks for the changes.
>>>>
>>>> In `DateTimeFormatter`, the code should be
>>>>
>>>> .parseLenient()
>>>> .appendOffsetId()
>>>> .parseStrict()
>>>>
>>>> and the same in the other case. This ensures that existing callers who
>>>> then embed the formatter in another formatter (like the
>>>> ZONED_DATE_TIME constant) are unaffected.
>>>>
>>>>
>>>> The logic for lenient parsing does not look right as it only handles
>>>> types 5 and 6. This table shows the mappings needed:
>>>>
>>>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>>>> "+HHmm",  -> "+HHmmss",
>>>> "+HH:mm",  -> "+HH:mm:ss",
>>>> "+HHMM",  -> "+HHmmss",
>>>> "+HH:MM",  -> "+HH:mm:ss",
>>>> "+HHMMss",  -> "+HHmmss",
>>>> "+HH:MM:ss",  -> "+HH:mm:ss",
>>>> "+HHMMSS",  -> "+HHmmss",
>>>> "+HH:MM:SS",  -> "+HH:mm:ss",
>>>> "+HHmmss",
>>>> "+HH:mm:ss",
>>>>
>>>> Note that the "+HH" pattern is a special case, as we don't know
>>>> whether to use the colon or non-colon pattern. Whether to require
>>>> colon or not is based on whether the next character after the HH is a
>>>> colon or not.
>>>>
>>>> Proposed appendOffsetId() Javadoc:
>>>>
>>>> * Appends the zone offset, such as '+01:00', to the formatter.
>>>> * 
      
>>>> * This appends an instruction to format/parse the offset ID to the
>>>> builder.
>>>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>>>> * See {@link #appendOffset(String, String)} for details on formatting
>>>> and parsing.
>>>>
>>>> Proposed appendOffset(String, String) Javadoc:
>>>>
>>>> * During parsing, the offset...
>>>>
>>>> changed to:
>>>>
>>>> * When parsing in strict mode, the input must contain the mandatory
>>>> and optional elements are defined by the specified pattern.
>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>> the section of the formatter is optional.
>>>> * 
      
>>>> * When parsing in lenient mode, only the hours are mandatory - minutes
>>>> and seconds are optional.
>>>> * The colons are required if the specified pattern contains a colon.
>>>> * If the specified pattern is "+HH", the presence of colons is
>>>> determined by whether the character after the hour digits is a colon
>>>> or not.
>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>> the section of the formatter is optional.
>>>>
>>>> thanks and sorry for delay
>>>> Stephen
>>>>
>>>>
>>>>
>>>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>>>
>>>> Hi all,
>>>>
>>>> Please review a fix for
>>>>
>>>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>>>
>>>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>>>
>>>> --
>>>> Thanks and Regards,
>>>> Nadeesh TV
>>>>
>>>> --
>>>> Thanks and Regards,
>>>> Nadeesh TV
>>>>
>>>>



From kumar.x.srinivasan at oracle.com  Mon Feb 29 16:06:17 2016
From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan)
Date: Mon, 29 Feb 2016 08:06:17 -0800
Subject: RFR: 8150469: unpack200 fails to compare crc correctly.
Message-ID: <56D46C79.80807@oracle.com>

Hello John, Alex,

Please review fix for https://bugs.openjdk.java.net/browse/JDK-8150469

The previous fix keeps the storage in the unpacker object which is
reset, for a multi-segmented pack archive, thus the computed crc32 is lost.

Now the storage is moved into the gzip container, which persists, across
multiple segments, I  also added the length check as specified by RFC 1952,
as  implemented by JDK.

http://hg.openjdk.java.net/jdk9/dev/jdk/file/e4af8119eba4/src/java.base/share/classes/java/util/zip/GZIPInputStream.java#l222

The review is at:
http://cr.openjdk.java.net/~ksrini/8150469/webrev.00/


Thanks
Kumar




From tprintezis at twitter.com  Mon Feb 29 16:30:45 2016
From: tprintezis at twitter.com (Tony Printezis)
Date: Mon, 29 Feb 2016 11:30:45 -0500
Subject: Frequent allocations / promotions of
	StringCoding$String{Encoder,Decoder} objects
In-Reply-To: <56CF6F2B.8010506@oracle.com>
References: 
	<56CF6F2B.8010506@oracle.com>
Message-ID: 

All,

Thanks for the replies!

Aleksey,

Thanks for pointing me to the CR (JDK-4806753). I?m a bit confused though. The CR was resolved as fixed but Mark?s patch doesn?t seem to have made it to the StringCoding.java file (I checked the JDK 8 and 9 source repos; also the class sources in src.zip in 1.5.0_1, the Fixed In release, and for good measure last 5 and 6 releases). However, I noticed that the cache in java.nio.cs.ThreadLocalCoders looks very similar to Mark's cache. Was it decided to maybe cache the CharsetEncoders/Decoders instead of the StringCoding$StringEncoders/Decoders (i.e., was that the fix for JDK-4806753)? Unfortunately, this is pre-mercurial so I can?t get the history on this. Any volunteers at Oracle?

One more observation: the change that introduced the SoftReferences:

-? ? /* The cached coders for each thread
- ? ? */
-? ? private static ThreadLocal decoder = new ThreadLocal();
-? ? private static ThreadLocal encoder = new ThreadLocal();
+? ? /** The cached coders for each thread */
+? ? private final static ThreadLocal> decoder =
+? ? ? ? new ThreadLocal>();
+? ? private final static ThreadLocal> encoder =
+? ? ? ? new ThreadLocal>();

was done by this changeset:

changeset: ? 18:b5da6145b050
user:? ? ? ? martin
date:? ? ? ? Sun Mar 09 21:56:42 2008 -0700
files: ? ? ? src/share/classes/java/lang/StringCoding.java
description:
6671834: (str) Eliminate StringCoding.java compile warnings
Reviewed-by: iris

Relevant CR:

https://bugs.openjdk.java.net/browse/JDK-6671834

There is no explanation of why SoftReferences were introduced. The only comment from Martin is ?A fine things to do.? Eliminating compilation warnings is indeed a fine thing to do. Piggy-backing functional changes on that without any explanation is not (FWIW). I personally propose to remove the use of SoftReferences moving forward.

Going back to introducing the caches to StringCoding: I can replicate what?s in java.nio.cs.ThreadLocalCoders or I can re-use the ThreadLocalCoders.Cache class. I?d rather do the latter. Any thoughts on what the best way to do this is (move it to a file by itself, make the class public, etc.)?

Tony

On February 25, 2016 at 4:16:32 PM, Aleksey Shipilev (aleksey.shipilev at oracle.com) wrote:

On 02/25/2016 11:48 PM, Tony Printezis wrote:  
> Has anyone identified this issue before?  

Hm, there is a blast from the past:  
https://bugs.openjdk.java.net/browse/JDK-4806753  

In comments there, Mark suggests a patch that apparently handles  
multiple coders. I wonder where did that go. Also, Martin Buchholz is  
still an assignee there :)  

> We believe that caching a small number of coders per thread (instead  
> of just one) could avoid this unnecessary churn. We?ll be happy to  
> contribute such a fix if there?s interest in getting it accepted.  

Yes, this seems important.  

Thanks,  
-Aleksey  

-----

Tony Printezis | JVM/GC Engineer / VM Team | Twitter

@TonyPrintezis
tprintezis at twitter.com



From omajid at redhat.com  Mon Feb 29 16:51:54 2016
From: omajid at redhat.com (Omair Majid)
Date: Mon, 29 Feb 2016 11:51:54 -0500
Subject: RFR: 8150469: unpack200 fails to compare crc correctly.
In-Reply-To: <56D46C79.80807@oracle.com>
References: <56D46C79.80807@oracle.com>
Message-ID: <20160229165154.GA3771@redhat.com>

Hi,

* Kumar Srinivasan  [2016-02-29 11:07]:
> Please review fix for https://bugs.openjdk.java.net/browse/JDK-8150469
> 
> The review is at:
> http://cr.openjdk.java.net/~ksrini/8150469/webrev.00/

I can confirm this patch fixes the problem with a particular pack.gz
that I was observing.

(This isn't a review of the patch.)

Thanks for the quick fix!

Regards,
Omair

-- 
PGP Key: 66484681 (http://pgp.mit.edu/)
Fingerprint = F072 555B 0A17 3957 4E95  0056 F286 F14F 6648 4681


From Roger.Riggs at Oracle.com  Mon Feb 29 16:58:50 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 29 Feb 2016 11:58:50 -0500
Subject: RFR [9] 8149122: Move sun.misc.URLClassPath and Resource to an
	internal package
In-Reply-To: <56D4598E.6050503@oracle.com>
References: <56D4598E.6050503@oracle.com>
Message-ID: <56D478CA.6030909@Oracle.com>

Hi Chris,

As a package move it looks fine.

Roger


On 2/29/2016 9:45 AM, Chris Hegarty wrote:
> sun.misc.URLClassPath, and its accompanying Resource, are not
> "Critical APIs", as defined by JEP 260, so they should be moved
> out of sun.misc and placed into a more appropriate package where
> they can be encapsulated.
>
> http://cr.openjdk.java.net/~chegar/8149122/
> https://bugs.openjdk.java.net/browse/JDK-8149122
>
> -Chris.



From Alan.Bateman at oracle.com  Mon Feb 29 17:04:44 2016
From: Alan.Bateman at oracle.com (Alan Bateman)
Date: Mon, 29 Feb 2016 17:04:44 +0000
Subject: RFR [9] 8149122: Move sun.misc.URLClassPath and Resource to an
	internal package
In-Reply-To: <56D4598E.6050503@oracle.com>
References: <56D4598E.6050503@oracle.com>
Message-ID: <56D47A2C.3070307@oracle.com>



On 29/02/2016 14:45, Chris Hegarty wrote:
> sun.misc.URLClassPath, and its accompanying Resource, are not
> "Critical APIs", as defined by JEP 260, so they should be moved
> out of sun.misc and placed into a more appropriate package where
> they can be encapsulated.
>
> http://cr.openjdk.java.net/~chegar/8149122/
> https://bugs.openjdk.java.net/browse/JDK-8149122
Looks okay to me too. You might be able to drop a few imports, like in 
URLClassPath importing FileURLMapper for example.

-Alan


From daniel.fuchs at oracle.com  Mon Feb 29 17:46:52 2016
From: daniel.fuchs at oracle.com (Daniel Fuchs)
Date: Mon, 29 Feb 2016 18:46:52 +0100
Subject: RFR: 8150856 - Inconsistent API documentation for @param caller in
	System.LoggerFinder.getLogger
Message-ID: <56D4840C.3030006@oracle.com>

Hi,

Please find below a trivial fix for:

https://bugs.openjdk.java.net/browse/JDK-8150856
8150856: Inconsistent API documentation for @param caller
          in System.LoggerFinder.getLogger

http://cr.openjdk.java.net/~dfuchs/webrev_8150856/webrev.00

The @param caller clause says that caller can be null, whereas
the @throws clause says that NPE will be thrown.

The @throws clause is correct and @param needs to be fixed.

best regards,

-- daniel

--- old/src/java.base/share/classes/java/lang/System.java	2016-02-29 
18:41:30.000000000 +0100
+++ new/src/java.base/share/classes/java/lang/System.java	2016-02-29 
18:41:30.000000000 +0100
@@ -1,5 +1,5 @@
  /*
- * Copyright (c) 1994, 2014, Oracle and/or its affiliates. All rights 
reserved.
+ * Copyright (c) 1994, 2016, Oracle and/or its affiliates. All rights 
reserved.
   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
   *
   * This code is free software; you can redistribute it and/or modify it
@@ -1419,7 +1419,6 @@
           *
           * @param name the name of the logger.
           * @param caller the class for which the logger is being 
requested;
-         *               can be {@code null}.
           *
           * @return a {@link Logger logger} suitable for the given caller's
           *         use.


From martinrb at google.com  Mon Feb 29 17:52:31 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 29 Feb 2016 09:52:31 -0800
Subject: RFR: jsr166 jdk9 integration wave 5
In-Reply-To: 
References: 
Message-ID: 

I added

8150523: improve jtreg test timeout handling, especially -timeout:

to this wave, inspired by smarks.
I stress tested this with the flags that caused JDK-8150523 to fail,
and they now seem robust, as long as a reasonable -timeout flag is
provided to jtreg.

On Tue, Feb 23, 2016 at 6:58 PM, Martin Buchholz  wrote:
> A very boring jsr166 integration, focused on reliability.
> This one has the promised "even more unnecessarily robust" ThreadLocalRandom.
>
> http://cr.openjdk.java.net/~martin/webrevs/openjdk9/jsr166-jdk9-integration/


From martinrb at google.com  Mon Feb 29 17:54:36 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 29 Feb 2016 09:54:36 -0800
Subject: RFR: 8150856 - Inconsistent API documentation for @param caller
	in System.LoggerFinder.getLogger
In-Reply-To: <56D4840C.3030006@oracle.com>
References: <56D4840C.3030006@oracle.com>
Message-ID: 

You need to delete the orphaned semicolon

On Mon, Feb 29, 2016 at 9:46 AM, Daniel Fuchs  wrote:
> Hi,
>
> Please find below a trivial fix for:
>
> https://bugs.openjdk.java.net/browse/JDK-8150856
> 8150856: Inconsistent API documentation for @param caller
>          in System.LoggerFinder.getLogger
>
> http://cr.openjdk.java.net/~dfuchs/webrev_8150856/webrev.00
>
> The @param caller clause says that caller can be null, whereas
> the @throws clause says that NPE will be thrown.
>
> The @throws clause is correct and @param needs to be fixed.
>
> best regards,
>
> -- daniel
>
> --- old/src/java.base/share/classes/java/lang/System.java       2016-02-29
> 18:41:30.000000000 +0100
> +++ new/src/java.base/share/classes/java/lang/System.java       2016-02-29
> 18:41:30.000000000 +0100
> @@ -1,5 +1,5 @@
>  /*
> - * Copyright (c) 1994, 2014, Oracle and/or its affiliates. All rights
> reserved.
> + * Copyright (c) 1994, 2016, Oracle and/or its affiliates. All rights
> reserved.
>   * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
>   *
>   * This code is free software; you can redistribute it and/or modify it
> @@ -1419,7 +1419,6 @@
>           *
>           * @param name the name of the logger.
>           * @param caller the class for which the logger is being requested;
> -         *               can be {@code null}.
>           *
>           * @return a {@link Logger logger} suitable for the given caller's
>           *         use.


From scolebourne at joda.org  Mon Feb 29 18:59:15 2016
From: scolebourne at joda.org (Stephen Colebourne)
Date: Mon, 29 Feb 2016 18:59:15 +0000
Subject: RFR:JDK-8032051:"ZonedDateTime" class "parse" method fails with
	short time zone offset ("+01")
In-Reply-To: <56D4692B.1030402@Oracle.com>
References: <56BCED89.7040007@oracle.com>
	
	<56CF2154.6050503@oracle.com>
	
	<56D05E0E.6030003@Oracle.com>
	
	<56D06876.4020000@Oracle.com>
	
	<56D08DA5.5030700@Oracle.com>
	
	<56D4692B.1030402@Oracle.com>
Message-ID: 

I'm happy to go back to the spec I proposed before. That spec would
determine colons dynamically only for pattern HH. Otherwise, it would
use the presence/absence of a colon in the pattern as the signal. That
would deal with the ISO-8601 problem and resolve the original issue
(as ISO_OFFSET_DATE_TIME uses HH:MM:ss, which would leniently parse
using colons).

Writing the spec wording is not easy however. I had:

When parsing in lenient mode, only the hours are mandatory - minutes
and seconds are optional. The colons are required if the specified
pattern contains a colon. If the specified pattern is "+HH", the
presence of colons is determined by whether the character after the
hour digits is a colon or not. If the offset cannot be parsed then an
exception is thrown unless the section of the formatter is optional.

which isn't too bad but alternatives are possible.

Stephen




On 29 February 2016 at 15:52, Roger Riggs  wrote:
> Hi Stephen,
>
> As a fix for the original issue[1], not correctly parsing a ISO defined
> offset, the use of lenient
> was a convenient implementation technique (hack).  But with the expanded
> definition of lenient,
> it will allow many forms of the offset that are not allowed by the ISO
> specification
> and should not be accepted forDateTimeFormatter. ISO_OFFSET_DATE_TIME.
> In particular, ISO requires the ":" to separate the minutes.
> I'm not sure how to correctly fix the original issue with the new
> specification of lenient offset
> parsing without introducing some more specific implementation information.
>
>
> WRT the lenient parsing mode for appendOffset:
>
> I was considering that the subfields of the offset were to be treated
> leniently but it seems
> you were treating the entire offset field and text as the unit to be treated
> leniently.
> The spec for lenient parsing would be clearer if it were specified as
> allowing any
> of the patterns of appendOffset.  The current wording around the character
> after the hour
> may be confusing.
>
> In the specification of appendOffset(pattern, noOffsetText) how about:
>
> "When parsing in lenient mode, the longest valid pattern that matches the
> input is used. Only the hours are mandatory, minutes  and seconds are
> optional."
>
> Roger
>
>
> [1] https://bugs.openjdk.java.net/browse/JDK-8032051
>
>
>
>
>
> On 2/26/2016 1:10 PM, Stephen Colebourne wrote:
>>
>> It is important to also consider the case where the user wants to
>> format using HH:MM but parse seconds if they are provided.
>>
>> As I said above, this is no different to SignStyle, where the user
>> requests something specific on format, but accepts anything on input.
>>
>> The pattern is still used for formatting and strict parsing under
>> these changes. It is effectively ignored in lenient parsing (which is
>> the very definition of leniency).
>>
>> Another way to look at it:
>>
>> using a pattern of HH:MM and strict:
>> +02 - disallowed
>> +02:00 - allowed
>> +02:00:00 - disallowed
>>
>> using a pattern of HH:mm and strict:
>> +02 - allowed
>> +02:00 - allowed
>> +02:00:00 - disallowed
>>
>> using any pattern and lenient:
>> +02 - allowed
>> +02:00 - allowed
>> +02:00:00 - allowed
>>
>> This covers pretty much anything a user needs when parsing.
>>
>> Stephen
>>
>>
>> On 26 February 2016 at 17:38, Roger Riggs  wrote:
>>>
>>> Hi Stephen,
>>>
>>> Even in lenient mode the parser needs to stick to the fields provided in
>>> the
>>> pattern.
>>> If the caller intends to parse seconds, the pattern should include
>>> seconds.
>>> Otherwise the caller has not been able to specify their intent.
>>> That's consistent with lenient mode used in the other fields.
>>> Otherwise, the pattern is irrelevant except for whether it contains a ":"
>>> and makes
>>> the spec nearly useless.
>>>
>>> Roger
>>>
>>>
>>>
>>> On 2/26/2016 12:09 PM, Stephen Colebourne wrote:
>>>>
>>>> On 26 February 2016 at 15:00, Roger Riggs 
>>>> wrote:
>>>>>
>>>>> Hi Stephen,
>>>>>
>>>>> It does not seem natural to me with a pattern of HHMM for it to parse
>>>>> more
>>>>> than 4 digits.
>>>>> I can see lenient modifying the behavior as it it were HHmm, but there
>>>>> is
>>>>> no
>>>>> indication in the pattern
>>>>> that seconds would be considered.  How it would it be implied from the
>>>>> spec?
>>>>
>>>> The spec is being expanded to define what happens. Previously it
>>>> didn't define it at all, and would throw an error.
>>>>
>>>> Lenient parsing typically accepts much more than the strict parsing.
>>>>
>>>> When parsing numbers, you may set the SignStyle to NEVER, but the sign
>>>> will still be parsed in lenient mode
>>>>
>>>> When parsing text, you may select the short output format, but any
>>>> length of text will be parsed in lenient mode.
>>>>
>>>> As such, it is very much in line with the behavour of the API to parse
>>>> a much broader format than the one requested in lenient mode. (None of
>>>> this affects strict mode).
>>>>
>>>> Stephen
>>>>
>>>>
>>>>> In the original issue, appendOffsetId is defined as using the +HH:MM:ss
>>>>> pattern and
>>>>> specific to ISO the MM should be allowed to be optional.   There is no
>>>>> question of parsing
>>>>> extra digits not included in the requested pattern.
>>>>>
>>>>> Separately, this is specifying the new lenient behavior of
>>>>> appendOffset(pattern, noffsetText).
>>>>> In that case, I don't think it will be understood that patterns
>>>>> 'shorter'
>>>>> than the input will
>>>>> gobble up extra digits and ':'s.
>>>>>
>>>>> Roger
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 2/26/2016 9:42 AM, Stephen Colebourne wrote:
>>>>>
>>>>> Lenient can be however lenient we define it to be. Allowing minutes
>>>>> and seconds to be parsed when not specified in the pattern is the key
>>>>> part of the change. Whether the parser copes with both colons and
>>>>> no-colons is the choice at hand here. It seems to me that since the
>>>>> parser can easily handle figuring out whether the colon is present or
>>>>> not, we should just allow the parser to be fully lenient.
>>>>>
>>>>> Stephen
>>>>>
>>>>>
>>>>> On 26 February 2016 at 14:15, Roger Riggs 
>>>>> wrote:
>>>>>
>>>>> HI Stephen,
>>>>>
>>>>> How lenient is lenient supposed to be? Looking at the offset test
>>>>> cases,
>>>>> it
>>>>> seems to allow  minutes
>>>>> and seconds digits to be parsed even if the pattern did not include
>>>>> them.
>>>>>
>>>>> + @DataProvider(name="lenientOffsetParseData")
>>>>> + Object[][] data_lenient_offset_parse() {
>>>>> + return new Object[][] {
>>>>> + {"+HH", "+01", 3600},
>>>>> + {"+HH", "+0101", 3660},
>>>>> + {"+HH", "+010101", 3661},
>>>>> + {"+HH", "+01", 3600},
>>>>> + {"+HH", "+01:01", 3660},
>>>>> + {"+HH", "+01:01:01", 3661},
>>>>>
>>>>> Thanks, Roger
>>>>>
>>>>>
>>>>>
>>>>> On 2/26/2016 6:16 AM, Stephen Colebourne wrote:
>>>>>
>>>>> I don't think this is quite right.
>>>>>
>>>>> if ((length > position + 3) && (text.charAt(position + 3) == ':')) {
>>>>>       parseType = 10;
>>>>> }
>>>>>
>>>>> This code will *always* select type 10 (colons) if a colon is found at
>>>>> position+3. Whereas the spec now says that it should only do this if
>>>>> the pattern is "HH". For other patterns, the colon/no-colon choice is
>>>>> defined to be based on the pattern.
>>>>>
>>>>> That said, I'm thinking it is better to make the spec more lenient to
>>>>> match the behaviour as implemented:
>>>>>
>>>>>
>>>>> When parsing in lenient mode, only the hours are mandatory - minutes
>>>>> and seconds are optional. If the character after the hour digits is a
>>>>> colon
>>>>> then the parser will parse using the pattern "HH:mm:ss", otherwise the
>>>>> parser will parse using the pattern "HHmmss".
>>>>>
>>>>>
>>>>> Additional TCKDateTimeFormatterBuilder tests will be needed to
>>>>> demonstrate the above. There should also be a test for data following
>>>>> the lenient parse. The following should all succeed:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendZoneId();
>>>>> "+01:00Europe/London"
>>>>> "+0100Europe/London"
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> DateTimeFormatterBuilder().parseLenient().appendOffset("HH:MM").appendLiteral(":").appendZoneId();
>>>>> "+01:Europe/London"
>>>>>
>>>>> Note this special case, where the colon affects the parse type, but is
>>>>> not ultimately part of the offset, thus it is left to match the
>>>>> appendLiteral(":")
>>>>>
>>>>> You may want to think of some additional nasty edge cases!
>>>>>
>>>>> Stephen
>>>>>
>>>>> On 25 February 2016 at 15:44, nadeesh tv  wrote:
>>>>>
>>>>> Hi all,
>>>>> Please see the updated webrev
>>>>> http://cr.openjdk.java.net/~ntv/8032051/webrev.02/
>>>>>
>>>>> Thanks and Regards,
>>>>> Nadeesh
>>>>>
>>>>> On 2/23/2016 5:17 PM, Stephen Colebourne wrote:
>>>>>
>>>>> Thanks for the changes.
>>>>>
>>>>> In `DateTimeFormatter`, the code should be
>>>>>
>>>>> .parseLenient()
>>>>> .appendOffsetId()
>>>>> .parseStrict()
>>>>>
>>>>> and the same in the other case. This ensures that existing callers who
>>>>> then embed the formatter in another formatter (like the
>>>>> ZONED_DATE_TIME constant) are unaffected.
>>>>>
>>>>>
>>>>> The logic for lenient parsing does not look right as it only handles
>>>>> types 5 and 6. This table shows the mappings needed:
>>>>>
>>>>> "+HH",  -> "+HHmmss" or "+HH:mm:ss"
>>>>> "+HHmm",  -> "+HHmmss",
>>>>> "+HH:mm",  -> "+HH:mm:ss",
>>>>> "+HHMM",  -> "+HHmmss",
>>>>> "+HH:MM",  -> "+HH:mm:ss",
>>>>> "+HHMMss",  -> "+HHmmss",
>>>>> "+HH:MM:ss",  -> "+HH:mm:ss",
>>>>> "+HHMMSS",  -> "+HHmmss",
>>>>> "+HH:MM:SS",  -> "+HH:mm:ss",
>>>>> "+HHmmss",
>>>>> "+HH:mm:ss",
>>>>>
>>>>> Note that the "+HH" pattern is a special case, as we don't know
>>>>> whether to use the colon or non-colon pattern. Whether to require
>>>>> colon or not is based on whether the next character after the HH is a
>>>>> colon or not.
>>>>>
>>>>> Proposed appendOffsetId() Javadoc:
>>>>>
>>>>> * Appends the zone offset, such as '+01:00', to the formatter.
>>>>> * 
      
>>>>> * This appends an instruction to format/parse the offset ID to the
>>>>> builder.
>>>>> * This is equivalent to calling {@code appendOffset("+HH:MM:ss", "Z")}.
>>>>> * See {@link #appendOffset(String, String)} for details on formatting
>>>>> and parsing.
>>>>>
>>>>> Proposed appendOffset(String, String) Javadoc:
>>>>>
>>>>> * During parsing, the offset...
>>>>>
>>>>> changed to:
>>>>>
>>>>> * When parsing in strict mode, the input must contain the mandatory
>>>>> and optional elements are defined by the specified pattern.
>>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>>> the section of the formatter is optional.
>>>>> * 
      
>>>>> * When parsing in lenient mode, only the hours are mandatory - minutes
>>>>> and seconds are optional.
>>>>> * The colons are required if the specified pattern contains a colon.
>>>>> * If the specified pattern is "+HH", the presence of colons is
>>>>> determined by whether the character after the hour digits is a colon
>>>>> or not.
>>>>> * If the offset cannot be parsed then an exception is thrown unless
>>>>> the section of the formatter is optional.
>>>>>
>>>>> thanks and sorry for delay
>>>>> Stephen
>>>>>
>>>>>
>>>>>
>>>>> On 11 February 2016 at 20:22, nadeesh tv  wrote:
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Please review a fix for
>>>>>
>>>>> Bug Id  https://bugs.openjdk.java.net/browse/JDK-8032051
>>>>>
>>>>> webrev http://cr.openjdk.java.net/~ntv/8032051/webrev.01/
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>> Nadeesh TV
>>>>>
>>>>> --
>>>>> Thanks and Regards,
>>>>> Nadeesh TV
>>>>>
>>>>>
>


From iris.clark at oracle.com  Mon Feb 29 21:02:14 2016
From: iris.clark at oracle.com (Iris Clark)
Date: Mon, 29 Feb 2016 13:02:14 -0800 (PST)
Subject: RFR [9] 8150162: Move sun.misc.Version to a truly internal package
In-Reply-To: <60EF1BB4-095F-4881-A37D-C508D570F2AF@oracle.com>
References: 
	<5b1a5288-3c75-4437-aa78-b63169615db7@default>
	
	<66f6308b-60e1-4874-ae21-25c19f822f26@default>
	<90AE881D-5940-4BFF-91E6-7637AEB7D4EE@oracle.com>
	<431be29a-f350-4790-81bd-13365b505b0e@default>
	
	<60EF1BB4-095F-4881-A37D-C508D570F2AF@oracle.com>
Message-ID: <2a0abfd7-5beb-41fa-89d0-846d7fd861b7@default>

Hi,Chris.

>  http://cr.openjdk.java.net/~chegar/8150162.01/

Looks good to me.

Regards,
iris

-----Original Message-----
From: Chris Hegarty 
Sent: Monday, February 29, 2016 1:05 AM
To: Mandy Chung; Iris Clark
Cc: core-libs-dev
Subject: Re: RFR [9] 8150162: Move sun.misc.Version to a truly internal package

Updated webrv:
  http://cr.openjdk.java.net/~chegar/8150162.01/

The *{Major,Minor,...}Version(), and supporting, methods have been removed. As well as the native methods that support their implementation.  Further, additional, clean up, on the hotspot side, can be done as a follow up.

-Chris.

Note: test/sun/misc/Version/Version.java is no longer interesting as it asserts the values returned by the *{Major,Minor,...}Version() methods, which no longer exist.


From Roger.Riggs at Oracle.com  Mon Feb 29 21:36:22 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 29 Feb 2016 16:36:22 -0500
Subject: RFR 9: 8150346 : java/lang/ProcessHandle/Infotest.java failed:
	startTime after process spawn completed
In-Reply-To: <56D06B37.9010901@oracle.com>
References: <56CCDD8F.3070807@Oracle.com> <56D068FF.20504@Oracle.com>
	<56D06B37.9010901@oracle.com>
Message-ID: <56D4B9D6.7000102@Oracle.com>

Hi,

Thanks for the review but I think I need a JDK 9 _R_eviewer for this 
test fix.

Roger


On 2/26/2016 10:11 AM, Claes Redestad wrote:
> Looks good to me.
>
> /Claes
>
> On 2016-02-26 16:02, Roger Riggs wrote:
>> Ping...
>>
>> On 2/23/2016 5:30 PM, Roger Riggs wrote:
>>> Please review a test fix to resolve a timing race with the start 
>>> time of a spawned process.
>>>
>>> Webrev:
>>>   http://cr.openjdk.java.net/~rriggs/webrev-start-8150346/
>>>
>>> Jira:
>>>    https://bugs.openjdk.java.net/browse/JDK-8150346
>>>
>>> Thanks, Roger
>>>
>>
>



From martinrb at google.com  Mon Feb 29 22:09:48 2016
From: martinrb at google.com (Martin Buchholz)
Date: Mon, 29 Feb 2016 14:09:48 -0800
Subject: RFR 9: 8150346 : java/lang/ProcessHandle/Infotest.java failed:
	startTime after process spawn completed
In-Reply-To: <56CCDD8F.3070807@Oracle.com>
References: <56CCDD8F.3070807@Oracle.com>
Message-ID: 

Just nitpicks - your change is approved!

                     while ((s = lines.readLine()) != null) {

I think it's better style to localize

                    for  (String s; (s = lines.readLine()) != null; ) {


 121                 String s = lines.readLine();

I think it's better style to write to a local that's never used.
In my old test TimedAcquireLeak I synchronize with child by
reading/writing just one byte.

        public static void main(String[] args) throws Throwable {
            // Synchronize with parent process, so that jps can find us
            sendByte(System.out);
            System.in.read();


On Tue, Feb 23, 2016 at 2:30 PM, Roger Riggs  wrote:
> Please review a test fix to resolve a timing race with the start time of a
> spawned process.
>
> Webrev:
>   http://cr.openjdk.java.net/~rriggs/webrev-start-8150346/
>
> Jira:
>    https://bugs.openjdk.java.net/browse/JDK-8150346
>
> Thanks, Roger
>


From Roger.Riggs at Oracle.com  Mon Feb 29 23:00:27 2016
From: Roger.Riggs at Oracle.com (Roger Riggs)
Date: Mon, 29 Feb 2016 18:00:27 -0500
Subject: RFR 9: 8150346 : java/lang/ProcessHandle/Infotest.java failed:
	startTime after process spawn completed
In-Reply-To: 
References: <56CCDD8F.3070807@Oracle.com>
	
Message-ID: <56D4CD8B.3090701@Oracle.com>

Hi Martin,

In this case the JavaChild is known to echo its arguments but they are 
not needed by the test.
I like the localized for-loop.

Thanks for the suggestion, Roger



On 2/29/2016 5:09 PM, Martin Buchholz wrote:
> Just nitpicks - your change is approved!
>
>                       while ((s = lines.readLine()) != null) {
>
> I think it's better style to localize
>
>                      for  (String s; (s = lines.readLine()) != null; ) {
>
>
>   121                 String s = lines.readLine();
>
> I think it's better style to write to a local that's never used.
> In my old test TimedAcquireLeak I synchronize with child by
> reading/writing just one byte.
>
>          public static void main(String[] args) throws Throwable {
>              // Synchronize with parent process, so that jps can find us
>              sendByte(System.out);
>              System.in.read();
>
>
> On Tue, Feb 23, 2016 at 2:30 PM, Roger Riggs  wrote:
>> Please review a test fix to resolve a timing race with the start time of a
>> spawned process.
>>
>> Webrev:
>>    http://cr.openjdk.java.net/~rriggs/webrev-start-8150346/
>>
>> Jira:
>>     https://bugs.openjdk.java.net/browse/JDK-8150346
>>
>> Thanks, Roger
>>



From marcus at lagergren.net  Fri Feb 26 16:56:34 2016
From: marcus at lagergren.net (Marcus Lagergren)
Date: Fri, 26 Feb 2016 17:56:34 +0100
Subject: Custom security policy without replacing files in the OpenJDK?
Message-ID: 

Hi!

Is it possible to override lib/security/local_policy on app level without patching jdk distro?
i.e. -Duse.this.policy.jar= ? or something?

Can?t find a way to do it

Regards
Marcus



From indeliblebluepen at gmail.com  Sat Feb 27 02:09:48 2016
From: indeliblebluepen at gmail.com (Jason C. McDonald)
Date: Fri, 26 Feb 2016 18:09:48 -0800
Subject: Replacement of Quicksort in java.util.Arrays with new Dual-Pivot
	Quicksort
In-Reply-To: <56D10470.9090509@oracle.com>
References: <4AAA27EA.8090902@Sun.COM>
	 <56D10470.9090509@oracle.com>
Message-ID: <56D1056C.4080208@gmail.com>

Hi Stuart,

I hate replying to an ancient threat, but I figured it was the best 
method. Thanks for the tips. The original paper was almost as hard to 
find as he is proving to be. :)

All the best,

-Jason C. McDonald

On 02/26/2016 06:05 PM, Stuart Marks wrote:
> Wow, is this a reply to a nearly seven-year-old email? [1]
>
> I don't know if Vladimir Yaroslavskiy is still on core-libs-dev. I dug 
> through tthe archives and found that he had posted a couple messages 
> somewhat later [2] [3] using different email addresses:
>
>   Vladimir Iaroslavski 
>   Vladimir Iaroslavski 
>
> You might try to contact him at one of these addresses. Note however 
> that the more recent one is still over five years old.
>
> He's also on LinkedIn. His profile says he works for Oracle, but as 
> far as I can see he no longer does.
>
> Good luck,
>
> s'marks
>
>
> [1] 
> http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-September/002630.html
>
> [2] 
> http://mail.openjdk.java.net/pipermail/core-libs-dev/2010-April/004133.html
>
> [3] 
> http://mail.openjdk.java.net/pipermail/core-libs-dev/2011-February/005821.html
>
>
>
> On 2/23/16 6:05 PM, Jason C. McDonald wrote:
>> I think this is the best place to contact the original authors.
>>
>> The link to Vladimir Yaroslavskiy's original whitepaper describing the
>> algorithm and its proofs was, unfortunately, broken. Using Archive.org's
>> Wayback Machine, I was able to get the last known revision. I 
>> reformatted
>> the document in LibreOffice for ease of reading, and fixed some minor
>> grammatical errors.
>>
>> I also implemented the algorithm in an open-source (MIT License) C++
>> library, which I hope to release in the coming few months.
>>
>> In order to make this algorithm and its proofs more easily accessible, I
>> would like to make the revised whitepaper publicly and freely 
>> available from
>> my own web servers, but I wanted to check with the original author(s) 
>> first.
>> Furthermore, I wanted to find out if there have been any revisions 
>> since the
>> 22 September 2009 version of the whitepaper I acquired.
>>
>> Thank you in advance!
>>

-- 
Jason C. McDonald

Check out my scribblings!
www.indeliblebluepen.com