From Alan.Bateman at oracle.com Tue Jan 2 16:14:39 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jan 2018 16:14:39 +0000 Subject: RFR [11] JDK-8179424: Remove terminally deprecated sun.reflect.Reflection.getCallerClass In-Reply-To: <0a57854d-b5d5-a30a-02cb-14617ef625ca@oracle.com> References: <90fbc317-4c39-d51e-f402-3803917d065d@oracle.com> <81e288b8-3b95-4fee-3ed8-f3c8a5be9d47@oracle.com> <0719bbfb-924d-9e70-1dfb-9ec782442080@oracle.com> <302E7962-B372-4F1F-92ED-BFDB5CF7B290@oracle.com> <628a9c02-8031-142c-8fef-18d0f00baa67@oracle.com> <99F52058-C1A7-459B-A731-188C70AE50F6@oracle.com> <6c71f4b2-3f4f-b4b3-84d8-59f9aaf128e8@oracle.com> <9BEBC253-F5FC-428C-988A-8D19E23A668F@oracle.com> <4e811ed4-54c6-b253-44cb-e2f6a9dd83fe@oracle.com> <34412585-eac6-0e62-a5b8-e8f8dd42645e@oracle.com> <0a57854d-b5d5-a30a-02cb-14617ef625ca@oracle.com> Message-ID: <9e92d615-319e-2901-e55f-72fc814236b5@oracle.com> On 22/12/2017 10:16, Chris Hegarty wrote: > > On 21/12/17 21:40, mandy chung wrote: >> ?... > >> ReflectionFactory or other class in sun.reflect package would do it. >>>> and create an issue to follow up sun.reflect.Reflection as flagged >>>> as a removed API. >>> That?s what the test now does with my changes. Why separate it >>> out into a separate issue??? If we need a test for an internal API, I >>> can add a scenario that uses sun.reflect.ReflectionFactory. >> >> Yes please. That's what this case intends to verify. > > Webrev with the test updated as suggested: > ? http://cr.openjdk.java.net/~chegar/8179424/webrev.03/ Updated webrev looks good to me too. -Alan From Alan.Bateman at oracle.com Tue Jan 2 16:35:03 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jan 2018 16:35:03 +0000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> Message-ID: On 29/12/2017 00:33, Steven Schlansker wrote: > Thanks for the discussion! > > So, it sounds like amending the message by default is going to be a non-starter -- but at least adding the information otherwise might be possible. > There are examples in other area where exceptions include detail information (for diagnostics purposes) when not running with a security manager. This may be something to look at here (and easy to try out too as it wouldn't require touching any native code either). -Alan From Alan.Bateman at oracle.com Tue Jan 2 17:01:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jan 2018 17:01:22 +0000 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> Message-ID: On 22/12/2017 21:51, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8194133 > http://cr.openjdk.java.net/~bpb/8194133/webrev.00/ > > Add jdk.internal.io.IOSupport with copy() methods for InputStream-to-OutputStream copying and modify some classes to use these new methods. > > One thing that I noticed when looking at this is that in the fix for https://bugs.openjdk.java.net/browse/JDK-8193842, the Files.copy() method had a loop like > > while ((n = read(?)) > 0) > > whereas InputStream.transferTo() had > > while((n = read(?)) >= 0) > > which is to say that Files.copy() would terminate if there were an empty read() but transferTo() would not. The patch for 8193842 therefore possibly introduced a subtle behavioral change which no one noticed. > read(byte[]) is blocking so it should only return 0 if called with a 0-length array. So not clear to me that Files.copy methods needs to use this. For JrtPath then the code is specific to the jimage -> jimage case. I think that can be left along too or just replaced with a better implementation for jrtfs. -Alan. From Alan.Bateman at oracle.com Tue Jan 2 16:42:12 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 2 Jan 2018 16:42:12 +0000 Subject: RFR: 8194154 patch for crash at File.getCanonicalPath() In-Reply-To: References: Message-ID: <5dc3e332-2c36-3261-e2f7-a7ecc532ce72@oracle.com> On 25/12/2017 09:40, Wenqian Pei wrote: > Hi: > > Bug:?https://bugs.openjdk.java.net/browse/JDK-8194154 > > We?found?that?if?user?defines?-Duser.dir?like?"/home/a/b/c/",?jvm?will?crash?at?File.getCanonicalPath()?in?java?process?bootstrap?(before?invoking?user's?java?code).?The?native?implematation?of?canonicalize_md.c:collapsible(char?*names)?has?problem?in?processing?double?'/',?parameter 'names' need?normalized?before?JNI_CALL. > user.dir is a "read-only" property and should never be changed on the command-line or in a running VM (it breaks APIs in other areas to have user.dir be different to the actual working directory). Someday we need to figure out how to enforce this. In the mean-time, the runtime shouldn't crash so I agree it should be fixed. -Alan. From stevenschlansker at gmail.com Tue Jan 2 21:25:28 2018 From: stevenschlansker at gmail.com (Steven Schlansker) Date: Tue, 2 Jan 2018 13:25:28 -0800 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> Message-ID: <862C913E-845F-4A8A-8090-0079F861B67B@gmail.com> On Dec 31, 2017, at 7:24 AM, Peter Levart wrote: > > Hi, >> >> I believe there are concerns with too much information that can be considered "sensitive" (like host names and IP addresses) appearing in error messages due to them ending up in log files and bug reports. >> >> David > > For debugging purposes it might sometimes be enough to get just a hint about the actual address / port but not reveal it entirely. The person doing debugging probably knows more about the environment than an average person so the hint might give him enough information to discern the actual address / port. Exposing just the last octet of an IP address and the last digit of the port might do. For example: > > java.net.ConnectException: Connection to X.X.X.205:XXX8 refused. > > So Steven, I'm curious whether such hint would help in your case? This would definitely be better than nothing! But it's still difficult, for example a common allocation pattern for us would be to assign networks to availability zones: 10.0.1.0/24 10.0.2.0/24 10.0.3.0/24 then if you pick the same last number for a well known service, you might end up with instances at 10.0.1.2, 10.0.2.2, 10.0.3.2 -- so depending on which octets are obscured you may end up with no useful information. The triggering incident for us was that one of our Amazon ELBs started responding incorrectly (blackholing data) -- so when you resolve "my-elb-1.amazonaws.amazon.com" you'd get three different IP addresses, and depending on which one is picked for the connect operation, you'll get all data blackholed. The author of the code in question obviously had not considered a failure mode where one of the "It Is In The Cloud So It Always Works" load balancer instances did not behave like the others -- so it did not emit any diagnostic information other than the hostname, and until we got a network operator in with WireShark it was quite difficult to determine that it was in fact exactly one instance of three failing. The pain of using WireShark to diagnose what should have been a quick scan through the logs, coupled with the sheer impossibility of annotating every network call in every application across our organization, pushed me to open this discussion to do it in one central place :) > An attacker that knows something about the environment could find out the missing pieces without such hints anyway (simply by scanning IPs / ports), so such partial information is not that sensitive nowadays. I've never really understood the justification behind this rule -- we have a similar policy in our organization, all IP addresses must be stripped. But knowing the layout of the network, I can tell you that if you are on the outside and have one of our 10.x.x.x IP addresses it gives you almost nothing (good luck routing to it) -- and if you are on the inside, it's not exactly difficult to enumerate these. DNS contains every name / IP you might find and it'll happily divulge all that information to you, and failing that, nmap can map out the majority of the network fairly reliably in seconds... I respect that the JVM has to be conservative, but there is a real usability problem here, and the security gain is a little questionable to me! > Another idea: define a one way function that maps the IP:port pair into a value which is displayed in the exception message. For debugging purposes this might be enough since the one doing debugging might know the set of possible IP:port pairs in advance. He could then apply the function to each of them in turn and find out the matching pair. This is interesting, but only if you can enumerate the IP:port that might match. We have a Apache Mesos cluster with O(100) nodes, each of which is assigned a pool of 1000 ports to assign to applications. Needing to enumerate a million possible hash matches could be a royal pain, and my understanding is that 100 nodes is not even near the size of the largest production clusters out there. It's not quite to the level of a non-starter, but as those numbers scale it could easily become useless. Some of the exceptions may not have a specific port (hostname resolution), may have port 0, or such a dynamically assigned port, so you'd have to guess these special cases too. Additionally, with cloud providers, you may not understand the space of IPs that an Abstract Cloud Device like ELB might come from -- it's coming from an AWS pool that you have no visibility into. > On Jan 2, 2018, at 8:35 AM, Alan Bateman wrote: > > On 29/12/2017 00:33, Steven Schlansker wrote: >> Thanks for the discussion! >> >> So, it sounds like amending the message by default is going to be a non-starter -- but at least adding the information otherwise might be possible. >> > There are examples in other area where exceptions include detail information (for diagnostics purposes) when not running with a security manager. This may be something to look at here (and easy to try out too as it wouldn't require touching any native code either). I like that! For now, I will proceed with improving my prototype to follow this suggestion -- unless there is a security manager, the exceptions are annotated. An alternate way to activate it, by configuring the platform logger e.g. "java.net.SocketException=TRACE", might be useful in case we want to allow end users to configure the feature explicitly and independent of security manager. Unfortunately I don't see how I can avoid changing the native code -- the exception message is initialized in native code, and by the time we call back to Java code, the necessary information is not passed in to the SocketException subclass constructor. So I may be misinterpreting your guidance here, but I'm not seeing it. I think I will hold off on adding Java level fields to the exception types. I expect if I do that, I will then have to get a spec change to add appropriate public API to make the data actually usable. Since the goal here is for log diagnostics as opposed to more structured data at the Java level, I'll avoid it. It also avoids any complications with regards to changing the serial format for such a common type. But I do think it means the work has to be done in the native code. From brian.burkhalter at oracle.com Tue Jan 2 23:43:57 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 2 Jan 2018 15:43:57 -0800 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> Message-ID: <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> On Jan 2, 2018, at 9:01 AM, Alan Bateman wrote: >> One thing that I noticed when looking at this is that in the fix for https://bugs.openjdk.java.net/browse/JDK-8193842, the Files.copy() method had a loop like >> >> while ((n = read(?)) > 0) >> >> whereas InputStream.transferTo() had >> >> while((n = read(?)) >= 0) >> >> which is to say that Files.copy() would terminate if there were an empty read() but transferTo() would not. The patch for 8193842 therefore possibly introduced a subtle behavioral change which no one noticed. >> > read(byte[]) is blocking so it should only return 0 if called with a 0-length array. Good point so ?>? should be used. > So not clear to me that Files.copy methods needs to use this. So you are suggesting leaving the call to InputStream.transferTo() in preference to IOSupport.copy()? > For JrtPath then the code is specific to the jimage -> jimage case. I think that can be left along too or just replaced with a better implementation for jrtfs. It would simplify this patch a little to leave this part out. Brian From hboehm at google.com Wed Jan 3 01:16:26 2018 From: hboehm at google.com (Hans Boehm) Date: Tue, 2 Jan 2018 17:16:26 -0800 Subject: FileOutputStream.getFD() vs finalization In-Reply-To: <3b973fb9-beac-b0cb-67ce-d61977dd6f6d@gmail.com> References: <3b973fb9-beac-b0cb-67ce-d61977dd6f6d@gmail.com> Message-ID: Thanks! I had missed the fact that FileDescriptor keeps a list of all owners. Apparently (1) closing a single File*Stream effectively closes all such Streams using the same FileDescriptor? And (2) whenever one such Stream is reachable they are all retained. Thus presumably none can be finalized until the last JNI instance method is called, which indeed should be enough to avoid the problem. (I'm not sure whether the JNI spec actually requires that, but certainly in the absence of cross-language optimization, it's sufficient.) Should (1) and (2) be documented? They both seem to potentially matter to users and, unless I missed something else, are not at all apparent from the current documentation. I do have to admit that I'm hazy on what the usage model here is intended to be, and why FileDescriptor is user visible. Hans On Sun, Dec 31, 2017 at 3:53 AM, Peter Levart wrote: > Hi, > > Maybe I'm missing something but... > > Hans Boehm je 28. 12. 2017 ob 21:32 napisal: > > The design of the getFD() calls in some Java *Stream classes seems > problematic, and doesn't seem cleanly fixable without a spec change. I > first noticed this in JDK 8 code, but Roger Riggs recent JDK 10 changes > also don't seem to fully address this specific problem, particularly since > the code paths remain more-or-less similar when a FOS subclass has a close > method. It probably remains easier to describe this in a JDK 8/9 setting, > so I'll do so initially. > > Close() on the *Stream closes the FD. The finalize() method on the Stream > (not the FD!) calls close(). Assuming JLS 12.6.2 finalization semantics, if > my program consists of: > > { > FileOutputStream f1 = new FileOutputStream("foo"); > FileDescriptor fd = f1.getFD(); > A: > FileOutputStream f2 = new FileOutputStream(fd); > } > > f2 may be associated with a closed fd, since f1 could have been finalized > at any pointer after its construction, in particular at point A. > > > I think there is no such problem here. Each stream (FileInputStream, > FileOutputStream and RandomAccessFile) keeps a reference to the > FileDescriptor (via 'fd' field) and FileDescriptor keeps a reference to all > associated streams (established in stream constructor via > FileDescriptor.attach(Closeable) where the Closeable instance is the > stream instance). So as long as 'fd' above is reachable, so is 'f1'. > > This is slightly aggravated by the fact that close() is implemented in a > way that doesn't guarantee reachability of the FileOutputStream while, or > before, it runs. Close() could easily be inlined, and the *Stream fields > promoted to registers. Thus even if you always call close() explicitly, I > think you are still affected by this problem. > > > File[Input|Output]Stream.close() is an instance method that among other > things, calls close0() native instance method which accesses field 'fd' in > native code via JNI. While close0() native instance method is being > executed, the stream instance is kept reachable (by the JNI spec). In fact, > all stream native methods that do make use of 'fd' are instance methods > that keep stream instance is reachable while being executed - making the > native resource access. > > So I think FileInputStream, FileOutputStream and RandomAccessFile are > examples of code where finalization was actually done right, because it is > consistently combined with native instance methods. Unlike direct NIO > buffers that keep native resource (long address) in a field and pass its > value around to other classes (Unsafe) not making sure that the buffer > instance which is tracked by Cleaner is kept reachable everywhere the > underlying native memory is being accessed. > > > Regards, Peter > > Fundamentally the result of getFD() is only valid as long as the underlying > *Stream is reachable, and that is not defined in a way that's > understandable by most programmers. That also makes this API un-Java-like, > in that the user has to worry about very subtle object lifetime rules. > > The only semi-plausible solution seems to involve judicious use of > ReachabilityFence() on FileOutputStreams after the getFD() result is no > longer needed. I personally don't think that's really a very plausible > request of the programmer. > > I think Roger's JDK 10 changes may help in the case in which there is not > an overridden close() method, since the cleanup action is registered on the > underlying FileDescriptor. But if a subclass overrides close(), I think > we're basically still in the same boat. For the reasons he already gave, > that seems hard to avoid. And again I think the problem applies even if > close() is explicitly called, since that doesn't really prevent > finalization of the Stream object. > > Has this been discussed anywhere? Opinions about how to deal with it? > > (This arose while going through the core libraries, looking for premature > finalization issues. So far, this one seems special, in that It really > seems to require an API change.) > > Hans > > > From Martin.Evans at kingfisher.com Tue Jan 2 17:55:18 2018 From: Martin.Evans at kingfisher.com (Evans, Martin) Date: Tue, 2 Jan 2018 17:55:18 +0000 Subject: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad In-Reply-To: References: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> Message-ID: Hi Volker, Happy new year! Do you have a rough idea as to when it will make it into the regular update? Kind regards, Martin -----Original Message----- From: Volker Simonis [mailto:volker.simonis at gmail.com] Sent: 22 December 2017 16:25 To: Evans, Martin Cc: Baesken, Matthias ; jdk8u-dev-request at openjdk.java.net; core-libs-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker ; build-dev Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad Hi Martin, I've just pushed the fix to jdk8u/jdk8u-dev [1] so it should be in the next regular 8u update (probably 8u172 ?). You can of course test it any time by building the tip of http://hg.openjdk.java.net/jdk8u/jdk8u-dev yourself :) Merry Christmas and a happy new year, Volker [1] http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/def07b5ce3be On Wed, Dec 20, 2017 at 11:18 AM, Evans, Martin wrote: > Many thanks guys, I look forward to testing. > > Regards, > > Martin > > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: 20 December 2017 09:33 > To: Baesken, Matthias > Cc: jdk8u-dev-request at openjdk.java.net; > core-libs-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; > Simonis, Volker ; Evans, Martin > ; build-dev > Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by > providing empty basic implementations of getSystemCpuLoad and > getProcessCpuLoad > > Hi Matthias, > > the change looks good! > I can sponsor it once we get the approval. > > Also forwarded to buil-dev for the minimal build change. > > Thank you and best regards, > Volker > > > On Wed, Dec 20, 2017 at 9:50 AM, Baesken, Matthias wrote: >> Hello , Mark reported this issue on AIX with OpenJDK8 : >> >>> >>>I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: >>> >>> java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D >>> getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 >>> at org/logstash/instrument/monitors/ProcessMonitor.java:40 >>> detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 >>> generate at >>> org/logstash/instrument/reports/ProcessReport.java:15 >>> >>> this is the line in logstash: >>> >>> this.cpuProcessPercent = >>> scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); >>> >>> https://github.com/elastic/logstash/blob/master/logstash-core/src/ma >>> i n/java/org/logstash/instrument/monitors/ProcessMonitor.java >>> >>> >>> Could anybody help steer me in the right direction? >>> >> >> This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . >> JDK8 is only affected , JDK9 and higher is not affected . >> >> Could I get a review for this change ? >> >> >> Change : >> >> http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ >> >> Bug : >> >> https://bugs.openjdk.java.net/browse/JDK-8193807 >> >> >> Thanks, Matthias > ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation. ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation. From wenqian.peiwq at alibaba-inc.com Wed Jan 3 02:09:26 2018 From: wenqian.peiwq at alibaba-inc.com (Wenqian Pei) Date: Wed, 03 Jan 2018 10:09:26 +0800 Subject: =?UTF-8?B?5Zue5aSN77yaUkZSOiA4MTk0MTU0IHBhdGNoIGZvciBjcmFzaCBhdCBGaWxlLmdldENhbm9u?= =?UTF-8?B?aWNhbFBhdGgoKQ==?= In-Reply-To: 5dc3e332-2c36-3261-e2f7-a7ecc532ce72@oracle.com References: , 5dc3e332-2c36-3261-e2f7-a7ecc532ce72@oracle.com Message-ID: <6e2f7f43-dcc0-4e1a-ad0d-6c52b25c5f71.wenqian.peiwq@alibaba-inc.com> Can you please review this patch? Thanks in advance! Wenqian Pei ------------------------------------------------------------------????Alan Bateman ?????2018?1?3?(???) 01:30???????(??) ; core-libs-dev ???????(??) ; ???(??) ????Re: RFR: 8194154 patch for crash at File.getCanonicalPath() On?25/12/2017?09:40,?Wenqian?Pei?wrote: >?Hi: > >?Bug:?https://bugs.openjdk.java.net/browse/JDK-8194154 > >?We?found?that?if?user?defines?-Duser.dir?like?"/home/a/b/c/",?jvm?will?crash?at?File.getCanonicalPath()?in?java?process?bootstrap?(before?invoking?user's?java?code).?The?native?implematation?of?canonicalize_md.c:collapsible(char?*names)?has?problem?in?processing?double?'/',?parameter?'names'?need?normalized?before?JNI_CALL. > user.dir?is?a?"read-only"?property?and?should?never?be?changed?on?the? command-line?or?in?a?running?VM?(it?breaks?APIs?in?other?areas?to?have? user.dir?be?different?to?the?actual?working?directory).?Someday?we?need? to?figure?out?how?to?enforce?this. In?the?mean-time,?the?runtime?shouldn't?crash?so?I?agree?it?should?be?fixed. -Alan. From peter.levart at gmail.com Wed Jan 3 07:01:18 2018 From: peter.levart at gmail.com (Peter Levart) Date: Wed, 3 Jan 2018 08:01:18 +0100 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: <862C913E-845F-4A8A-8090-0079F861B67B@gmail.com> References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> <862C913E-845F-4A8A-8090-0079F861B67B@gmail.com> Message-ID: Hi Steven, Steven Schlansker je 02. 01. 2018 ob 22:25?napisal: >> On Jan 2, 2018, at 8:35 AM, Alan Bateman wrote: >> >> On 29/12/2017 00:33, Steven Schlansker wrote: >>> Thanks for the discussion! >>> >>> So, it sounds like amending the message by default is going to be a non-starter -- but at least adding the information otherwise might be possible. >>> >> There are examples in other area where exceptions include detail information (for diagnostics purposes) when not running with a security manager. This may be something to look at here (and easy to try out too as it wouldn't require touching any native code either). > I like that! > > For now, I will proceed with improving my prototype to follow this suggestion -- unless there is a security manager, the exceptions are annotated. > > An alternate way to activate it, by configuring the platform logger e.g. "java.net.SocketException=TRACE", might be useful in case we want to allow end users to configure the feature explicitly and independent of security manager. > > Unfortunately I don't see how I can avoid changing the native code -- the exception message is initialized in native code, and by the time we call back to Java code, the necessary information is not passed in to the SocketException subclass constructor. So I may be misinterpreting your guidance here, but I'm not seeing it. > > I think I will hold off on adding Java level fields to the exception types. I expect if I do that, I will then have to get a spec change to add appropriate public API to make the data actually usable. Since the goal here is for log diagnostics as opposed to more structured data at the Java level, I'll avoid it. It also avoids any complications with regards to changing the serial format for such a common type. But I do think it means the work has to be done in the native code. > I haven't studied the actual code, but one thing to consider might be to create a java.net package-private exception type (unrelated to java.net.SocketException) with a constructor that takes necessary information (host, IP, port, but no message) and then throw this exception from native code instead of java.net.SocketException. On the Java side, you then wrap calls to native methods with try/catch and throw java.net.SocketException with appropriately formatted message instead (and with no cause). Moving formatting logic to Java might simplify native code and it's always good to keep native code simple (not having to deal with SecurityManager, system properties, logger(s) etc.). What do you think? Regards, Peter From Alan.Bateman at oracle.com Wed Jan 3 11:23:50 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Jan 2018 11:23:50 +0000 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> Message-ID: <9f81b074-5a62-2869-ad24-256eff0c0837@oracle.com> On 02/01/2018 23:43, Brian Burkhalter wrote: > : > >> So not clear to me that Files.copy methods needs to use this. > > So you are suggesting leaving the call to InputStream.transferTo() in > preference to IOSupport.copy()? Yes, I think these two should use transferTo. -Alan From Alan.Bateman at oracle.com Wed Jan 3 11:37:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Jan 2018 11:37:04 +0000 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: <862C913E-845F-4A8A-8090-0079F861B67B@gmail.com> References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> <862C913E-845F-4A8A-8090-0079F861B67B@gmail.com> Message-ID: <78354b1e-34e2-adc4-00f4-be85e2323dae@oracle.com> On 02/01/2018 21:25, Steven Schlansker wrote: > : > This would definitely be better than nothing! But it's still difficult, for example a common allocation pattern for us would be to assign networks to availability zones: > > 10.0.1.0/24 10.0.2.0/24 10.0.3.0/24 > > then if you pick the same last number for a well known service, you might end up with instances at 10.0.1.2, 10.0.2.2, 10.0.3.2 -- so depending on which octets are obscured you may end up with no useful information. > > The triggering incident for us was that one of our Amazon ELBs started responding incorrectly (blackholing data) -- > so when you resolve "my-elb-1.amazonaws.amazon.com" you'd get three different IP addresses, and depending on which one > is picked for the connect operation, you'll get all data blackholed. Socket/SocketChannel connect take a single address, not a hostname that is potentially mapped to multiple addresses. Maybe you mean the Socket constructors that takes a hostname and do the lookup before attempt to establish a connection? In any case, if the exception message includes the address/port details then it shouldn't matter which constructors are used. Also note that when you look at the non-blocking cases then you'll see the exception is thrown by finishConnect, not connect. This is nothing reason to handle this completely in java and avoid changes to the native code. -Alan From christoph.dreis at freenet.de Wed Jan 3 12:05:59 2018 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Wed, 3 Jan 2018 13:05:59 +0100 Subject: Fix typo in InetSocketAddress.getAddress() documentation Message-ID: <000001d3848b$382b78d0$a8826a70$@freenet.de> Hi, I just found a small typo in the documentation of InetSocketAddress.getAddress(). I'd be happy if the attached patch is reviewed and eventually sponsored. Cheers, Christoph ============== PATCH ================ diff -r 3a52333a5e57 src/java.base/share/classes/java/net/InetSocketAddress.java --- a/src/java.base/share/classes/java/net/InetSocketAddress.java Tue Jan 02 16:35:04 2018 -0500 +++ b/src/java.base/share/classes/java/net/InetSocketAddress.java Wed Jan 03 12:55:26 2018 +0100 @@ -317,10 +317,9 @@ } /** - * * Gets the {@code InetAddress}. * - * @return the InetAdress or {@code null} if it is unresolved. + * @return the InetAddress or {@code null} if it is unresolved. */ public final InetAddress getAddress() { return holder.getAddress(); From christoph.dreis at freenet.de Wed Jan 3 12:49:48 2018 From: christoph.dreis at freenet.de (Christoph Dreis) Date: Wed, 3 Jan 2018 13:49:48 +0100 Subject: Fix typo in documentation of ModuleDescriptor.read() variant Message-ID: <000101d38491$570c13d0$05243b70$@freenet.de> Hi, I just found a typo in the documentation of ModuleDescriptor.read(ByteBuffer bb, Supplier> packageFinder). I'd be happy if the attached patch is reviewed and eventually sponsored. Cheers, Christoph ====== PATCH ======= diff -r 3a52333a5e57 src/java.base/share/classes/java/lang/module/ModuleDescriptor.java --- a/src/java.base/share/classes/java/lang/module/ModuleDescriptor.java Tue Jan 02 16:35:04 2018 -0500 +++ b/src/java.base/share/classes/java/lang/module/ModuleDescriptor.java Wed Jan 03 13:42:27 2018 +0100 @@ -2521,7 +2521,7 @@ * the {@code packageFinder} throws an {@link UncheckedIOException} then * {@link IOException} cause will be re-thrown.

* - *

The module descriptor is read from the buffer stating at index + *

The module descriptor is read from the buffer starting at index * {@code p}, where {@code p} is the buffer's {@link ByteBuffer#position() * position} when this method is invoked. Upon return the buffer's position * will be equal to {@code p + n} where {@code n} is the number of bytes From Alan.Bateman at oracle.com Wed Jan 3 12:56:14 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Jan 2018 12:56:14 +0000 Subject: Fix typo in InetSocketAddress.getAddress() documentation In-Reply-To: <000001d3848b$382b78d0$a8826a70$@freenet.de> References: <000001d3848b$382b78d0$a8826a70$@freenet.de> Message-ID: <1e253538-020a-9376-4f15-53c1085ebb35@oracle.com> On 03/01/2018 12:05, Christoph Dreis wrote: > Hi, > > I just found a small typo in the documentation of > InetSocketAddress.getAddress(). > > I'd be happy if the attached patch is reviewed and eventually sponsored. > net-dev is the mailing list for the java.net code but what you have looks fine. -Alan From david.lloyd at redhat.com Wed Jan 3 13:47:00 2018 From: david.lloyd at redhat.com (David Lloyd) Date: Wed, 3 Jan 2018 07:47:00 -0600 Subject: Adding SocketChannel toString to connection exception messages In-Reply-To: References: <82A01DB9-7DB5-4D16-B890-FCB4306455DF@gmail.com> <160EE294-4802-4A5B-8EDE-1EB68EF329CE@gmail.com> <3874615e-a801-67d9-b466-bdc70fa9e835@oracle.com> <1b83507a-bded-4e2b-1818-cd51e94d5176@oracle.com> <8546d3d3-dd76-059c-dd28-03ca2f782c01@oracle.com> Message-ID: On Fri, Dec 29, 2017 at 11:15 AM, Chris Hegarty wrote: > On 29 Dec 2017, at 00:33, Steven Schlansker wrote: >> Thanks for the discussion! >> >> So, it sounds like amending the message by default is going to be a non-starter -- but at least adding the information otherwise might be possible. >> >> One possibility would be to add new fields to SocketException, for example an InetSocketAddress representing both the local and remote (if available). > > You would need to careful to not disclose resolved addresses to untrusted code. SocketException, since a subtype of IOException, can wind up in many places. > > Would you be proposing to add these new fields to the serial-form of SocketException? What behaviour do you envisage for deserializing instances from previous releases? This will have an impact of any potential API. This is an advantage to a setter-only message supplement method: if the supplementary field is set, the writeReplace method can swap it for a new instance with the combined message. -- - DML From volker.simonis at gmail.com Wed Jan 3 14:30:05 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 3 Jan 2018 15:30:05 +0100 Subject: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad In-Reply-To: References: <6838c67f5a7f44c0a48a79e8ac47e889@sap.com> Message-ID: Hi Martin, I'm afraid that's not easy to answer. The fix didn't made it into 8u162 which will be released in January 2018 [1]. It is targeted for 8u172 but there is no time-line for 8u172 yet [2]. The best you can do is to ask on the 8u-dev [3] mailing list or ask the people at AdoptOpenJDK (adoption-discuss at openjdk.java.net [4]) to merge the fix into their current builds. That would be actually a good smoke test for their tool chain :) Regards, Volker [1] http://openjdk.java.net/projects/jdk8u/releases/8u162.html [2] http://openjdk.java.net/projects/jdk8u/ [3] http://mail.openjdk.java.net/mailman/listinfo/jdk8u-dev [4] http://mail.openjdk.java.net/mailman/listinfo/adoption-discuss On Tue, Jan 2, 2018 at 6:55 PM, Evans, Martin wrote: > Hi Volker, > > Happy new year! > > Do you have a rough idea as to when it will make it into the regular update? > > Kind regards, > > Martin > > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: 22 December 2017 16:25 > To: Evans, Martin > Cc: Baesken, Matthias ; jdk8u-dev-request at openjdk.java.net; core-libs-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; Simonis, Volker ; build-dev > Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by providing empty basic implementations of getSystemCpuLoad and getProcessCpuLoad > > Hi Martin, > > I've just pushed the fix to jdk8u/jdk8u-dev [1] so it should be in the next regular 8u update (probably 8u172 ?). > > You can of course test it any time by building the tip of http://hg.openjdk.java.net/jdk8u/jdk8u-dev yourself :) > > Merry Christmas and a happy new year, > Volker > > [1] http://hg.openjdk.java.net/jdk8u/jdk8u-dev/jdk/rev/def07b5ce3be > > On Wed, Dec 20, 2017 at 11:18 AM, Evans, Martin wrote: >> Many thanks guys, I look forward to testing. >> >> Regards, >> >> Martin >> >> -----Original Message----- >> From: Volker Simonis [mailto:volker.simonis at gmail.com] >> Sent: 20 December 2017 09:33 >> To: Baesken, Matthias >> Cc: jdk8u-dev-request at openjdk.java.net; >> core-libs-dev at openjdk.java.net; ppc-aix-port-dev at openjdk.java.net; >> Simonis, Volker ; Evans, Martin >> ; build-dev >> Subject: Re: RFR [jdk8] : 8193807 : AIX: avoid UnsatisfiedLinkError by >> providing empty basic implementations of getSystemCpuLoad and >> getProcessCpuLoad >> >> Hi Matthias, >> >> the change looks good! >> I can sponsor it once we get the approval. >> >> Also forwarded to buil-dev for the minimal build change. >> >> Thank you and best regards, >> Volker >> >> >> On Wed, Dec 20, 2017 at 9:50 AM, Baesken, Matthias wrote: >>> Hello , Mark reported this issue on AIX with OpenJDK8 : >>> >>>> >>>>I'm getting an unsatisfied link error when using logstash on AIX but I suspect the issue is with AIX or the JRE, the exception is: >>>> >>>> java.lang.UnsatisfiedLinkError: sun.management.OperatingSystemImpl.getProcessCpuLoad()D >>>> getProcessCpuLoad at sun/management/OperatingSystemImpl.java:-2 >>>> at org/logstash/instrument/monitors/ProcessMonitor.java:40 >>>> detect at org/logstash/instrument/monitors/ProcessMonitor.java:79 >>>> generate at >>>> org/logstash/instrument/reports/ProcessReport.java:15 >>>> >>>> this is the line in logstash: >>>> >>>> this.cpuProcessPercent = >>>> scaleLoadToPercent(unixOsBean.getProcessCpuLoad()); >>>> >>>> https://github.com/elastic/logstash/blob/master/logstash-core/src/ma >>>> i n/java/org/logstash/instrument/monitors/ProcessMonitor.java >>>> >>>> >>>> Could anybody help steer me in the right direction? >>>> >>> >>> This fix addresses the missing getSystemCpuLoad and getProcessCpuLoad on AIX . >>> JDK8 is only affected , JDK9 and higher is not affected . >>> >>> Could I get a review for this change ? >>> >>> >>> Change : >>> >>> http://cr.openjdk.java.net/~mbaesken/webrevs/8193807/ >>> >>> Bug : >>> >>> https://bugs.openjdk.java.net/browse/JDK-8193807 >>> >>> >>> Thanks, Matthias >> ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation. > ------------ Kingfisher plc Registered Office: 3 Sheldon Square, Paddington, London W2 6PX Registered in England, Number 1664812 This e-mail is only intended for the person(s) to whom it is addressed and may contain confidential information. Unless stated to the contrary, any opinions or comments are personal to the writer and do not represent the official view of the company. If you have received this e-mail in error, please notify us immediately by reply e-mail and then delete this message from your system. Please do not copy it or use it for any purpose, or disclose its contents to any other person. Thank you for your co-operation. From vitalyd at gmail.com Wed Jan 3 16:56:23 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 3 Jan 2018 11:56:23 -0500 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay Message-ID: Hi all, Consider the following stacktraces of the main app thread and a background thread: "main" #1 prio=5 os_prio=0 tid=0x0000000001bfd000 nid=0x1958 waiting for monitor entry [0x00002b98ceba3000] java.lang.Thread.State: BLOCKED (on object monitor) at java.lang.Runtime.loadLibrary0(Runtime.java:862) - waiting to lock <0x00000007bf834b20> (a java.lang.Runtime) at java.lang.System.loadLibrary(System.java:1122) at sun.nio.fs.UnixNativeDispatcher$1.run( UnixNativeDispatcher.java:573) at sun.nio.fs.UnixNativeDispatcher$1.run( UnixNativeDispatcher.java:571) at java.security.AccessController.doPrivileged(Native Method) at sun.nio.fs.UnixNativeDispatcher.( UnixNativeDispatcher.java:571) at sun.nio.fs.UnixFileSystem.(UnixFileSystem.java:67) at sun.nio.fs.LinuxFileSystem.< init>(LinuxFileSystem.java:39) at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( LinuxFileSystemProvider.java:46) at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( LinuxFileSystemProvider.java:39) at sun.nio.fs.UnixFileSystemProvider.( UnixFileSystemProvider.java:56) at sun.nio.fs.LinuxFileSystemProvider. (LinuxFileSystemProvider.java:41) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorI mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor. newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at sun.nio.fs.DefaultFileSystemProvider.createProvider( DefaultFileSystemProvider.java:48) at sun.nio.fs.DefaultFileSystemProvider.create( DefaultFileSystemProvider.java:63) at java.nio.file.FileSystems$DefaultFileSystemHolder. getDefaultProvider(FileSystems.java:108) at java.nio.file.FileSystems$DefaultFileSystemHolder. access$000(FileSystems.java:89) at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run( FileSystems.java:98) at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run( FileSystems.java:96) at java.security.AccessController.doPrivileged(Native Method) at java.nio.file.FileSystems$DefaultFileSystemHolder. defaultFileSystem(FileSystems.java:96) at java.nio.file.FileSystems$DefaultFileSystemHolder.< clinit>(FileSystems.java:90) at java.nio.file.FileSystems.getDefault(FileSystems.java: 176) at java.nio.file.Paths.get(Paths.java:84) at .ServiceLoader. setup(ServiceLoader.java:446) at .MetadataAdminLoader.main( MetadataAdminLoader.java:52) "shardDbExecutor-0-pool-0" #33 prio=5 os_prio=0 tid=0x0000000001ebf000 nid=0x1d01 in Object.wait() [0x00002b991f941000] java.lang.Thread.State: RUNNABLE at java.nio.file.FileSystems.getDefault(FileSystems.java: 176) at java.nio.file.Paths.get(Paths.java:138) at sun.misc.Launcher$ExtClassLoader.findLibrary( Launcher.java:224) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1830) at java.lang.Runtime.loadLibrary0(Runtime.java:870) - locked <0x00000007bf834b20> (a java.lang.Runtime) at java.lang.System.loadLibrary(System.java:1122) at sun.security.ec.SunEC$1.run(SunEC.java:60) at sun.security.ec.SunEC$1.run(SunEC.java:58) at java.security.AccessController.doPrivileged(Native Method) at sun.security.ec.SunEC.(SunEC.java:58) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorI mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor. newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at sun.security.jca.ProviderConfig$2.run( ProviderConfig.java:221) at sun.security.jca.ProviderConfig$2.run( ProviderConfig.java:206) at java.security.AccessController.doPrivileged(Native Method) at sun.security.jca.ProviderConfig.doLoadProvider( ProviderConfig.java:206) at sun.security.jca.ProviderConfig.getProvider( ProviderConfig.java:187) - locked <0x00000007bea03f48> (a sun.security.jca. ProviderConfig) at sun.security.jca.ProviderList. getProvider(ProviderList.java:233) at sun.security.jca.ProviderList. getService(ProviderList.java:331) at sun.security.jca.GetInstance. getInstance(GetInstance.java:157) at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156) at com.microsoft.sqlserver.jdbc. TDSChannel.enableSSL(IOBuffer.java:1658) at com.microsoft.sqlserver.jdbc.SQLServerConnection. connectHelper(SQLServerConnection.java:1976) at com.microsoft.sqlserver.jdbc.SQLServerConnection.login( SQLServerConnection.java:1627) at com.microsoft.sqlserver.jdbc.SQLServerConnection. connectInternal(SQLServerConnection.java:1458) at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect( SQLServerConnection.java:772) at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect( SQLServerDriver.java:1168) at java.sql.DriverManager.getConnection(DriverManager. java:664) at java.sql.DriverManager.getConnection(DriverManager. java:208) at .ConnectionPool.createGoodConnection( ConnectionPool.java:565) at .ConnectionPool.createNewConnection( ConnectionPool.java:519) at .ConnectionPool. getConnection(ConnectionPool.java:407) at .StatementCachingConnectionImpl .setupConnection(StatementCachingConnectionImpl.java:94) at .StatementCachingConnectionImpl.( StatementCachingConnectionImpl.java:54) at .TestConnectionFactories$ FaultInjectingConnection.(TestConnectionFactories.java:90) at .TestConnectionFactories.lambda$static$3( TestConnectionFactories.java:33) at .TestConnectionFactories$$ Lambda$12/1738236591.newConnection(Unknown Source) at .SedaExecutor. setupConnections(SedaExecutor.java:130) at .SedaExecutor$SedaThreadFactory.lambda$ newThread$0(SedaExecutor.java:165) at .SedaExecutor$SedaThreadFactory$$Lambda$22/1392419022.run(Unknown Source) at java.lang.Thread.run(Thread.java:748) The application intermittently hangs on startup, and the above callstacks are present. It seems like there's a deadlock due to concurrent class loading and default FileSystem initialization (which itself triggers class loading here). I wasn't able to find any existing JBS entries reporting anything similar. Am I seeing this right? Is this a known issue? Thanks From vitalyd at gmail.com Wed Jan 3 16:57:28 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 3 Jan 2018 11:57:28 -0500 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: References: Message-ID: Sorry, I should've mentioned that this is 8u152. On Wed, Jan 3, 2018 at 11:56 AM, Vitaly Davidovich wrote: > Hi all, > > Consider the following stacktraces of the main app thread and a background > thread: > > "main" #1 prio=5 os_prio=0 tid=0x0000000001bfd000 nid=0x1958 waiting for > monitor entry [0x00002b98ceba3000] > > java.lang.Thread.State: BLOCKED (on object monitor) > > at java.lang.Runtime.loadLibrary0(Runtime.java:862) > > - waiting to lock <0x00000007bf834b20> (a > java.lang.Runtime) > > at java.lang.System.loadLibrary(System.java:1122) > > at sun.nio.fs.UnixNativeDispatche > r$1.run(UnixNativeDispatcher.java:573) > > at sun.nio.fs.UnixNativeDispatche > r$1.run(UnixNativeDispatcher.java:571) > > at java.security.AccessController.doPrivileged(Native > Method) > > at sun.nio.fs.UnixNativeDispatche > r.(UnixNativeDispatcher.java:571) > > at sun.nio.fs.UnixFileSystem. t>(UnixFileSystem.java:67) > > at sun.nio.fs.LinuxFileSystem. it>(LinuxFileSystem.java:39) > > at sun.nio.fs.LinuxFileSystemProv > ider.newFileSystem(LinuxFileSystemProvider.java:46) > > at sun.nio.fs.LinuxFileSystemProv > ider.newFileSystem(LinuxFileSystemProvider.java:39) > > at sun.nio.fs.UnixFileSystemProvi > der.(UnixFileSystemProvider.java:56) > > at sun.nio.fs.LinuxFileSystemProvider.( > LinuxFileSystemProvider.java:41) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at sun.reflect.NativeConstructorA > ccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at sun.reflect.DelegatingConstruc > torAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor. > newInstance(Constructor.java:423) > > at java.lang.Class.newInstance(Class.java:442) > > at sun.nio.fs.DefaultFileSystemPr > ovider.createProvider(DefaultFileSystemProvider.java:48) > > at sun.nio.fs.DefaultFileSystemPr > ovider.create(DefaultFileSystemProvider.java:63) > > at java.nio.file.FileSystems$Defa > ultFileSystemHolder.getDefaultProvider(FileSystems.java:108) > > at java.nio.file.FileSystems$Defa > ultFileSystemHolder.access$000(FileSystems.java:89) > > at java.nio.file.FileSystems$Defa > ultFileSystemHolder$1.run(FileSystems.java:98) > > at java.nio.file.FileSystems$Defa > ultFileSystemHolder$1.run(FileSystems.java:96) > > at java.security.AccessController.doPrivileged(Native > Method) > > at java.nio.file.FileSystems$Defa > ultFileSystemHolder.defaultFileSystem(FileSystems.java:96) > > at java.nio.file.FileSystems$Defa > ultFileSystemHolder.(FileSystems.java:90) > > at java.nio.file.FileSystems.getD > efault(FileSystems.java:176) > > at java.nio.file.Paths.get(Paths.java:84) > > at .ServiceLoader.s > etup(ServiceLoader.java:446) > > at .MetadataAdminLo > ader.main(MetadataAdminLoader.java:52) > > > > > > "shardDbExecutor-0-pool-0" #33 prio=5 os_prio=0 tid=0x0000000001ebf000 > nid=0x1d01 in Object.wait() [0x00002b991f941000] > > java.lang.Thread.State: RUNNABLE > > at java.nio.file.FileSystems.getD > efault(FileSystems.java:176) > > at java.nio.file.Paths.get(Paths.java:138) > > at sun.misc.Launcher$ExtClassLoad > er.findLibrary(Launcher.java:224) > > at java.lang.ClassLoader.loadLibr > ary(ClassLoader.java:1830) > > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > > - locked <0x00000007bf834b20> (a java.lang.Runtime) > > at java.lang.System.loadLibrary(System.java:1122) > > at sun.security.ec.SunEC$1.run(SunEC.java:60) > > at sun.security.ec.SunEC$1.run(SunEC.java:58) > > at java.security.AccessController.doPrivileged(Native > Method) > > at sun.security.ec.SunEC.(SunEC.java:58) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at sun.reflect.NativeConstructorA > ccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) > > at sun.reflect.DelegatingConstruc > torAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor. > newInstance(Constructor.java:423) > > at java.lang.Class.newInstance(Class.java:442) > > at sun.security.jca.ProviderConfi > g$2.run(ProviderConfig.java:221) > > at sun.security.jca.ProviderConfi > g$2.run(ProviderConfig.java:206) > > at java.security.AccessController.doPrivileged(Native > Method) > > at sun.security.jca.ProviderConfi > g.doLoadProvider(ProviderConfig.java:206) > > at sun.security.jca.ProviderConfi > g.getProvider(ProviderConfig.java:187) > > - locked <0x00000007bea03f48> (a > sun.security.jca.ProviderConfig) > > at sun.security.jca.ProviderList. > getProvider(ProviderList.java:233) > > at sun.security.jca.ProviderList. > getService(ProviderList.java:331) > > at sun.security.jca.GetInstance.g > etInstance(GetInstance.java:157) > > at javax.net.ssl.SSLContext.getIn > stance(SSLContext.java:156) > > at com.microsoft.sqlserver.jdbc.T > DSChannel.enableSSL(IOBuffer.java:1658) > > at com.microsoft.sqlserver.jdbc.S > QLServerConnection.connectHelper(SQLServerConnection.java:1976) > > at com.microsoft.sqlserver.jdbc.S > QLServerConnection.login(SQLServerConnection.java:1627) > > at com.microsoft.sqlserver.jdbc.S > QLServerConnection.connectInternal(SQLServerConnection.java:1458) > > at com.microsoft.sqlserver.jdbc.S > QLServerConnection.connect(SQLServerConnection.java:772) > > at com.microsoft.sqlserver.jdbc.S > QLServerDriver.connect(SQLServerDriver.java:1168) > > at java.sql.DriverManager.getConn > ection(DriverManager.java:664) > > at java.sql.DriverManager.getConn > ection(DriverManager.java:208) > > at .ConnectionPool. > createGoodConnection(ConnectionPool.java:565) > > at .ConnectionPool. > createNewConnection(ConnectionPool.java:519) > > at .ConnectionPool. > getConnection(ConnectionPool.java:407) > > at .StatementCachingConnectionImpl. > setupConnection(StatementCachingConnectionImpl.java:94) > > at .StatementCachin > gConnectionImpl.(StatementCachingConnectionImpl.java:54) > > at .TestConnectionF > actories$FaultInjectingConnection.(TestConnectionFactories.java:90) > > at .TestConnectionF > actories.lambda$static$3(TestConnectionFactories.java:33) > > at .TestConnectionFactories$$Lambda$12/ > 1738236591.newConnection(Unknown Source) > > at .SedaExecutor.se > tupConnections(SedaExecutor.java:130) > > at .SedaExecutor$Se > daThreadFactory.lambda$newThread$0(SedaExecutor.java:165) > > at .SedaExecutor$Se > daThreadFactory$$Lambda$22/1392419022.run(Unknown Source) > > at java.lang.Thread.run(Thread.java:748) > > > The application intermittently hangs on startup, and the above callstacks > are present. It seems like there's a deadlock due to concurrent class > loading and default FileSystem initialization (which itself triggers class > loading here). > > I wasn't able to find any existing JBS entries reporting anything > similar. Am I seeing this right? Is this a known issue? > > Thanks > From Alan.Bateman at oracle.com Wed Jan 3 17:00:32 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 3 Jan 2018 17:00:32 +0000 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: References: Message-ID: The stack trace looks like JDK 8 or older. The initialization has changed quite a bit in JDK 9+ and would be interesting to see if you can duplicate it there. -Alan On 03/01/2018 16:56, Vitaly Davidovich wrote: > Hi all, > > Consider the following stacktraces of the main app thread and a background > thread: > > "main" #1 prio=5 os_prio=0 tid=0x0000000001bfd000 nid=0x1958 waiting for > monitor entry [0x00002b98ceba3000] > > java.lang.Thread.State: BLOCKED (on object monitor) > > at java.lang.Runtime.loadLibrary0(Runtime.java:862) > > - waiting to lock <0x00000007bf834b20> (a java.lang.Runtime) > > at java.lang.System.loadLibrary(System.java:1122) > > at sun.nio.fs.UnixNativeDispatcher$1.run( > UnixNativeDispatcher.java:573) > > at sun.nio.fs.UnixNativeDispatcher$1.run( > UnixNativeDispatcher.java:571) > > at java.security.AccessController.doPrivileged(Native > Method) > > at sun.nio.fs.UnixNativeDispatcher.( > UnixNativeDispatcher.java:571) > > at sun.nio.fs.UnixFileSystem.(UnixFileSystem.java:67) > > at sun.nio.fs.LinuxFileSystem.< > init>(LinuxFileSystem.java:39) > > at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( > LinuxFileSystemProvider.java:46) > > at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( > LinuxFileSystemProvider.java:39) > > at sun.nio.fs.UnixFileSystemProvider.( > UnixFileSystemProvider.java:56) > > at sun.nio.fs.LinuxFileSystemProvider. > (LinuxFileSystemProvider.java:41) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( > NativeConstructorAccessorImpl.java:62) > > at sun.reflect.DelegatingConstructorAccessorI > mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor. > newInstance(Constructor.java:423) > > at java.lang.Class.newInstance(Class.java:442) > > at sun.nio.fs.DefaultFileSystemProvider.createProvider( > DefaultFileSystemProvider.java:48) > > at sun.nio.fs.DefaultFileSystemProvider.create( > DefaultFileSystemProvider.java:63) > > at java.nio.file.FileSystems$DefaultFileSystemHolder. > getDefaultProvider(FileSystems.java:108) > > at java.nio.file.FileSystems$DefaultFileSystemHolder. > access$000(FileSystems.java:89) > > at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run( > FileSystems.java:98) > > at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run( > FileSystems.java:96) > > at java.security.AccessController.doPrivileged(Native > Method) > > at java.nio.file.FileSystems$DefaultFileSystemHolder. > defaultFileSystem(FileSystems.java:96) > > at java.nio.file.FileSystems$DefaultFileSystemHolder.< > clinit>(FileSystems.java:90) > > at java.nio.file.FileSystems.getDefault(FileSystems.java: > 176) > > at java.nio.file.Paths.get(Paths.java:84) > > at .ServiceLoader. > setup(ServiceLoader.java:446) > > at .MetadataAdminLoader.main( > MetadataAdminLoader.java:52) > > > > > > "shardDbExecutor-0-pool-0" #33 prio=5 os_prio=0 tid=0x0000000001ebf000 > nid=0x1d01 in Object.wait() [0x00002b991f941000] > > java.lang.Thread.State: RUNNABLE > > at java.nio.file.FileSystems.getDefault(FileSystems.java: > 176) > > at java.nio.file.Paths.get(Paths.java:138) > > at sun.misc.Launcher$ExtClassLoader.findLibrary( > Launcher.java:224) > > at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1830) > > at java.lang.Runtime.loadLibrary0(Runtime.java:870) > > - locked <0x00000007bf834b20> (a java.lang.Runtime) > > at java.lang.System.loadLibrary(System.java:1122) > > at sun.security.ec.SunEC$1.run(SunEC.java:60) > > at sun.security.ec.SunEC$1.run(SunEC.java:58) > > at java.security.AccessController.doPrivileged(Native > Method) > > at sun.security.ec.SunEC.(SunEC.java:58) > > at > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > at sun.reflect.NativeConstructorAccessorImpl.newInstance( > NativeConstructorAccessorImpl.java:62) > > at sun.reflect.DelegatingConstructorAccessorI > mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor. > newInstance(Constructor.java:423) > > at java.lang.Class.newInstance(Class.java:442) > > at sun.security.jca.ProviderConfig$2.run( > ProviderConfig.java:221) > > at sun.security.jca.ProviderConfig$2.run( > ProviderConfig.java:206) > > at java.security.AccessController.doPrivileged(Native > Method) > > at sun.security.jca.ProviderConfig.doLoadProvider( > ProviderConfig.java:206) > > at sun.security.jca.ProviderConfig.getProvider( > ProviderConfig.java:187) > > - locked <0x00000007bea03f48> (a sun.security.jca. > ProviderConfig) > > at sun.security.jca.ProviderList. > getProvider(ProviderList.java:233) > > at sun.security.jca.ProviderList. > getService(ProviderList.java:331) > > at sun.security.jca.GetInstance. > getInstance(GetInstance.java:157) > > at javax.net.ssl.SSLContext.getInstance(SSLContext.java:156) > > at com.microsoft.sqlserver.jdbc. > TDSChannel.enableSSL(IOBuffer.java:1658) > > at com.microsoft.sqlserver.jdbc.SQLServerConnection. > connectHelper(SQLServerConnection.java:1976) > > at com.microsoft.sqlserver.jdbc.SQLServerConnection.login( > SQLServerConnection.java:1627) > > at com.microsoft.sqlserver.jdbc.SQLServerConnection. > connectInternal(SQLServerConnection.java:1458) > > at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect( > SQLServerConnection.java:772) > > at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect( > SQLServerDriver.java:1168) > > at java.sql.DriverManager.getConnection(DriverManager. > java:664) > > at java.sql.DriverManager.getConnection(DriverManager. > java:208) > > at .ConnectionPool.createGoodConnection( > ConnectionPool.java:565) > > at .ConnectionPool.createNewConnection( > ConnectionPool.java:519) > > at .ConnectionPool. > getConnection(ConnectionPool.java:407) > > at .StatementCachingConnectionImpl > .setupConnection(StatementCachingConnectionImpl.java:94) > > at .StatementCachingConnectionImpl.( > StatementCachingConnectionImpl.java:54) > > at .TestConnectionFactories$ > FaultInjectingConnection.(TestConnectionFactories.java:90) > > at .TestConnectionFactories.lambda$static$3( > TestConnectionFactories.java:33) > > at .TestConnectionFactories$$ > Lambda$12/1738236591.newConnection(Unknown Source) > > at .SedaExecutor. > setupConnections(SedaExecutor.java:130) > > at .SedaExecutor$SedaThreadFactory.lambda$ > newThread$0(SedaExecutor.java:165) > > at > .SedaExecutor$SedaThreadFactory$$Lambda$22/1392419022.run(Unknown > Source) > > at java.lang.Thread.run(Thread.java:748) > > > The application intermittently hangs on startup, and the above callstacks > are present. It seems like there's a deadlock due to concurrent class > loading and default FileSystem initialization (which itself triggers class > loading here). > > I wasn't able to find any existing JBS entries reporting anything similar. > Am I seeing this right? Is this a known issue? > > Thanks From vitalyd at gmail.com Wed Jan 3 17:13:52 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Wed, 3 Jan 2018 12:13:52 -0500 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: References: Message-ID: On Wed, Jan 3, 2018 at 12:00 PM, Alan Bateman wrote: > The stack trace looks like JDK 8 or older. The initialization has changed > quite a bit in JDK 9+ and would be interesting to see if you can duplicate > it there. > Yes - it's 8u152 as I mentioned in the follow-up email (sorry, should've mentioned that initially). Trying this on JDK9 is a bit tricky as the code (and its dependencies) in question isn't easily runnable on 9 without quite a bit of work. Do you agree with the deadlock assessment though? That's what it looks like to me but wanted to make sure I'm not missing anything. Thanks Alan > > -Alan > > > On 03/01/2018 16:56, Vitaly Davidovich wrote: > >> Hi all, >> >> Consider the following stacktraces of the main app thread and a background >> thread: >> >> "main" #1 prio=5 os_prio=0 tid=0x0000000001bfd000 nid=0x1958 waiting for >> monitor entry [0x00002b98ceba3000] >> >> java.lang.Thread.State: BLOCKED (on object monitor) >> >> at java.lang.Runtime.loadLibrary0(Runtime.java:862) >> >> - waiting to lock <0x00000007bf834b20> (a >> java.lang.Runtime) >> >> at java.lang.System.loadLibrary(System.java:1122) >> >> at sun.nio.fs.UnixNativeDispatcher$1.run( >> UnixNativeDispatcher.java:573) >> >> at sun.nio.fs.UnixNativeDispatcher$1.run( >> UnixNativeDispatcher.java:571) >> >> at java.security.AccessController.doPrivileged(Native >> Method) >> >> at sun.nio.fs.UnixNativeDispatcher.( >> UnixNativeDispatcher.java:571) >> >> at sun.nio.fs.UnixFileSystem.> t>(UnixFileSystem.java:67) >> >> at sun.nio.fs.LinuxFileSystem.< >> init>(LinuxFileSystem.java:39) >> >> at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( >> LinuxFileSystemProvider.java:46) >> >> at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( >> LinuxFileSystemProvider.java:39) >> >> at sun.nio.fs.UnixFileSystemProvider.( >> UnixFileSystemProvider.java:56) >> >> at sun.nio.fs.LinuxFileSystemProvider. >> (LinuxFileSystemProvider.java:41) >> >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> >> at sun.reflect.NativeConstructorA >> ccessorImpl.newInstance( >> NativeConstructorAccessorImpl.java:62) >> >> at sun.reflect.DelegatingConstructorAccessorI >> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> >> at java.lang.reflect.Constructor. >> newInstance(Constructor.java:423) >> >> at java.lang.Class.newInstance(Class.java:442) >> >> at sun.nio.fs.DefaultFileSystemProvider.createProvider( >> DefaultFileSystemProvider.java:48) >> >> at sun.nio.fs.DefaultFileSystemProvider.create( >> DefaultFileSystemProvider.java:63) >> >> at java.nio.file.FileSystems$DefaultFileSystemHolder. >> getDefaultProvider(FileSystems.java:108) >> >> at java.nio.file.FileSystems$DefaultFileSystemHolder. >> access$000(FileSystems.java:89) >> >> at java.nio.file.FileSystems$Defa >> ultFileSystemHolder$1.run( >> FileSystems.java:98) >> >> at java.nio.file.FileSystems$Defa >> ultFileSystemHolder$1.run( >> FileSystems.java:96) >> >> at java.security.AccessController.doPrivileged(Native >> Method) >> >> at java.nio.file.FileSystems$DefaultFileSystemHolder. >> defaultFileSystem(FileSystems.java:96) >> >> at java.nio.file.FileSystems$DefaultFileSystemHolder.< >> clinit>(FileSystems.java:90) >> >> at java.nio.file.FileSystems.getD >> efault(FileSystems.java: >> 176) >> >> at java.nio.file.Paths.get(Paths.java:84) >> >> at .ServiceLoader. >> setup(ServiceLoader.java:446) >> >> at .MetadataAdminLoader.main( >> MetadataAdminLoader.java:52) >> >> >> >> >> >> "shardDbExecutor-0-pool-0" #33 prio=5 os_prio=0 tid=0x0000000001ebf000 >> nid=0x1d01 in Object.wait() [0x00002b991f941000] >> >> java.lang.Thread.State: RUNNABLE >> >> at java.nio.file.FileSystems.getD >> efault(FileSystems.java: >> 176) >> >> at java.nio.file.Paths.get(Paths.java:138) >> >> at sun.misc.Launcher$ExtClassLoader.findLibrary( >> Launcher.java:224) >> >> at java.lang.ClassLoader.loadLibr >> ary(ClassLoader.java:1830) >> >> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >> >> - locked <0x00000007bf834b20> (a java.lang.Runtime) >> >> at java.lang.System.loadLibrary(System.java:1122) >> >> at sun.security.ec.SunEC$1.run(SunEC.java:60) >> >> at sun.security.ec.SunEC$1.run(SunEC.java:58) >> >> at java.security.AccessController.doPrivileged(Native >> Method) >> >> at sun.security.ec.SunEC.(SunEC.java:58) >> >> at >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >> Method) >> >> at sun.reflect.NativeConstructorA >> ccessorImpl.newInstance( >> NativeConstructorAccessorImpl.java:62) >> >> at sun.reflect.DelegatingConstructorAccessorI >> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >> >> at java.lang.reflect.Constructor. >> newInstance(Constructor.java:423) >> >> at java.lang.Class.newInstance(Class.java:442) >> >> at sun.security.jca.ProviderConfig$2.run( >> ProviderConfig.java:221) >> >> at sun.security.jca.ProviderConfig$2.run( >> >> ProviderConfig.java:206) >> >> at java.security.AccessController.doPrivileged(Native >> Method) >> >> at sun.security.jca.ProviderConfig.doLoadProvider( >> ProviderConfig.java:206) >> >> at sun.security.jca.ProviderConfig.getProvider( >> ProviderConfig.java:187) >> >> - locked <0x00000007bea03f48> (a sun.security.jca. >> ProviderConfig) >> >> at sun.security.jca.ProviderList. >> getProvider(ProviderList.java:233) >> >> at sun.security.jca.ProviderList. >> getService(ProviderList.java:331) >> >> at sun.security.jca.GetInstance. >> getInstance(GetInstance.java:157) >> >> at javax.net.ssl.SSLContext.getIn >> stance(SSLContext.java:156) >> >> at com.microsoft.sqlserver.jdbc. >> TDSChannel.enableSSL(IOBuffer.java:1658) >> >> at com.microsoft.sqlserver.jdbc.SQLServerConnection. >> connectHelper(SQLServerConnection.java:1976) >> >> at com.microsoft.sqlserver.jdbc.S >> QLServerConnection.login( >> SQLServerConnection.java:1627) >> >> at com.microsoft.sqlserver.jdbc.SQLServerConnection. >> connectInternal(SQLServerConnection.java:1458) >> >> at com.microsoft.sqlserver.jdbc.S >> QLServerConnection.connect( >> SQLServerConnection.java:772) >> >> at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect( >> SQLServerDriver.java:1168) >> >> at java.sql.DriverManager.getConnection(DriverManager. >> java:664) >> >> at java.sql.DriverManager.getConnection(DriverManager. >> java:208) >> >> at .ConnectionPool.createGoodConnection( >> ConnectionPool.java:565) >> >> at .ConnectionPool.createNewConnection( >> ConnectionPool.java:519) >> >> at .ConnectionPool. >> getConnection(ConnectionPool.java:407) >> >> at .StatementCachingConnectionImpl >> .setupConnection(StatementCachingConnectionImpl.java:94) >> >> at .StatementCachingConnectionImpl.( >> StatementCachingConnectionImpl.java:54) >> >> at .TestConnectionFactories$ >> FaultInjectingConnection.(TestConnectionFactories.java:90) >> >> at .TestConnectionF >> actories.lambda$static$3( >> TestConnectionFactories.java:33) >> >> at .TestConnectionFactories$$ >> Lambda$12/1738236591.newConnection(Unknown Source) >> >> at .SedaExecutor. >> setupConnections(SedaExecutor.java:130) >> >> at .SedaExecutor$SedaThreadFactory.lambda$ >> newThread$0(SedaExecutor.java:165) >> >> at >> .SedaExecutor$SedaThreadFactory$$Lambda$22/139 >> 2419022.run(Unknown >> Source) >> >> at java.lang.Thread.run(Thread.java:748) >> >> >> The application intermittently hangs on startup, and the above callstacks >> are present. It seems like there's a deadlock due to concurrent class >> loading and default FileSystem initialization (which itself triggers class >> loading here). >> >> I wasn't able to find any existing JBS entries reporting anything similar. >> Am I seeing this right? Is this a known issue? >> >> Thanks >> > > From adam.petcher at oracle.com Wed Jan 3 19:36:20 2018 From: adam.petcher at oracle.com (Adam Petcher) Date: Wed, 3 Jan 2018 14:36:20 -0500 Subject: API review for X25519/X448 In-Reply-To: <9425dc17-1099-9544-d314-95ab9d56bc47@oracle.com> References: <9425dc17-1099-9544-d314-95ab9d56bc47@oracle.com> Message-ID: <9a46a28b-1fbe-13a2-8b81-507e9438667d@oracle.com> +core-libs-dev (to get some additional API guidance) On 1/3/2018 11:26 AM, Adam Petcher wrote: > Now that the JEP[1] for X25519/X448 key agreement is a candidate, we > can proceed with the API and specification review. Please review the > proposed API spec[2] and provide comments by the end of Saturday,? > January 13, anywhere on earth. At that point, I will combine your > feedback with the initial feedback from the CSR group[3] and submit > the API for final review by the CSR. > > The only significant change to the API since our last discussion[4] is > that I changed the names of the key specs and interfaces from "XDH..." > to "XEC..." This makes them more general and reusable in things like > XEdDSA[5] and other non-Diffie-Hellman cryptosystems based on the > representations/operations defined in RFC 7748[6]. > > [1] http://openjdk.java.net/jeps/324 > [2] https://bugs.openjdk.java.net/browse/JDK-8189806 > [3] https://wiki.openjdk.java.net/display/csr/Main > [4] > http://mail.openjdk.java.net/pipermail/security-dev/2017-September/016325.html > [5] https://signal.org/docs/specifications/xeddsa/ > [6] https://tools.ietf.org/html/rfc7748 > From martinrb at google.com Wed Jan 3 21:40:45 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 3 Jan 2018 13:40:45 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache Message-ID: Here at Google we tried to fix JDK-8145371 because it looked like it was causing rare problems in production. But after looking at the code, I'm not sure... Anyways, http://cr.openjdk.java.net/~martin/webrevs/jdk/MethodHandle.form/ https://bugs.openjdk.java.net/browse/JDK-8145371 Copying to a local in customize() must be an improvement. The UNSAFE publication in updateForm looks fishy, as does that comment in MethodHandleImpl.java. Is there something the fullFence() is supposed to do that isn't taken care of by putObjectVolatile ? Feel free to take ownership of this bug from me. --- a/src/java.base/share/classes/java/lang/invoke/MethodHandle.java +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandle.java @@ -1660,13 +1660,13 @@ assert(newForm.customized == null || newForm.customized == this); if (form == newForm) return; newForm.prepare(); // as in MethodHandle. - UNSAFE.putObject(this, FORM_OFFSET, newForm); - UNSAFE.fullFence(); + UNSAFE.putObjectVolatile(this, FORM_OFFSET, newForm); } /** Craft a LambdaForm customized for this particular MethodHandle */ /*non-public*/ void customize() { + final LambdaForm form = this.form; if (form.customized == null) { LambdaForm newForm = form.customize(this); updateForm(newForm); diff --git a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java --- a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java @@ -909,7 +909,7 @@ int c = count; maybeCustomizeTarget(); if (c <= 1) { - // Try to limit number of updates. MethodHandle.updateForm() doesn't guarantee LF update visibility. + // Try to limit number of updates. if (isCounting) { isCounting = false; return true; From martinrb at google.com Wed Jan 3 21:51:45 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 3 Jan 2018 13:51:45 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: Let's try again without mail bounces ... On Wed, Jan 3, 2018 at 1:40 PM, Martin Buchholz wrote: > Here at Google we tried to fix JDK-8145371 because it looked like it was > causing rare problems in production. But after looking at the code, I'm > not sure... Anyways, > > http://cr.openjdk.java.net/~martin/webrevs/jdk/MethodHandle.form/ > https://bugs.openjdk.java.net/browse/JDK-8145371 > Copying to a local in customize() must be an improvement. > The UNSAFE publication in updateForm looks fishy, as does that comment > in MethodHandleImpl.java. Is there something the fullFence() is supposed > to do that isn't taken care of by putObjectVolatile ? > > Feel free to take ownership of this bug from me. > > --- a/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > @@ -1660,13 +1660,13 @@ > assert(newForm.customized == null || newForm.customized == this); > if (form == newForm) return; > newForm.prepare(); // as in MethodHandle. > - UNSAFE.putObject(this, FORM_OFFSET, newForm); > - UNSAFE.fullFence(); > + UNSAFE.putObjectVolatile(this, FORM_OFFSET, newForm); > } > > /** Craft a LambdaForm customized for this particular MethodHandle */ > /*non-public*/ > void customize() { > + final LambdaForm form = this.form; > if (form.customized == null) { > LambdaForm newForm = form.customize(this); > updateForm(newForm); > diff --git a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > --- a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > @@ -909,7 +909,7 @@ > int c = count; > maybeCustomizeTarget(); > if (c <= 1) { > - // Try to limit number of updates. MethodHandle.updateForm() doesn't guarantee LF update visibility. > + // Try to limit number of updates. > if (isCounting) { > isCounting = false; > return true; > > > From huizhe.wang at oracle.com Wed Jan 3 23:32:55 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 3 Jan 2018 15:32:55 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8189704: broken links in the javax/xml/namespace package Message-ID: <556cae77-3efb-ad7b-3310-b1f7c8389387@oracle.com> Hi, Please review a doc-only fix for two namespace classes. The broken links to the Errata are removed since the reference(s) to the Errata is part of the Namespaces in XML specification itself. There is no need for the extra link(s).? Also, for NamespaceContext, the link to "Namespaces in XML" shall be to the specification, not the QName section, the fragment portion of the link, e.g. "#ns-qualnames" therefore is also removed. JBS: https://bugs.openjdk.java.net/browse/JDK-8189704 webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8189704/webrev/ Thanks, Joe From huizhe.wang at oracle.com Wed Jan 3 23:53:26 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 3 Jan 2018 15:53:26 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8189704: broken links in the javax/xml/namespace package In-Reply-To: <556cae77-3efb-ad7b-3310-b1f7c8389387@oracle.com> References: <556cae77-3efb-ad7b-3310-b1f7c8389387@oracle.com> Message-ID: This change also fix: JBS: https://bugs.openjdk.java.net/browse/JDK-8184046 that contains the same broken link to "Namespaces in XML Errata". As Daniel indicated, the correct link was with a ".html" suffix. Since the content in this Errata has already been incorporated in the current specification that the link to "Namespaces in XML" points to, there's no more reason to link to the Errata. Thanks, Joe On 1/3/2018 3:32 PM, huizhe wang wrote: > Hi, > > Please review a doc-only fix for two namespace classes. The broken > links to the Errata are removed since the reference(s) to the Errata > is part of the Namespaces in XML specification itself. There is no > need for the extra link(s).? Also, for NamespaceContext, the link to > "Namespaces in XML" shall be to the specification, not the QName > section, the fragment portion of the link, e.g. "#ns-qualnames" > therefore is also removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8189704 > webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8189704/webrev/ > > Thanks, > Joe From joe.darcy at oracle.com Thu Jan 4 00:47:41 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Wed, 03 Jan 2018 16:47:41 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8189704: broken links in the javax/xml/namespace package In-Reply-To: <556cae77-3efb-ad7b-3310-b1f7c8389387@oracle.com> References: <556cae77-3efb-ad7b-3310-b1f7c8389387@oracle.com> Message-ID: <5A4D79AD.4040000@oracle.com> Looks fine Joe; cheers, -Joe On 1/3/2018 3:32 PM, huizhe wang wrote: > Hi, > > Please review a doc-only fix for two namespace classes. The broken > links to the Errata are removed since the reference(s) to the Errata > is part of the Namespaces in XML specification itself. There is no > need for the extra link(s). Also, for NamespaceContext, the link to > "Namespaces in XML" shall be to the specification, not the QName > section, the fragment portion of the link, e.g. "#ns-qualnames" > therefore is also removed. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8189704 > webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8189704/webrev/ > > Thanks, > Joe From huizhe.wang at oracle.com Thu Jan 4 01:05:54 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Wed, 3 Jan 2018 17:05:54 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8189704: broken links in the javax/xml/namespace package In-Reply-To: <5A4D79AD.4040000@oracle.com> References: <556cae77-3efb-ad7b-3310-b1f7c8389387@oracle.com> <5A4D79AD.4040000@oracle.com> Message-ID: <0cf557b4-14df-5722-e2a9-02cee6fb2219@oracle.com> Thanks Joe! Cheers, Joe On 1/3/2018 4:47 PM, Joseph D. Darcy wrote: > Looks fine Joe; cheers, > > -Joe > > On 1/3/2018 3:32 PM, huizhe wang wrote: >> Hi, >> >> Please review a doc-only fix for two namespace classes. The broken >> links to the Errata are removed since the reference(s) to the Errata >> is part of the Namespaces in XML specification itself. There is no >> need for the extra link(s).? Also, for NamespaceContext, the link to >> "Namespaces in XML" shall be to the specification, not the QName >> section, the fragment portion of the link, e.g. "#ns-qualnames" >> therefore is also removed. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8189704 >> webrevs: http://cr.openjdk.java.net/~joehw/jdk10/8189704/webrev/ >> >> Thanks, >> Joe > From david.holmes at oracle.com Thu Jan 4 01:35:53 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Jan 2018 11:35:53 +1000 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: References: Message-ID: On 4/01/2018 3:13 AM, Vitaly Davidovich wrote: > On Wed, Jan 3, 2018 at 12:00 PM, Alan Bateman > wrote: > >> The stack trace looks like JDK 8 or older. The initialization has changed >> quite a bit in JDK 9+ and would be interesting to see if you can duplicate >> it there. >> > Yes - it's 8u152 as I mentioned in the follow-up email (sorry, should've > mentioned that initially). > > Trying this on JDK9 is a bit tricky as the code (and its dependencies) in > question isn't easily runnable on 9 without quite a bit of work. > > Do you agree with the deadlock assessment though? That's what it looks like > to me but wanted to make sure I'm not missing anything. You're not missing anything. It's a class initialization related "deadlock". Thread A has called FileSystems.getDefault() which is doing of DefaultFileSystemHolder, which in turn leads to Runtime.loadLibrary0 which needs to lock the Runtime instance. Thread B is already doing a loadLibrary and holds the Runtime lock, the loadLibrary code then needs to do FileSystems.getDefault() which has to load and initialize DefaultFileSystemHolder, but that initialization is already in progress so internally the thread does a wait(). So Thread B is waiting for Thread A to finish initialization, but holds the monitor lock that Thread A needs to finish the initialization. Deadlock. AFAICS this will still be possible in 9/10/11 Cheers, David > Thanks Alan > >> >> -Alan >> >> >> On 03/01/2018 16:56, Vitaly Davidovich wrote: >> >>> Hi all, >>> >>> Consider the following stacktraces of the main app thread and a background >>> thread: >>> >>> "main" #1 prio=5 os_prio=0 tid=0x0000000001bfd000 nid=0x1958 waiting for >>> monitor entry [0x00002b98ceba3000] >>> >>> java.lang.Thread.State: BLOCKED (on object monitor) >>> >>> at java.lang.Runtime.loadLibrary0(Runtime.java:862) >>> >>> - waiting to lock <0x00000007bf834b20> (a >>> java.lang.Runtime) >>> >>> at java.lang.System.loadLibrary(System.java:1122) >>> >>> at sun.nio.fs.UnixNativeDispatcher$1.run( >>> UnixNativeDispatcher.java:573) >>> >>> at sun.nio.fs.UnixNativeDispatcher$1.run( >>> UnixNativeDispatcher.java:571) >>> >>> at java.security.AccessController.doPrivileged(Native >>> Method) >>> >>> at sun.nio.fs.UnixNativeDispatcher.( >>> UnixNativeDispatcher.java:571) >>> >>> at sun.nio.fs.UnixFileSystem.>> t>(UnixFileSystem.java:67) >>> >>> at sun.nio.fs.LinuxFileSystem.< >>> init>(LinuxFileSystem.java:39) >>> >>> at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( >>> LinuxFileSystemProvider.java:46) >>> >>> at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( >>> LinuxFileSystemProvider.java:39) >>> >>> at sun.nio.fs.UnixFileSystemProvider.( >>> UnixFileSystemProvider.java:56) >>> >>> at sun.nio.fs.LinuxFileSystemProvider. >>> (LinuxFileSystemProvider.java:41) >>> >>> at >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>> Method) >>> >>> at sun.reflect.NativeConstructorA >>> ccessorImpl.newInstance( >>> NativeConstructorAccessorImpl.java:62) >>> >>> at sun.reflect.DelegatingConstructorAccessorI >>> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>> >>> at java.lang.reflect.Constructor. >>> newInstance(Constructor.java:423) >>> >>> at java.lang.Class.newInstance(Class.java:442) >>> >>> at sun.nio.fs.DefaultFileSystemProvider.createProvider( >>> DefaultFileSystemProvider.java:48) >>> >>> at sun.nio.fs.DefaultFileSystemProvider.create( >>> DefaultFileSystemProvider.java:63) >>> >>> at java.nio.file.FileSystems$DefaultFileSystemHolder. >>> getDefaultProvider(FileSystems.java:108) >>> >>> at java.nio.file.FileSystems$DefaultFileSystemHolder. >>> access$000(FileSystems.java:89) >>> >>> at java.nio.file.FileSystems$Defa >>> ultFileSystemHolder$1.run( >>> FileSystems.java:98) >>> >>> at java.nio.file.FileSystems$Defa >>> ultFileSystemHolder$1.run( >>> FileSystems.java:96) >>> >>> at java.security.AccessController.doPrivileged(Native >>> Method) >>> >>> at java.nio.file.FileSystems$DefaultFileSystemHolder. >>> defaultFileSystem(FileSystems.java:96) >>> >>> at java.nio.file.FileSystems$DefaultFileSystemHolder.< >>> clinit>(FileSystems.java:90) >>> >>> at java.nio.file.FileSystems.getD >>> efault(FileSystems.java: >>> 176) >>> >>> at java.nio.file.Paths.get(Paths.java:84) >>> >>> at .ServiceLoader. >>> setup(ServiceLoader.java:446) >>> >>> at .MetadataAdminLoader.main( >>> MetadataAdminLoader.java:52) >>> >>> >>> >>> >>> >>> "shardDbExecutor-0-pool-0" #33 prio=5 os_prio=0 tid=0x0000000001ebf000 >>> nid=0x1d01 in Object.wait() [0x00002b991f941000] >>> >>> java.lang.Thread.State: RUNNABLE >>> >>> at java.nio.file.FileSystems.getD >>> efault(FileSystems.java: >>> 176) >>> >>> at java.nio.file.Paths.get(Paths.java:138) >>> >>> at sun.misc.Launcher$ExtClassLoader.findLibrary( >>> Launcher.java:224) >>> >>> at java.lang.ClassLoader.loadLibr >>> ary(ClassLoader.java:1830) >>> >>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) >>> >>> - locked <0x00000007bf834b20> (a java.lang.Runtime) >>> >>> at java.lang.System.loadLibrary(System.java:1122) >>> >>> at sun.security.ec.SunEC$1.run(SunEC.java:60) >>> >>> at sun.security.ec.SunEC$1.run(SunEC.java:58) >>> >>> at java.security.AccessController.doPrivileged(Native >>> Method) >>> >>> at sun.security.ec.SunEC.(SunEC.java:58) >>> >>> at >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>> Method) >>> >>> at sun.reflect.NativeConstructorA >>> ccessorImpl.newInstance( >>> NativeConstructorAccessorImpl.java:62) >>> >>> at sun.reflect.DelegatingConstructorAccessorI >>> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) >>> >>> at java.lang.reflect.Constructor. >>> newInstance(Constructor.java:423) >>> >>> at java.lang.Class.newInstance(Class.java:442) >>> >>> at sun.security.jca.ProviderConfig$2.run( >>> ProviderConfig.java:221) >>> >>> at sun.security.jca.ProviderConfig$2.run( >>> >>> ProviderConfig.java:206) >>> >>> at java.security.AccessController.doPrivileged(Native >>> Method) >>> >>> at sun.security.jca.ProviderConfig.doLoadProvider( >>> ProviderConfig.java:206) >>> >>> at sun.security.jca.ProviderConfig.getProvider( >>> ProviderConfig.java:187) >>> >>> - locked <0x00000007bea03f48> (a sun.security.jca. >>> ProviderConfig) >>> >>> at sun.security.jca.ProviderList. >>> getProvider(ProviderList.java:233) >>> >>> at sun.security.jca.ProviderList. >>> getService(ProviderList.java:331) >>> >>> at sun.security.jca.GetInstance. >>> getInstance(GetInstance.java:157) >>> >>> at javax.net.ssl.SSLContext.getIn >>> stance(SSLContext.java:156) >>> >>> at com.microsoft.sqlserver.jdbc. >>> TDSChannel.enableSSL(IOBuffer.java:1658) >>> >>> at com.microsoft.sqlserver.jdbc.SQLServerConnection. >>> connectHelper(SQLServerConnection.java:1976) >>> >>> at com.microsoft.sqlserver.jdbc.S >>> QLServerConnection.login( >>> SQLServerConnection.java:1627) >>> >>> at com.microsoft.sqlserver.jdbc.SQLServerConnection. >>> connectInternal(SQLServerConnection.java:1458) >>> >>> at com.microsoft.sqlserver.jdbc.S >>> QLServerConnection.connect( >>> SQLServerConnection.java:772) >>> >>> at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect( >>> SQLServerDriver.java:1168) >>> >>> at java.sql.DriverManager.getConnection(DriverManager. >>> java:664) >>> >>> at java.sql.DriverManager.getConnection(DriverManager. >>> java:208) >>> >>> at .ConnectionPool.createGoodConnection( >>> ConnectionPool.java:565) >>> >>> at .ConnectionPool.createNewConnection( >>> ConnectionPool.java:519) >>> >>> at .ConnectionPool. >>> getConnection(ConnectionPool.java:407) >>> >>> at .StatementCachingConnectionImpl >>> .setupConnection(StatementCachingConnectionImpl.java:94) >>> >>> at .StatementCachingConnectionImpl.( >>> StatementCachingConnectionImpl.java:54) >>> >>> at .TestConnectionFactories$ >>> FaultInjectingConnection.(TestConnectionFactories.java:90) >>> >>> at .TestConnectionF >>> actories.lambda$static$3( >>> TestConnectionFactories.java:33) >>> >>> at .TestConnectionFactories$$ >>> Lambda$12/1738236591.newConnection(Unknown Source) >>> >>> at .SedaExecutor. >>> setupConnections(SedaExecutor.java:130) >>> >>> at .SedaExecutor$SedaThreadFactory.lambda$ >>> newThread$0(SedaExecutor.java:165) >>> >>> at >>> .SedaExecutor$SedaThreadFactory$$Lambda$22/139 >>> 2419022.run(Unknown >>> Source) >>> >>> at java.lang.Thread.run(Thread.java:748) >>> >>> >>> The application intermittently hangs on startup, and the above callstacks >>> are present. It seems like there's a deadlock due to concurrent class >>> loading and default FileSystem initialization (which itself triggers class >>> loading here). >>> >>> I wasn't able to find any existing JBS entries reporting anything similar. >>> Am I seeing this right? Is this a known issue? >>> >>> Thanks >>> >> >> From brian.burkhalter at oracle.com Thu Jan 4 01:48:43 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 3 Jan 2018 17:48:43 -0800 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: <9f81b074-5a62-2869-ad24-256eff0c0837@oracle.com> References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> <9f81b074-5a62-2869-ad24-256eff0c0837@oracle.com> Message-ID: <8A10FB3C-A0FD-427E-BFDD-0315EE4D8A8A@oracle.com> On Jan 3, 2018, at 3:23 AM, Alan Bateman wrote: > On 02/01/2018 23:43, Brian Burkhalter wrote: >> : >> >>> So not clear to me that Files.copy methods needs to use this. >> >> So you are suggesting leaving the call to InputStream.transferTo() in preference to IOSupport.copy()? > Yes, I think these two should use transferTo. I don?t think that transferTo() can be used in JrtPath right now as BUILD_JRTFS sets the compiler option ??release 8? and transferTo() was added in 9. So unless this can be changed to 9, then JrtPath would need either to remain unchanged or use IOSupport.copy(). Brian From vitalyd at gmail.com Thu Jan 4 02:26:18 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 04 Jan 2018 02:26:18 +0000 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: References: Message-ID: On Wed, Jan 3, 2018 at 8:36 PM David Holmes wrote: > On 4/01/2018 3:13 AM, Vitaly Davidovich wrote: > > On Wed, Jan 3, 2018 at 12:00 PM, Alan Bateman > > wrote: > > > >> The stack trace looks like JDK 8 or older. The initialization has > changed > >> quite a bit in JDK 9+ and would be interesting to see if you can > duplicate > >> it there. > >> > > Yes - it's 8u152 as I mentioned in the follow-up email (sorry, should've > > mentioned that initially). > > > > Trying this on JDK9 is a bit tricky as the code (and its dependencies) in > > question isn't easily runnable on 9 without quite a bit of work. > > > > Do you agree with the deadlock assessment though? That's what it looks > like > > to me but wanted to make sure I'm not missing anything. > > You're not missing anything. It's a class initialization related > "deadlock". > > Thread A has called FileSystems.getDefault() which is doing of > DefaultFileSystemHolder, which in turn leads to Runtime.loadLibrary0 > which needs to lock the Runtime instance. > > Thread B is already doing a loadLibrary and holds the Runtime lock, the > loadLibrary code then needs to do FileSystems.getDefault() which has to > load and initialize DefaultFileSystemHolder, but that initialization is > already in progress so internally the thread does a wait(). > > So Thread B is waiting for Thread A to finish initialization, but holds > the monitor lock that Thread A needs to finish the initialization. > Deadlock. > > AFAICS this will still be possible in 9/10/11 That?s what I thought - thanks for confirming David. I?m slightly surprised this hasn?t been reported before, although perhaps I?ve failed in finding the report. Does this deserve a bug report? I know Hotspot and some core classes in the JDK do an elaborate dance to avoid classloading issues, but this is ?user? code. This has been worked around on our end by forcing the default FS to load before this code runs. Perhaps something similar can be done as the JDK bootstraps. > > > Cheers, > David > > > Thanks Alan > > > >> > >> -Alan > >> > >> > >> On 03/01/2018 16:56, Vitaly Davidovich wrote: > >> > >>> Hi all, > >>> > >>> Consider the following stacktraces of the main app thread and a > background > >>> thread: > >>> > >>> "main" #1 prio=5 os_prio=0 tid=0x0000000001bfd000 nid=0x1958 waiting > for > >>> monitor entry [0x00002b98ceba3000] > >>> > >>> java.lang.Thread.State: BLOCKED (on object monitor) > >>> > >>> at java.lang.Runtime.loadLibrary0(Runtime.java:862) > >>> > >>> - waiting to lock <0x00000007bf834b20> (a > >>> java.lang.Runtime) > >>> > >>> at java.lang.System.loadLibrary(System.java:1122) > >>> > >>> at sun.nio.fs.UnixNativeDispatcher$1.run( > >>> UnixNativeDispatcher.java:573) > >>> > >>> at sun.nio.fs.UnixNativeDispatcher$1.run( > >>> UnixNativeDispatcher.java:571) > >>> > >>> at java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>> at sun.nio.fs.UnixNativeDispatcher.( > >>> UnixNativeDispatcher.java:571) > >>> > >>> at sun.nio.fs.UnixFileSystem. >>> t>(UnixFileSystem.java:67) > >>> > >>> at sun.nio.fs.LinuxFileSystem.< > >>> init>(LinuxFileSystem.java:39) > >>> > >>> at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( > >>> LinuxFileSystemProvider.java:46) > >>> > >>> at sun.nio.fs.LinuxFileSystemProvider.newFileSystem( > >>> LinuxFileSystemProvider.java:39) > >>> > >>> at sun.nio.fs.UnixFileSystemProvider.( > >>> UnixFileSystemProvider.java:56) > >>> > >>> at sun.nio.fs.LinuxFileSystemProvider. > >>> (LinuxFileSystemProvider.java:41) > >>> > >>> at > >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>> Method) > >>> > >>> at sun.reflect.NativeConstructorA > >>> ccessorImpl.newInstance( > >>> NativeConstructorAccessorImpl.java:62) > >>> > >>> at sun.reflect.DelegatingConstructorAccessorI > >>> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > >>> > >>> at java.lang.reflect.Constructor. > >>> newInstance(Constructor.java:423) > >>> > >>> at java.lang.Class.newInstance(Class.java:442) > >>> > >>> at > sun.nio.fs.DefaultFileSystemProvider.createProvider( > >>> DefaultFileSystemProvider.java:48) > >>> > >>> at sun.nio.fs.DefaultFileSystemProvider.create( > >>> DefaultFileSystemProvider.java:63) > >>> > >>> at java.nio.file.FileSystems$DefaultFileSystemHolder. > >>> getDefaultProvider(FileSystems.java:108) > >>> > >>> at java.nio.file.FileSystems$DefaultFileSystemHolder. > >>> access$000(FileSystems.java:89) > >>> > >>> at java.nio.file.FileSystems$Defa > >>> ultFileSystemHolder$1.run( > >>> FileSystems.java:98) > >>> > >>> at java.nio.file.FileSystems$Defa > >>> ultFileSystemHolder$1.run( > >>> FileSystems.java:96) > >>> > >>> at java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>> at java.nio.file.FileSystems$DefaultFileSystemHolder. > >>> defaultFileSystem(FileSystems.java:96) > >>> > >>> at > java.nio.file.FileSystems$DefaultFileSystemHolder.< > >>> clinit>(FileSystems.java:90) > >>> > >>> at java.nio.file.FileSystems.getD > >>> efault(FileSystems.java: > >>> 176) > >>> > >>> at java.nio.file.Paths.get(Paths.java:84) > >>> > >>> at .ServiceLoader. > >>> setup(ServiceLoader.java:446) > >>> > >>> at .MetadataAdminLoader.main( > >>> MetadataAdminLoader.java:52) > >>> > >>> > >>> > >>> > >>> > >>> "shardDbExecutor-0-pool-0" #33 prio=5 os_prio=0 tid=0x0000000001ebf000 > >>> nid=0x1d01 in Object.wait() [0x00002b991f941000] > >>> > >>> java.lang.Thread.State: RUNNABLE > >>> > >>> at java.nio.file.FileSystems.getD > >>> efault(FileSystems.java: > >>> 176) > >>> > >>> at java.nio.file.Paths.get(Paths.java:138) > >>> > >>> at sun.misc.Launcher$ExtClassLoader.findLibrary( > >>> Launcher.java:224) > >>> > >>> at java.lang.ClassLoader.loadLibr > >>> ary(ClassLoader.java:1830) > >>> > >>> at java.lang.Runtime.loadLibrary0(Runtime.java:870) > >>> > >>> - locked <0x00000007bf834b20> (a java.lang.Runtime) > >>> > >>> at java.lang.System.loadLibrary(System.java:1122) > >>> > >>> at sun.security.ec.SunEC$1.run(SunEC.java:60) > >>> > >>> at sun.security.ec.SunEC$1.run(SunEC.java:58) > >>> > >>> at java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>> at sun.security.ec.SunEC.(SunEC.java:58) > >>> > >>> at > >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>> Method) > >>> > >>> at sun.reflect.NativeConstructorA > >>> ccessorImpl.newInstance( > >>> NativeConstructorAccessorImpl.java:62) > >>> > >>> at sun.reflect.DelegatingConstructorAccessorI > >>> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > >>> > >>> at java.lang.reflect.Constructor. > >>> newInstance(Constructor.java:423) > >>> > >>> at java.lang.Class.newInstance(Class.java:442) > >>> > >>> at sun.security.jca.ProviderConfig$2.run( > >>> ProviderConfig.java:221) > >>> > >>> at sun.security.jca.ProviderConfig$2.run( > >>> > >>> ProviderConfig.java:206) > >>> > >>> at java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>> at sun.security.jca.ProviderConfig.doLoadProvider( > >>> ProviderConfig.java:206) > >>> > >>> at sun.security.jca.ProviderConfig.getProvider( > >>> ProviderConfig.java:187) > >>> > >>> - locked <0x00000007bea03f48> (a sun.security.jca. > >>> ProviderConfig) > >>> > >>> at sun.security.jca.ProviderList. > >>> getProvider(ProviderList.java:233) > >>> > >>> at sun.security.jca.ProviderList. > >>> getService(ProviderList.java:331) > >>> > >>> at sun.security.jca.GetInstance. > >>> getInstance(GetInstance.java:157) > >>> > >>> at javax.net.ssl.SSLContext.getIn > >>> stance(SSLContext.java:156) > >>> > >>> at com.microsoft.sqlserver.jdbc. > >>> TDSChannel.enableSSL(IOBuffer.java:1658) > >>> > >>> at com.microsoft.sqlserver.jdbc.SQLServerConnection. > >>> connectHelper(SQLServerConnection.java:1976) > >>> > >>> at com.microsoft.sqlserver.jdbc.S > >>> QLServerConnection.login( > >>> SQLServerConnection.java:1627) > >>> > >>> at com.microsoft.sqlserver.jdbc.SQLServerConnection. > >>> connectInternal(SQLServerConnection.java:1458) > >>> > >>> at com.microsoft.sqlserver.jdbc.S > >>> QLServerConnection.connect( > >>> SQLServerConnection.java:772) > >>> > >>> at > com.microsoft.sqlserver.jdbc.SQLServerDriver.connect( > >>> SQLServerDriver.java:1168) > >>> > >>> at > java.sql.DriverManager.getConnection(DriverManager. > >>> java:664) > >>> > >>> at > java.sql.DriverManager.getConnection(DriverManager. > >>> java:208) > >>> > >>> at > .ConnectionPool.createGoodConnection( > >>> ConnectionPool.java:565) > >>> > >>> at .ConnectionPool.createNewConnection( > >>> ConnectionPool.java:519) > >>> > >>> at .ConnectionPool. > >>> getConnection(ConnectionPool.java:407) > >>> > >>> at .StatementCachingConnectionImpl > >>> .setupConnection(StatementCachingConnectionImpl.java:94) > >>> > >>> at > .StatementCachingConnectionImpl.( > >>> StatementCachingConnectionImpl.java:54) > >>> > >>> at .TestConnectionFactories$ > >>> FaultInjectingConnection.(TestConnectionFactories.java:90) > >>> > >>> at .TestConnectionF > >>> actories.lambda$static$3( > >>> TestConnectionFactories.java:33) > >>> > >>> at .TestConnectionFactories$$ > >>> Lambda$12/1738236591.newConnection(Unknown Source) > >>> > >>> at .SedaExecutor. > >>> setupConnections(SedaExecutor.java:130) > >>> > >>> at > .SedaExecutor$SedaThreadFactory.lambda$ > >>> newThread$0(SedaExecutor.java:165) > >>> > >>> at > >>> .SedaExecutor$SedaThreadFactory$$Lambda$22/139 > >>> 2419022.run(Unknown > >>> Source) > >>> > >>> at java.lang.Thread.run(Thread.java:748) > >>> > >>> > >>> The application intermittently hangs on startup, and the above > callstacks > >>> are present. It seems like there's a deadlock due to concurrent class > >>> loading and default FileSystem initialization (which itself triggers > class > >>> loading here). > >>> > >>> I wasn't able to find any existing JBS entries reporting anything > similar. > >>> Am I seeing this right? Is this a known issue? > >>> > >>> Thanks > >>> > >> > >> > -- Sent from my phone From david.holmes at oracle.com Thu Jan 4 03:30:15 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 4 Jan 2018 13:30:15 +1000 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: References: Message-ID: <8b8aff4d-bb8a-f30b-7ff3-d0b8b83da59e@oracle.com> On 4/01/2018 12:26 PM, Vitaly Davidovich wrote: > > On Wed, Jan 3, 2018 at 8:36 PM David Holmes > wrote: > > On 4/01/2018 3:13 AM, Vitaly Davidovich wrote: > > On Wed, Jan 3, 2018 at 12:00 PM, Alan Bateman > > > > wrote: > > > >> The stack trace looks like JDK 8 or older. The initialization > has changed > >> quite a bit in JDK 9+ and would be interesting to see if you can > duplicate > >> it there. > >> > > Yes - it's 8u152 as I mentioned in the follow-up email (sorry, > should've > > mentioned that initially). > > > > Trying this on JDK9 is a bit tricky as the code (and its > dependencies) in > > question isn't easily runnable on 9 without quite a bit of work. > > > > Do you agree with the deadlock assessment though? That's what it > looks like > > to me but wanted to make sure I'm not missing anything. > > You're not missing anything. It's a class initialization related > "deadlock". > > Thread A has called FileSystems.getDefault() which is doing of > DefaultFileSystemHolder, which in turn leads to Runtime.loadLibrary0 > which needs to lock the Runtime instance. > > Thread B is already doing a loadLibrary and holds the Runtime lock, the > loadLibrary code then needs to do FileSystems.getDefault() which has to > load and initialize DefaultFileSystemHolder, but that initialization is > already in progress so internally the thread does a wait(). > > So Thread B is waiting for Thread A to finish initialization, but holds > the monitor lock that Thread A needs to finish the initialization. > Deadlock. > > AFAICS this will still be possible in 9/10/11 > > > That?s what I thought - thanks for confirming David.? I?m slightly > surprised this hasn?t been reported before, although perhaps I?ve failed > in finding the report. I couldn't find anything either. Although it seems "obvious" upon analysis it may be very rare to have such an initialization race. > > Does this deserve a bug report? I know Hotspot and some core classes in > the JDK do an elaborate dance to avoid classloading issues, but this is > ?user? code.? This has been worked around on our end by forcing the > default FS to load before this code runs.? Perhaps something similar can > be done as the JDK bootstraps. Definitely worth a bug report. Not sure what we can do on the JDK side as I'm unclear whether the default filesystem necessarily gets initialized after the module system is up. Cheers, David > > > Cheers, > David > > > Thanks Alan > > > >> > >> -Alan > >> > >> > >> On 03/01/2018 16:56, Vitaly Davidovich wrote: > >> > >>> Hi all, > >>> > >>> Consider the following stacktraces of the main app thread and a > background > >>> thread: > >>> > >>> "main" #1 prio=5 os_prio=0 tid=0x0000000001bfd000 nid=0x1958 > waiting for > >>> monitor entry [0x00002b98ceba3000] > >>> > >>>? ? ? java.lang.Thread.State: BLOCKED (on object monitor) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.lang.Runtime.loadLibrary0(Runtime.java:862) > >>> > >>>? ? ? ? ? ? ? ? ? ?- waiting to lock <0x00000007bf834b20> (a > >>> java.lang.Runtime) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.System.loadLibrary(System.java:1122) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.UnixNativeDispatcher$1.run( > >>> UnixNativeDispatcher.java:573) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.UnixNativeDispatcher$1.run( > >>> UnixNativeDispatcher.java:571) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.UnixNativeDispatcher.( > >>> UnixNativeDispatcher.java:571) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.UnixFileSystem. >>> t>(UnixFileSystem.java:67) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.LinuxFileSystem.< > >>> init>(LinuxFileSystem.java:39) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > sun.nio.fs.LinuxFileSystemProvider.newFileSystem( > >>> LinuxFileSystemProvider.java:46) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > sun.nio.fs.LinuxFileSystemProvider.newFileSystem( > >>> LinuxFileSystemProvider.java:39) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.UnixFileSystemProvider.( > >>> UnixFileSystemProvider.java:56) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.LinuxFileSystemProvider. > >>> (LinuxFileSystemProvider.java:41) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>> Method) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.reflect.NativeConstructorA > >>> ccessorImpl.newInstance( > >>> NativeConstructorAccessorImpl.java:62) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.reflect.DelegatingConstructorAccessorI > >>> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.reflect.Constructor. > >>> newInstance(Constructor.java:423) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.Class.newInstance(Class.java:442) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > sun.nio.fs.DefaultFileSystemProvider.createProvider( > >>> DefaultFileSystemProvider.java:48) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.nio.fs.DefaultFileSystemProvider.create( > >>> DefaultFileSystemProvider.java:63) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.nio.file.FileSystems$DefaultFileSystemHolder. > >>> getDefaultProvider(FileSystems.java:108) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.nio.file.FileSystems$DefaultFileSystemHolder. > >>> access$000(FileSystems.java:89) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.nio.file.FileSystems$Defa > >>> ultFileSystemHolder$1.run( > >>> FileSystems.java:98) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.nio.file.FileSystems$Defa > >>> ultFileSystemHolder$1.run( > >>> FileSystems.java:96) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.nio.file.FileSystems$DefaultFileSystemHolder. > >>> defaultFileSystem(FileSystems.java:96) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.nio.file.FileSystems$DefaultFileSystemHolder.< > >>> clinit>(FileSystems.java:90) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.nio.file.FileSystems.getD > >>> efault(FileSystems.java: > >>> 176) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.nio.file.Paths.get(Paths.java:84) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .ServiceLoader. > >>> setup(ServiceLoader.java:446) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .MetadataAdminLoader.main( > >>> MetadataAdminLoader.java:52) > >>> > >>> > >>> > >>> > >>> > >>> "shardDbExecutor-0-pool-0" #33 prio=5 os_prio=0 > tid=0x0000000001ebf000 > >>> nid=0x1d01 in Object.wait() [0x00002b991f941000] > >>> > >>>? ? ? java.lang.Thread.State: RUNNABLE > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.nio.file.FileSystems.getD > >>> efault(FileSystems.java: > >>> 176) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.nio.file.Paths.get(Paths.java:138) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.misc.Launcher$ExtClassLoader.findLibrary( > >>> Launcher.java:224) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.ClassLoader.loadLibr > >>> ary(ClassLoader.java:1830) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.lang.Runtime.loadLibrary0(Runtime.java:870) > >>> > >>>? ? ? ? ? ? ? ? ? ?- locked <0x00000007bf834b20> (a > java.lang.Runtime) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.System.loadLibrary(System.java:1122) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.ec.SunEC$1.run(SunEC.java:60) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.ec.SunEC$1.run(SunEC.java:58) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.ec.SunEC.(SunEC.java:58) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > >>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>> Method) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.reflect.NativeConstructorA > >>> ccessorImpl.newInstance( > >>> NativeConstructorAccessorImpl.java:62) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.reflect.DelegatingConstructorAccessorI > >>> mpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.reflect.Constructor. > >>> newInstance(Constructor.java:423) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.Class.newInstance(Class.java:442) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.jca.ProviderConfig$2.run( > >>> ProviderConfig.java:221) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.jca.ProviderConfig$2.run( > >>> > >>> ProviderConfig.java:206) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.security.AccessController.doPrivileged(Native > >>> Method) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > sun.security.jca.ProviderConfig.doLoadProvider( > >>> ProviderConfig.java:206) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.jca.ProviderConfig.getProvider( > >>> ProviderConfig.java:187) > >>> > >>>? ? ? ? ? ? ? ? ? ?- locked <0x00000007bea03f48> (a > sun.security.jca. > >>> ProviderConfig) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.jca.ProviderList. > >>> getProvider(ProviderList.java:233) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.jca.ProviderList. > >>> getService(ProviderList.java:331) > >>> > >>>? ? ? ? ? ? ? ? ? ?at sun.security.jca.GetInstance. > >>> getInstance(GetInstance.java:157) > >>> > >>>? ? ? ? ? ? ? ? ? ?at javax.net.ssl.SSLContext.getIn > >>> stance(SSLContext.java:156) > >>> > >>>? ? ? ? ? ? ? ? ? ?at com.microsoft.sqlserver.jdbc. > >>> TDSChannel.enableSSL(IOBuffer.java:1658) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > com.microsoft.sqlserver.jdbc.SQLServerConnection. > >>> connectHelper(SQLServerConnection.java:1976) > >>> > >>>? ? ? ? ? ? ? ? ? ?at com.microsoft.sqlserver.jdbc.S > >>> QLServerConnection.login( > >>> SQLServerConnection.java:1627) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > com.microsoft.sqlserver.jdbc.SQLServerConnection. > >>> connectInternal(SQLServerConnection.java:1458) > >>> > >>>? ? ? ? ? ? ? ? ? ?at com.microsoft.sqlserver.jdbc.S > >>> QLServerConnection.connect( > >>> SQLServerConnection.java:772) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > com.microsoft.sqlserver.jdbc.SQLServerDriver.connect( > >>> SQLServerDriver.java:1168) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.sql.DriverManager.getConnection(DriverManager. > >>> java:664) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > java.sql.DriverManager.getConnection(DriverManager. > >>> java:208) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > .ConnectionPool.createGoodConnection( > >>> ConnectionPool.java:565) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > .ConnectionPool.createNewConnection( > >>> ConnectionPool.java:519) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .ConnectionPool. > >>> getConnection(ConnectionPool.java:407) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .StatementCachingConnectionImpl > >>> .setupConnection(StatementCachingConnectionImpl.java:94) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > .StatementCachingConnectionImpl.( > >>> StatementCachingConnectionImpl.java:54) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .TestConnectionFactories$ > >>> FaultInjectingConnection.(TestConnectionFactories.java:90) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .TestConnectionF > >>> actories.lambda$static$3( > >>> TestConnectionFactories.java:33) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .TestConnectionFactories$$ > >>> Lambda$12/1738236591.newConnection(Unknown Source) > >>> > >>>? ? ? ? ? ? ? ? ? ?at .SedaExecutor. > >>> setupConnections(SedaExecutor.java:130) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > .SedaExecutor$SedaThreadFactory.lambda$ > >>> newThread$0(SedaExecutor.java:165) > >>> > >>>? ? ? ? ? ? ? ? ? ?at > >>> .SedaExecutor$SedaThreadFactory$$Lambda$22/139 > >>> 2419022.run(Unknown > >>> Source) > >>> > >>>? ? ? ? ? ? ? ? ? ?at java.lang.Thread.run(Thread.java:748) > >>> > >>> > >>> The application intermittently hangs on startup, and the above > callstacks > >>> are present.? It seems like there's a deadlock due to > concurrent class > >>> loading and default FileSystem initialization (which itself > triggers class > >>> loading here). > >>> > >>> I wasn't able to find any existing JBS entries reporting > anything similar. > >>> Am I seeing this right? Is this a known issue? > >>> > >>> Thanks > >>> > >> > >> > > -- > Sent from my phone From Alan.Bateman at oracle.com Thu Jan 4 08:03:08 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 Jan 2018 08:03:08 +0000 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: References: Message-ID: <356f9d2a-2682-ae17-6425-6b55d8cf5a36@oracle.com> On 04/01/2018 01:35, David Holmes wrote: > > You're not missing anything. It's a class initialization related > "deadlock". > > Thread A has called FileSystems.getDefault() which is doing > of DefaultFileSystemHolder, which in turn leads to > Runtime.loadLibrary0 which needs to lock the Runtime instance. > > Thread B is already doing a loadLibrary and holds the Runtime lock, > the loadLibrary code then needs to do FileSystems.getDefault() which > has to load and initialize DefaultFileSystemHolder, but that > initialization is already in progress so internally the thread does a > wait(). > > So Thread B is waiting for Thread A to finish initialization, but > holds the monitor lock that Thread A needs to finish the > initialization. Deadlock. > > AFAICS this will still be possible in 9/10/11 Startup and initialization has changed quite a bit since JDK 8. In JDK 9 the file system is initialized early (long before main). However, it changes again in JDK 10 due to ongoing work to makes things lazy and improve startup time. Separately, FilePermission and the JarFile support have been re-written so it should be loaded before any user code executes. So I think needs to be studied again, I agree with your other mail to create a bug to track that. -Alan From Alan.Bateman at oracle.com Thu Jan 4 08:06:38 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 4 Jan 2018 08:06:38 +0000 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: <8A10FB3C-A0FD-427E-BFDD-0315EE4D8A8A@oracle.com> References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> <9f81b074-5a62-2869-ad24-256eff0c0837@oracle.com> <8A10FB3C-A0FD-427E-BFDD-0315EE4D8A8A@oracle.com> Message-ID: On 04/01/2018 01:48, Brian Burkhalter wrote: > On Jan 3, 2018, at 3:23 AM, Alan Bateman > wrote: > >> On 02/01/2018 23:43, Brian Burkhalter wrote: >>> : >>> >>>> So not clear to me that Files.copy methods needs to use this. >>> >>> So you are suggesting leaving the call to InputStream.transferTo() >>> in preference to IOSupport.copy()? >> Yes, I think these two should use transferTo. > > I don?t think that transferTo() can be used in JrtPath right now as > BUILD_JRTFS sets the compiler option ??release 8? and transferTo() was > added in 9. So unless this can be changed to 9, then JrtPath would > need either to remain unchanged or use IOSupport.copy(). > Sorry, when I said "these two should use transferTo" then I meant the Files.copy(Path, OutputStream) and Files.copy(InputStream, Path) methods, not the jrtfs code. You are right that jrtfs is complicated as it is compiled and packaged into jrtfs.jar for use by tools running on JDK 8. You'll have to leave that as is (it can't use IOSupport.copy or any other internal API). -Alan From vitalyd at gmail.com Thu Jan 4 12:05:05 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Thu, 4 Jan 2018 07:05:05 -0500 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: <356f9d2a-2682-ae17-6425-6b55d8cf5a36@oracle.com> References: <356f9d2a-2682-ae17-6425-6b55d8cf5a36@oracle.com> Message-ID: On Thu, Jan 4, 2018 at 3:03 AM, Alan Bateman wrote: > On 04/01/2018 01:35, David Holmes wrote: > >> >> You're not missing anything. It's a class initialization related >> "deadlock". >> >> Thread A has called FileSystems.getDefault() which is doing of >> DefaultFileSystemHolder, which in turn leads to Runtime.loadLibrary0 which >> needs to lock the Runtime instance. >> >> Thread B is already doing a loadLibrary and holds the Runtime lock, the >> loadLibrary code then needs to do FileSystems.getDefault() which has to >> load and initialize DefaultFileSystemHolder, but that initialization is >> already in progress so internally the thread does a wait(). >> >> So Thread B is waiting for Thread A to finish initialization, but holds >> the monitor lock that Thread A needs to finish the initialization. Deadlock. >> >> AFAICS this will still be possible in 9/10/11 >> > Startup and initialization has changed quite a bit since JDK 8. In JDK 9 > the file system is initialized early (long before main). However, it > changes again in JDK 10 due to ongoing work to makes things lazy and > improve startup time. Separately, FilePermission and the JarFile support > have been re-written so it should be loaded before any user code executes. > So I think needs to be studied again, I agree with your other mail to > create a bug to track that. Alan, David - thanks for your comments. Alan - I'm assuming someone from Oracle will file a bug report for this? > > > -Alan > From brian.burkhalter at oracle.com Thu Jan 4 17:18:47 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 4 Jan 2018 09:18:47 -0800 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> <9f81b074-5a62-2869-ad24-256eff0c0837@oracle.com> <8A10FB3C-A0FD-427E-BFDD-0315EE4D8A8A@oracle.com> Message-ID: On Jan 4, 2018, at 12:06 AM, Alan Bateman wrote: >>>>> So not clear to me that Files.copy methods needs to use this. >>>> >>>> So you are suggesting leaving the call to InputStream.transferTo() in preference to IOSupport.copy()? >>> Yes, I think these two should use transferTo. >> >> I don?t think that transferTo() can be used in JrtPath right now as BUILD_JRTFS sets the compiler option ??release 8? and transferTo() was added in 9. So unless this can be changed to 9, then JrtPath would need either to remain unchanged or use IOSupport.copy(). >> > Sorry, when I said "these two should use transferTo" then I meant the Files.copy(Path, OutputStream) and Files.copy(InputStream, Path) methods, not the jrtfs code. You are right that jrtfs is complicated as it is compiled and packaged into jrtfs.jar for use by tools running on JDK 8. You'll have to leave that as is (it can't use IOSupport.copy or any other internal API). Patch updated per this e-mail thread: http://cr.openjdk.java.net/~bpb/8194133/webrev.01/ Thanks, Brian From brian.burkhalter at oracle.com Thu Jan 4 18:14:33 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 4 Jan 2018 10:14:33 -0800 Subject: RFR 8194649: Minor cleanup of parameter checking in ByteArrayOutputStream and ObjectInputStream Message-ID: <8EF32576-288E-49A8-81E1-341CB4C649F7@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8194649 Please see the proposed change included below. In the OIS case the requireNonNull() call is not needed as a NPE would be thrown at the next line anyway if arrayType were null. Thanks, Brian --- a/src/java.base/share/classes/java/io/ByteArrayOutputStream.java +++ b/src/java.base/share/classes/java/io/ByteArrayOutputStream.java @@ -27,6 +27,7 @@ import java.nio.charset.Charset; import java.util.Arrays; +import java.util.Objects; /** * This class implements an output stream in which the data is @@ -147,10 +148,7 @@ * @param len the number of bytes to write. */ public synchronized void write(byte b[], int off, int len) { - if ((off < 0) || (off > b.length) || (len < 0) || - ((off + len) - b.length > 0)) { - throw new IndexOutOfBoundsException(); - } + Objects.checkFromIndexSize(off, len, b.length); ensureCapacity(count + len); System.arraycopy(b, off, buf, count, len); count += len; --- a/src/java.base/share/classes/java/io/ObjectInputStream.java +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java @@ -1296,7 +1296,6 @@ * @throws InvalidClassException if the filter rejects creation */ private void checkArray(Class arrayType, int arrayLength) throws InvalidClassException { - Objects.requireNonNull(arrayType); if (! arrayType.isArray()) { throw new IllegalArgumentException("not an array type"); } From roger.riggs at oracle.com Thu Jan 4 18:20:37 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Thu, 4 Jan 2018 13:20:37 -0500 Subject: RFR 8194649: Minor cleanup of parameter checking in ByteArrayOutputStream and ObjectInputStream In-Reply-To: <8EF32576-288E-49A8-81E1-341CB4C649F7@oracle.com> References: <8EF32576-288E-49A8-81E1-341CB4C649F7@oracle.com> Message-ID: <5e996ff6-d44d-9fb0-aa81-fb82730407da@oracle.com> +1, (There has been some variation in opinion about use of requireNonNull even where the VM would check a reference for null; but it does seem redundant on the face of it, even if hotspot would optimize it) On 1/4/18 1:14 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8194649 > > Please see the proposed change included below. In the OIS case the requireNonNull() call is not needed as a NPE would be thrown at the next line anyway if arrayType were null. > > Thanks, > > Brian > > --- a/src/java.base/share/classes/java/io/ByteArrayOutputStream.java > +++ b/src/java.base/share/classes/java/io/ByteArrayOutputStream.java > @@ -27,6 +27,7 @@ > > import java.nio.charset.Charset; > import java.util.Arrays; > +import java.util.Objects; > > /** > * This class implements an output stream in which the data is > @@ -147,10 +148,7 @@ > * @param len the number of bytes to write. > */ > public synchronized void write(byte b[], int off, int len) { > - if ((off < 0) || (off > b.length) || (len < 0) || > - ((off + len) - b.length > 0)) { > - throw new IndexOutOfBoundsException(); > - } > + Objects.checkFromIndexSize(off, len, b.length); > ensureCapacity(count + len); > System.arraycopy(b, off, buf, count, len); > count += len; > > --- a/src/java.base/share/classes/java/io/ObjectInputStream.java > +++ b/src/java.base/share/classes/java/io/ObjectInputStream.java > @@ -1296,7 +1296,6 @@ > * @throws InvalidClassException if the filter rejects creation > */ > private void checkArray(Class arrayType, int arrayLength) throws InvalidClassException { > - Objects.requireNonNull(arrayType); > if (! arrayType.isArray()) { > throw new IllegalArgumentException("not an array type"); > } From brian.burkhalter at oracle.com Thu Jan 4 21:05:29 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 4 Jan 2018 13:05:29 -0800 Subject: RFR 8193861: Typo in API documentation of File.toPath() Message-ID: <55253F55-6C85-4070-8B05-C5C24121BCD2@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8193861 http://cr.openjdk.java.net/~bpb/8193861/webrev.00/ Thanks, Brian From roger.riggs at oracle.com Thu Jan 4 21:25:21 2018 From: roger.riggs at oracle.com (Roger Riggs) Date: Thu, 4 Jan 2018 16:25:21 -0500 Subject: RFR 8193861: Typo in API documentation of File.toPath() In-Reply-To: <55253F55-6C85-4070-8B05-C5C24121BCD2@oracle.com> References: <55253F55-6C85-4070-8B05-C5C24121BCD2@oracle.com> Message-ID: +1 On 1/4/18 4:05 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8193861 > http://cr.openjdk.java.net/~bpb/8193861/webrev.00/ > > Thanks, > > Brian From brian.burkhalter at oracle.com Thu Jan 4 21:29:12 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 4 Jan 2018 13:29:12 -0800 Subject: Fix typo in InetSocketAddress.getAddress() documentation In-Reply-To: References: <001501d38571$b87943f0$296bcbd0$@freenet.de> Message-ID: Hi Chris, Sure, no problem: http://cr.openjdk.java.net/~bpb/8193861/webrev.01/index.html Brian On Jan 4, 2018, at 1:13 PM, Chris Hegarty wrote: > Would you mind adding the below typo to another you are fixing under JDK-8193861 ( to reduce the overhead ). From brian.burkhalter at oracle.com Thu Jan 4 21:39:00 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 4 Jan 2018 13:39:00 -0800 Subject: RFR 8193861: Typo in API documentation of File.toPath() In-Reply-To: <55253F55-6C85-4070-8B05-C5C24121BCD2@oracle.com> References: <55253F55-6C85-4070-8B05-C5C24121BCD2@oracle.com> Message-ID: Updated to include a fix for a typo in InetSocketAddress.getAddress() [1]: http://cr.openjdk.java.net/~bpb/8193861/webrev.01/index.html Thanks, Brian [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-January/050814.html On Jan 4, 2018, at 1:05 PM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8193861 > http://cr.openjdk.java.net/~bpb/8193861/webrev.00/ > > Thanks, > > Brian From lance.andersen at oracle.com Thu Jan 4 21:41:43 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 4 Jan 2018 16:41:43 -0500 Subject: RFR 8193861: Typo in API documentation of File.toPath() In-Reply-To: References: <55253F55-6C85-4070-8B05-C5C24121BCD2@oracle.com> Message-ID: <8A22C281-55A0-415A-BEAF-D068EF346825@oracle.com> +1 > On Jan 4, 2018, at 4:39 PM, Brian Burkhalter wrote: > > Updated to include a fix for a typo in InetSocketAddress.getAddress() [1]: > > http://cr.openjdk.java.net/~bpb/8193861/webrev.01/index.html > > Thanks, > > Brian > > [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2018-January/050814.html > > On Jan 4, 2018, at 1:05 PM, Brian Burkhalter wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8193861 >> http://cr.openjdk.java.net/~bpb/8193861/webrev.00/ >> >> Thanks, >> >> Brian > Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From mandy.chung at oracle.com Thu Jan 4 22:52:45 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 4 Jan 2018 14:52:45 -0800 Subject: Possible VM deadlock due to FileSystems.getDefault and System.loadLibrary interplay In-Reply-To: <356f9d2a-2682-ae17-6425-6b55d8cf5a36@oracle.com> References: <356f9d2a-2682-ae17-6425-6b55d8cf5a36@oracle.com> Message-ID: <8c35cdc9-dfeb-22f3-bc44-e8532f1f4626@oracle.com> On 1/4/18 12:03 AM, Alan Bateman wrote: > On 04/01/2018 01:35, David Holmes wrote: >> >> You're not missing anything. It's a class initialization related >> "deadlock". >> >> Thread A has called FileSystems.getDefault() which is doing >> of DefaultFileSystemHolder, which in turn leads to >> Runtime.loadLibrary0 which needs to lock the Runtime instance. >> >> Thread B is already doing a loadLibrary and holds the Runtime lock, >> the loadLibrary code then needs to do FileSystems.getDefault() which >> has to load and initialize DefaultFileSystemHolder, but that >> initialization is already in progress so internally the thread does a >> wait(). >> >> So Thread B is waiting for Thread A to finish initialization, but >> holds the monitor lock that Thread A needs to finish the >> initialization. Deadlock. >> >> AFAICS this will still be possible in 9/10/11 > Startup and initialization has changed quite a bit since JDK 8. In JDK > 9 the file system is initialized early (long before main). However, it > changes again in JDK 10 due to ongoing work to makes things lazy and > improve startup time. Separately, FilePermission and the JarFile > support have been re-written so it should be loaded before any user > code executes. So I think needs to be studied again, I agree with your > other mail to create a bug to track that. > This is indeed a deadlock.?? I created https://bugs.openjdk.java.net/browse/JDK-8194653 to track this. Mandy From brian.burkhalter at oracle.com Thu Jan 4 23:42:08 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 4 Jan 2018 15:42:08 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> Message-ID: On Dec 11, 2017, at 12:52 PM, Brian Burkhalter wrote: > On Dec 8, 2017, at 3:12 PM, Brian Burkhalter wrote: > >> All previous suggested changes have been made in >> >> http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ >> >> except for the possible change of name for the IS and OS nullStream() methods which awaits consensus. > > As long as we?re still at it, given java.util.stream.Stream then > > InputStream.nullInput() > OutputStream.nullOutput() > Reader.nullReader() > Writer.nullWriter() > > might not be a bad alternative. Re-reading the discussion [1] of last year regarding [2], it seems that there was convergence on all points raised except the names of the new methods. The alternatives are: A) InputStream.nullStream() and OutputStream.nullStream() as in the most recent version, webrev.03, B) InputStream.nullInput() and OutputStream.nullOutput() as shown above, and C) InputStream.nullInputStream() and OutputStream.nullOutputStream(). Does anyone have any more comments on this point? I would be inclined to choose option C. Thanks, Brian [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050367.html [2] https://bugs.openjdk.java.net/browse/JDK-4358774 From chris.hegarty at oracle.com Thu Jan 4 21:39:02 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Thu, 4 Jan 2018 21:39:02 +0000 Subject: Fix typo in InetSocketAddress.getAddress() documentation In-Reply-To: References: <001501d38571$b87943f0$296bcbd0$@freenet.de> Message-ID: <30211254-FBF6-41CE-9C4A-0C76475C5D02@oracle.com> > On 4 Jan 2018, at 21:29, Brian Burkhalter wrote: > > Hi Chris, > > Sure, no problem: http://cr.openjdk.java.net/~bpb/8193861/webrev.01/index.html Reviewed. Thank you. -Chris. From amy.lu at oracle.com Fri Jan 5 03:55:52 2018 From: amy.lu at oracle.com (Amy Lu) Date: Fri, 5 Jan 2018 11:55:52 +0800 Subject: [10] RFR 8194666: ProblemList update for bugid associated with PreferredKey.java, ConcurrentHashMapTest and SSLSocketParametersTest.sh Message-ID: <1c6a6aa2-e632-1749-ffb3-c2bc1bd5d98d@oracle.com> Please review this minor cleanup for test/jdk/ProblemList.txt on bugid that associated with tests. bug: https://bugs.openjdk.java.net/browse/JDK-8194666 webrev: http://cr.openjdk.java.net/~amlu/8194666/webrev.00/ Thanks, Amy --- old/test/jdk/ProblemList.txt 2018-01-05 11:41:00.000000000 +0800 +++ new/test/jdk/ProblemList.txt 2018-01-05 11:41:00.000000000 +0800 @@ -264,7 +264,7 @@ sun/security/krb5/auto/UnboundSSL.java 8180265 windows-all sun/security/provider/KeyStore/DKSTest.sh 8180266 windows-all -sun/security/ssl/X509KeyManager/PreferredKey.java 8176354 generic-all +sun/security/ssl/X509KeyManager/PreferredKey.java 8190333 generic-all ############################################################################ @@ -363,12 +363,12 @@ com/sun/jndi/ldap/DeadSSLLdapTimeoutTest.java 8169942 linux-i586,macosx-all,windows-x64 -javax/rmi/PortableRemoteObject/8146975/RmiIiopReturnValueTest.java 8169737 generic-all +javax/rmi/PortableRemoteObject/8146975/RmiIiopReturnValueTest.java 8194663 generic-all -org/omg/CORBA/OrbPropertiesTest.java 8175177 generic-all +org/omg/CORBA/OrbPropertiesTest.java 8194663 generic-all -javax/rmi/PortableRemoteObject/ConcurrentHashMapTest.java 8080643 generic-all +javax/rmi/PortableRemoteObject/ConcurrentHashMapTest.java 8194663 generic-all -javax/rmi/ssl/SSLSocketParametersTest.sh 8162906 generic-all +javax/rmi/ssl/SSLSocketParametersTest.sh 8194663 generic-all ############################################################################ From xuelei.fan at oracle.com Fri Jan 5 04:20:36 2018 From: xuelei.fan at oracle.com (Xuelei Fan) Date: Thu, 4 Jan 2018 20:20:36 -0800 Subject: [10] RFR 8194666: ProblemList update for bugid associated with PreferredKey.java, ConcurrentHashMapTest and SSLSocketParametersTest.sh In-Reply-To: <1c6a6aa2-e632-1749-ffb3-c2bc1bd5d98d@oracle.com> References: <1c6a6aa2-e632-1749-ffb3-c2bc1bd5d98d@oracle.com> Message-ID: I'm not very sure of the fix problems of JDK-8176354. But this changeset Looks fine to me. Thanks, Xuelei On 1/4/2018 7:55 PM, Amy Lu wrote: > Please review this minor cleanup for test/jdk/ProblemList.txt on bugid > that associated with tests. > > bug: https://bugs.openjdk.java.net/browse/JDK-8194666 > webrev: http://cr.openjdk.java.net/~amlu/8194666/webrev.00/ > > Thanks, > Amy > > --- old/test/jdk/ProblemList.txt 2018-01-05 11:41:00.000000000 +0800 > +++ new/test/jdk/ProblemList.txt 2018-01-05 11:41:00.000000000 +0800 > @@ -264,7 +264,7 @@ > > sun/security/krb5/auto/UnboundSSL.java 8180265 windows-all > sun/security/provider/KeyStore/DKSTest.sh 8180266 windows-all > -sun/security/ssl/X509KeyManager/PreferredKey.java 8176354 generic-all > +sun/security/ssl/X509KeyManager/PreferredKey.java 8190333 generic-all > > ############################################################################ > > @@ -363,12 +363,12 @@ > > com/sun/jndi/ldap/DeadSSLLdapTimeoutTest.java 8169942 linux-i586,macosx-all,windows-x64 > > -javax/rmi/PortableRemoteObject/8146975/RmiIiopReturnValueTest.java 8169737 generic-all > +javax/rmi/PortableRemoteObject/8146975/RmiIiopReturnValueTest.java 8194663 generic-all > > -org/omg/CORBA/OrbPropertiesTest.java 8175177 generic-all > +org/omg/CORBA/OrbPropertiesTest.java 8194663 generic-all > > -javax/rmi/PortableRemoteObject/ConcurrentHashMapTest.java 8080643 generic-all > +javax/rmi/PortableRemoteObject/ConcurrentHashMapTest.java 8194663 generic-all > > -javax/rmi/ssl/SSLSocketParametersTest.sh 8162906 generic-all > +javax/rmi/ssl/SSLSocketParametersTest.sh 8194663 generic-all > > ############################################################################ > From chris.hegarty at oracle.com Fri Jan 5 08:42:34 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Fri, 5 Jan 2018 08:42:34 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> Message-ID: <2F4256A9-65E6-4A3D-888E-7F947D25F5DB@oracle.com> > On 4 Jan 2018, at 23:42, Brian Burkhalter wrote: > >> On Dec 11, 2017, at 12:52 PM, Brian Burkhalter wrote: >> >>> On Dec 8, 2017, at 3:12 PM, Brian Burkhalter wrote: >>> >>> All previous suggested changes have been made in >>> >>> http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ >>> >>> except for the possible change of name for the IS and OS nullStream() methods which awaits consensus. >> >> As long as we?re still at it, given java.util.stream.Stream then >> >> InputStream.nullInput() >> OutputStream.nullOutput() >> Reader.nullReader() >> Writer.nullWriter() >> >> might not be a bad alternative. > > Re-reading the discussion [1] of last year regarding [2], it seems that there was convergence on all points raised except the names of the new methods. The alternatives are: > > A) InputStream.nullStream() and OutputStream.nullStream() as in the most recent version, webrev.03, > B) InputStream.nullInput() and OutputStream.nullOutput() as shown above, and > C) InputStream.nullInputStream() and OutputStream.nullOutputStream(). > > Does anyone have any more comments on this point? I would be inclined to choose option C. After re-reading the email thread, I would choose option C too. It is unambiguous, and the ?null{classname}? pattern can be used for Reader/Writer in the future. -Chris > Thanks, > > Brian > > [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050367.html > [2] https://bugs.openjdk.java.net/browse/JDK-4358774 From Alan.Bateman at oracle.com Fri Jan 5 11:27:29 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 5 Jan 2018 11:27:29 +0000 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> <9f81b074-5a62-2869-ad24-256eff0c0837@oracle.com> <8A10FB3C-A0FD-427E-BFDD-0315EE4D8A8A@oracle.com> Message-ID: <091d87af-5374-0bb8-ef98-e0be442316e7@oracle.com> On 04/01/2018 17:18, Brian Burkhalter wrote: > > Patch updated per this e-mail thread: > > http://cr.openjdk.java.net/~bpb/8194133/webrev.01/ > > Sorry for prolonging this one but is sun.net.NetworkServer used by anything? I suspect it's a left over from early JDK releases and can be deleted. -Alan From Roger.Riggs at Oracle.com Fri Jan 5 15:31:37 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 5 Jan 2018 10:31:37 -0500 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <2F4256A9-65E6-4A3D-888E-7F947D25F5DB@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> <2F4256A9-65E6-4A3D-888E-7F947D25F5DB@oracle.com> Message-ID: <327ed71a-dfe0-f375-4e34-699db99c51d4@Oracle.com> +1 C) InputStream.nullInputStream() and OutputStream.nullOutputStream(). On 1/5/2018 3:42 AM, Chris Hegarty wrote: >> On 4 Jan 2018, at 23:42, Brian Burkhalter wrote: >> >>> On Dec 11, 2017, at 12:52 PM, Brian Burkhalter wrote: >>> >>>> On Dec 8, 2017, at 3:12 PM, Brian Burkhalter wrote: >>>> >>>> All previous suggested changes have been made in >>>> >>>> http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ >>>> >>>> except for the possible change of name for the IS and OS nullStream() methods which awaits consensus. >>> As long as we?re still at it, given java.util.stream.Stream then >>> >>> InputStream.nullInput() >>> OutputStream.nullOutput() >>> Reader.nullReader() >>> Writer.nullWriter() >>> >>> might not be a bad alternative. >> Re-reading the discussion [1] of last year regarding [2], it seems that there was convergence on all points raised except the names of the new methods. The alternatives are: >> >> A) InputStream.nullStream() and OutputStream.nullStream() as in the most recent version, webrev.03, >> B) InputStream.nullInput() and OutputStream.nullOutput() as shown above, and >> C) InputStream.nullInputStream() and OutputStream.nullOutputStream(). >> >> Does anyone have any more comments on this point? I would be inclined to choose option C. > After re-reading the email thread, I would choose option C too. It is unambiguous, and the ?null{classname}? pattern can be used for Reader/Writer in the future. > > -Chris > > >> Thanks, >> >> Brian >> >> [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050367.html >> [2] https://bugs.openjdk.java.net/browse/JDK-4358774 From patrick at reini.net Fri Jan 5 17:20:36 2018 From: patrick at reini.net (Patrick Reinhart) Date: Fri, 5 Jan 2018 18:20:36 +0100 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <327ed71a-dfe0-f375-4e34-699db99c51d4@Oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> <2F4256A9-65E6-4A3D-888E-7F947D25F5DB@oracle.com> <327ed71a-dfe0-f375-4e34-699db99c51d4@Oracle.com> Message-ID: <7A4FDFBD-6137-4F12-90C9-2F5871FFE9A7@reini.net> That?s my favorite too? -Patrick > Am 05.01.2018 um 16:31 schrieb Roger Riggs : > > +1 > > C) InputStream.nullInputStream() and OutputStream.nullOutputStream(). From brian.burkhalter at oracle.com Fri Jan 5 17:54:08 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 5 Jan 2018 09:54:08 -0800 Subject: RFR 8193842: Refactor InputStream-to-OutputStream copy into a utility method In-Reply-To: <091d87af-5374-0bb8-ef98-e0be442316e7@oracle.com> References: <448C891C-06DA-41E9-87DC-6B0A8AB0A781@oracle.com> <232750C7-7D37-4F36-A842-AE4D38273C91@oracle.com> <9f81b074-5a62-2869-ad24-256eff0c0837@oracle.com> <8A10FB3C-A0FD-427E-BFDD-0315EE4D8A8A@oracle.com> <091d87af-5374-0bb8-ef98-e0be442316e7@oracle.com> Message-ID: <04B4C968-7C23-4CE2-B3C7-D6715A8C6724@oracle.com> Indeed it looks like NetworkServer could be removed. Question is what else in that package could be removed? Could be a separate issue. On Jan 5, 2018, at 3:27 AM, Alan Bateman wrote: > Sorry for prolonging this one but is sun.net.NetworkServer used by anything? I suspect it's a left over from early JDK releases and can be deleted. From volker.simonis at gmail.com Sun Jan 7 12:10:05 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Sun, 07 Jan 2018 12:10:05 +0000 Subject: Runtime code generation and barriers in migrating away from JVM-interal APIs In-Reply-To: References: Message-ID: Hi Rafael, thanks for the nice write up and your IMO reasonable suggestions. I?m forwarding your mail to jdk-dev and core-libs-dev which are more appropriate for this topic (adoption-discuss is more about bundling and distros). Regards, Volker ------- Weitergeleitete Nachricht ------ Von: Rafael Winterhalter Datum: So. 7. Jan. 2018 um 00:08 Betreff: Runtime code generation and barriers in migrating away from JVM-interal APIs An: Hello, I am the author of Byte Buddy and a maintainer of cglib, two of the major code generation libraries in the Java ecosystem. Both libraries are downloaded about 160 million times a year and I wanted to give a report and opinion on the current state of moving away from JVM-internal APIs to save and officially supported alternatives. When code generation tools define classes at runtime, there are currently different alternatives to achieving that: 1) Using sun.misc.Unsafe::defineClass to define a class directly. This is a fairly easy API that even allows defining classes in the bootstrap loader. 2) Accessing protected methods of java.lang.ClassLoader via Java reflection. This allows more fine-grained access to class loading by respecting class loading locks etc. 3) Creating a custom class loader as a parent of the class loader of a proxied class. This avoids any use of internal API but limits proxying to public (and since Java 9 also exported) classes and their protected and public methods. 4) Using a Java agent to define classes using the Instrumentation API. Using this API, it is also possible to gain access to internal APIs as it becomes possible to open encapsulated APIs. 5) Using JNI to define classes using its APIs or to avoid encapsulation altogether. Of course, strategies (1) and (2) were always discouraged and might no longer work in a future Java release due to the encapsulation of internal APIs. Yet, as of Java 9, most code generation tools achieved Java 9 compatibility by migrating from solution (2) to solution (1) thanks to the jdk.unsupported module. Method (5) is rarely used as it requires the inclusion of C code for something that can be achieved easier. As of Java 9, the JVM offers a new approach to defining classes: 6) Using java.lang.invoke.MethodHandles.Lookup::defineClass While Byte Buddy supports this new approach as a user-chosen class definition strategy, for most use cases, the API does not offer sufficient comfort. Code generation is mainly used for the following two purposes: A) When defining a proxy, the proxy class is normally defined in the same package as the proxied class. Doing so, a proxy can be created for package-private classes and it can proxy package-private methods. Using strategy (6), it is however not possible to define a class in a package outside of the package that has created the lookup as this would require PACKAGE access for the target package. If the proxy is created by another module then the module of the proxied class, this access right is never available, even if the proxied class?s module opens its package to the module that generates the proxy. In this context, Strategy (3) is not an option either as the runtime package of the child class loader would be different to the user class package?s class loader. B) When programming a Java agent, a class enhancement makes it sometimes necessary to define an auxiliary class in the same package as the instrumented class. This is similar to javac?s need for such classes where it sometimes defines anonymous classes to provide a certain type for using an API. Unfortunately, the ClassFileTransformer::transform method does not provide a method handle lookup for the package of the instrumented class. At the same time, the Java agent itself typically lives in a different package then the instrumented class such that it cannot create its own lookup what makes (6) inapplicable. Of course, (3) is not an option in this case either. For scenario (A) one could argue that for many use cases, access to package-private classes and methods is not necessary as it breaks the Java programming language's encapsulation model. However, giving such access has been proven useful in the past: the Spring framework does for example induce a bean scope when defining a Java configuration class?s method as package-private. And for the Mockito framework, such access allows for the creation of package-private mocks what avoids that users have to extend the scope of such classes only for a unit test. For (B), a Java agent is able to access internal APIs by opening packages. Providing a method handle lookup for the target class would however offer a cleaner, more standardized approach. It is however unclear what lookupClass the method handle lookup would be assigned to as the instrumented class is not necessarily loaded when the class file transformer is applied. Additionally, some proxying tools such as Mockito require an API to instantiate a class without invoking a constructor. This way, a mock can be created without triggering any user code which might have unwanted side-effects or throw an exception for invalid inputs that are unknown to the mocking framework. I understand that such instantiations are frowned upon as they break the object model. But again, this possibility has proven to be very useful in the past and it would be too bad if such libraries could no longer be maintained in the future. To create instances without invoking a constructor, there are currently several options: 7) Use sun.misc.Unsafe::allocateInstance or the also internal reflection factory. Such use is often done via the Objenesis library. If such access was encapsulated, a Java agent could still open these APIs. 8) Using JNI to avoid encapsulation or allocating an instance without a constructor call from JNI. Again, (8) is a rarely chosen approach but (7) via the use of Objenesis is still common. At a result, even with Java 9 being supported by many popular frameworks, a migration away from internal APIs has not yet been achieved. I would therefore like to suggest the following extensions: C) When a module opens a package, other modules should gain package access to this package when creating method handle lookups. This way, if a user opens a package containing Spring beans to the Spring framework, it could proxy all of these beans as it does today. Since opening a package also permits reflection on package-private types and methods of this package, this is not a security concern either. D) A class file transformer should be provided with an instance of a method handle lookup for the instrumented class as an argument. This way, Java agents gain an easy and standardized way of defining auxiliary classes what is currently rather cumbersome. E) There should be a jdk.test module that is not resolved by default and that is not part of a non-JDK distribution of the JVM that contains an API that allows for the instantiation of classes without a constructor invocation. By depending on this module, test libraries that offer such insecure abilities can also make their intention clear that a library is meant for test and not for production. With Mockito, we regularly get inquiries about performance issues when the library is used in production systems what it is not designed for. This module could also include an API for getting hold of an Instrumentation instance for the current JVM process. This would be useful for many testing libraries such as Mockito and also for testing Java agents under development. Currently, it is necessary to self-attach using the attachment API. Since Java 9, it is additionally required to explicitly allow such self-attachment or to use an intermediate Java process to avoid the constraint. With these three extensions, I believe that the many users of code generation tools could easily migrate away from the use of internal APIs in a few months what would allow a full encapsulation of JVM-internal APIs without any major disruptions. Thank you for your time and feedback on my proposal! Best regards, Rafael From peter.levart at gmail.com Sun Jan 7 16:40:40 2018 From: peter.levart at gmail.com (Peter Levart) Date: Sun, 7 Jan 2018 17:40:40 +0100 Subject: Runtime code generation and barriers in migrating away from JVM-interal APIs In-Reply-To: References: Message-ID: Hi Rafael, On 01/07/18 13:10, Volker Simonis wrote: > At a result, even with Java 9 being supported by many popular frameworks, a > migration away from internal APIs has not yet been achieved. I would > therefore like to suggest the following extensions: > > C) When a module opens a package, other modules should gain package access > to this package when creating method handle lookups. This way, if a user > opens a package containing Spring beans to the Spring framework, it could > proxy all of these beans as it does today. Since opening a package also > permits reflection on package-private types and methods of this package, > this is not a security concern either. Have you checked the new JDK 9 method: java.lang.invoke.MethodHandles#privateLookupIn ? I believe it should do the trick. Regards, Peter From rafael.wth at gmail.com Sun Jan 7 17:46:33 2018 From: rafael.wth at gmail.com (Rafael Winterhalter) Date: Sun, 7 Jan 2018 18:46:33 +0100 Subject: Runtime code generation and barriers in migrating away from JVM-interal APIs In-Reply-To: References: Message-ID: You are right Peter, that does work. I assumed that the module to which a package was opened automatically received the access rights for any lookup object, thanks for pointing it out to me. This makes my proposal of C obsolete. I hope that D and E are considered nevertheless! Also, thank you Volker for forwarding this to the correct mailinglists. 2018-01-07 17:40 GMT+01:00 Peter Levart : > Hi Rafael, > > On 01/07/18 13:10, Volker Simonis wrote: > > At a result, even with Java 9 being supported by many popular frameworks, a > migration away from internal APIs has not yet been achieved. I would > therefore like to suggest the following extensions: > > C) When a module opens a package, other modules should gain package access > to this package when creating method handle lookups. This way, if a user > opens a package containing Spring beans to the Spring framework, it could > proxy all of these beans as it does today. Since opening a package also > permits reflection on package-private types and methods of this package, > this is not a security concern either. > > > Have you checked the new JDK 9 method: java.lang.invoke.MethodHandles#privateLookupIn > ? > > I believe it should do the trick. > > Regards, Peter > > From amy.lu at oracle.com Mon Jan 8 02:41:22 2018 From: amy.lu at oracle.com (Amy Lu) Date: Mon, 8 Jan 2018 10:41:22 +0800 Subject: [10] RFR 8194662: Problem list com/sun/jndi/ldap/LdapTimeoutTest.java Message-ID: Please review the patch to problem list com/sun/jndi/ldap/LdapTimeoutTest.java This test fails frequently and should be problem listed before?JDK-8151678?fixed. bug: https://bugs.openjdk.java.net/browse/JDK-8194662 webrev: http://cr.openjdk.java.net/~amlu/8194662/webrev.00/ Thanks, Amy --- old/test/jdk/ProblemList.txt??? 2018-01-08 10:37:20.000000000 +0800 +++ new/test/jdk/ProblemList.txt??? 2018-01-08 10:37:20.000000000 +0800 @@ -363,6 +363,8 @@ ?com/sun/jndi/ldap/DeadSSLLdapTimeoutTest.java 8169942 linux-i586,macosx-all,windows-x64 +com/sun/jndi/ldap/LdapTimeoutTest.java 8151678 linux-all + ?javax/rmi/PortableRemoteObject/8146975/RmiIiopReturnValueTest.java 8194663 generic-all ?org/omg/CORBA/OrbPropertiesTest.java??? ??? ??? ??????? 8194663 generic-all From david.holmes at oracle.com Mon Jan 8 03:07:12 2018 From: david.holmes at oracle.com (David Holmes) Date: Mon, 8 Jan 2018 13:07:12 +1000 Subject: [10] RFR 8194662: Problem list com/sun/jndi/ldap/LdapTimeoutTest.java In-Reply-To: References: Message-ID: Looks good Amy! Thanks, David On 8/01/2018 12:41 PM, Amy Lu wrote: > Please review the patch to problem list > com/sun/jndi/ldap/LdapTimeoutTest.java > > This test fails frequently and should be problem listed > before?JDK-8151678?fixed. > > bug: https://bugs.openjdk.java.net/browse/JDK-8194662 > webrev: http://cr.openjdk.java.net/~amlu/8194662/webrev.00/ > > Thanks, > Amy > > --- old/test/jdk/ProblemList.txt??? 2018-01-08 10:37:20.000000000 +0800 > +++ new/test/jdk/ProblemList.txt??? 2018-01-08 10:37:20.000000000 +0800 > @@ -363,6 +363,8 @@ > > ?com/sun/jndi/ldap/DeadSSLLdapTimeoutTest.java 8169942 > linux-i586,macosx-all,windows-x64 > > +com/sun/jndi/ldap/LdapTimeoutTest.java 8151678 linux-all > + > ?javax/rmi/PortableRemoteObject/8146975/RmiIiopReturnValueTest.java > 8194663 generic-all > > ?org/omg/CORBA/OrbPropertiesTest.java??? ??? ??? ??????? 8194663 > generic-all > > > From claes.redestad at oracle.com Mon Jan 8 11:37:10 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 8 Jan 2018 12:37:10 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: Hi Stuart et al, I've gone through the feedback (thanks, everyone!) and made the following adjustments which I hope covers most, if not all, concerns: - Per John's suggestion I've moved most of the methods in List12/ListN to AbstractImmutableList, using for-loops rather than iterators etc. I've not found evidence that the suggested @ForceInline + trivial @Override trick is necessary for the methods implemented here to match ArrayList, so left this out for now (examining this in-depth might be a good follow-up). - Extracted common code between immutable List and Set implementations to AbstractImmutableCollection and addedAbstractImmutableSet (the only method that was pulled in and not overridden from AbstractSet was equals, so this seems like a cleaner abstraction with minimal copy-paste) - Rolled up the changes requested by JDK-8191418 into this patch, i.e., the following calls now throw a NPE as expected: ? List.of(..).subList(..).indexOf(null); ? List.of(..).subList(..).lastIndexOf(null); ? List.of(..).subList(..).contains(null); Additionally Set.of(..).contains(null) and Map.of(..).containsValue/-Key(null) had an inconsistency discovered when adding tests to this. This is going against offline suggestions to break this patch into smaller parts, e.g. implement JDK-8191418[1] separately, merge List0 into ListN in a standalone patch etc.. I agree, but think teasing the constituent pieces at play *now* would be a large effort for small gain. If faced with another round of overwhelming feedback/criticism I'm open to but not enthusiastic about splitting this apart. :-) - Per Jason Mehren's suggestion, I've moved EMPTY_* static finals to their respective ListN/SetN/MapN class. I think they might all be initialized early in the bootstrap, but felt cleaner regardless. - I've NOT changed the subList() to return a Serializable view for now. Experimentally verified that having List.of().subList() return a List12 or ListN as appropriate *could* give a speedup at call-sites that observe List12, ListN and SubList, but to achieve this we would either have to redesign ListN to keep track of an offset + size (which has a small performance and footprint cost), or have subList() actually copy the backing array. Either alternative seems somewhat unappealing, and take the conservative approach for now. And making SubList Serializable would be a simple change at a later date if deemed appropriate.. - Added a set of indexOf / lastIndexOf sanity tests for all(?) List implementations in java.util to MOAT - Added tests to ListFactories ensuring indexOf(null) / lastIndexOf(null) / contains(null) are consistent across immutable list and subList implementations - Added a test to SetFactories ensuring contains(null) throw NPE for all implementations - Added tests to MapFactories ensuring containsKey(null)/containsValue(null) consistently throw NPE for all implementations - Verified micro performance numbers hold up, and added micros for hashCode, indexOf.. [2] http://cr.openjdk.java.net/~redestad/8193128/open.05/ Regards /Claes [1] https://bugs.openjdk.java.net/browse/JDK-8191418 [2] http://cr.openjdk.java.net/~redestad/8193128/ListMorphism.java Baseline: Benchmark??????????????????????????? Mode? Cnt ?? Score?? Error Units ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.985 ? 0.550 ns/op ListMorphism.finalGetFromList??????? avgt?? 25?? 15.081 ? 0.016 ns/op ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.474 ? 0.343 ns/op ListMorphism.finalSumSizesList?????? avgt?? 25?? 13.817 ? 0.009 ns/op ListMorphism.getFromArrayList??????? avgt?? 25?? 46.473 ? 0.049 ns/op ListMorphism.getFromArrayList12????? avgt?? 25?? 34.711 ? 1.680 ns/op ListMorphism.getFromList???????????? avgt?? 25?? 88.202 ? 0.500 ns/op ListMorphism.getFromList12?????????? avgt?? 25?? 28.904 ? 0.041 ns/op ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.603 ? 0.012 ns/op ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.584 ? 0.019 ns/op ListMorphism.sumSizesList??????????? avgt?? 25?? 59.781 ? 2.666 ns/op ListMorphism.sumSizesList12????????? avgt?? 25?? 17.589 ? 0.036 ns/op ListMorphism.arrayListHash?????????? avgt?? 25? 121.163 ? 7.160 ns/op ListMorphism.listHash??????????????? avgt?? 25? 104.532 ? 0.783 ns/op ListMorphism.arrayListIndexOf??????? avgt?? 25 ? 74.204 ? 0.515 ns/op ListMorphism.listIndexOf???????????? avgt?? 25? 529.105 ? 8.281 ns/op Patch: Benchmark??????????????????????????? Mode? Cnt??? Score?? Error Units ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.744 ? 0.110 ns/op ListMorphism.finalGetFromList??????? avgt?? 25?? 13.862 ? 0.085 ns/op ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.326 ? 0.012 ns/op ListMorphism.finalSumSizesList?????? avgt?? 25?? 15.249 ? 0.658 ns/op ListMorphism.getFromArrayList??????? avgt?? 25?? 46.520 ? 0.220 ns/op ListMorphism.getFromArrayList12????? avgt?? 25?? 33.922 ? 0.051 ns/op ListMorphism.getFromList???????????? avgt?? 25?? 40.186 ? 0.019 ns/op ListMorphism.getFromList12?????????? avgt?? 25?? 27.741 ? 0.241 ns/op ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.908 ? 1.051 ns/op ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.656 ? 0.121 ns/op ListMorphism.sumSizesList??????????? avgt?? 25?? 29.342 ? 1.410 ns/op ListMorphism.sumSizesList12????????? avgt?? 25?? 20.180 ? 0.148 ns/op ListMorphism.arrayListHash?????????? avgt?? 25? 117.427 ? 7.805 ns/op ListMorphism.listHash ? ? ? ?? ????? avgt?? 25? 110.268 ? 7.485 ns/op ListMorphism.arrayListIndexOf??????? avgt?? 25?? 75.051 ? 2.539 ns/op ListMorphism.listIndexOf????? ? ? ?? avgt?? 25?? 76.609 ? 0.121 ns/op The arrayListHash/listHash results are inconclusive due to a bimodal run-to-run distribution in my micro but are close enough for comfort. The ~7x listIndexOf improvement is large and stable, and stem mainly from moving from the inefficient use of iterators previously inherited from AbstractList. On 2017-12-22 02:04, Stuart Marks wrote: > Finally catching up with this thread.... > > Yes, there should be some additional test cases added. When I added > the immutable collection classes in JDK 9, I did modify MOAT.java so > that its test cases would apply to the new implementations. > > A few more cases could be added that apply not only to the immutable > lists but also to lists in general. > > I think the bugs mentioned here are with indexOf() and lastIndexOf(), > with the latter accidentally being a copy of the former. Later in the > thread John pointed out an off-by-one error with lastIndexOf(), so > edge cases should be added as well. These would apply to all lists, > not just the immutable ones. > > There is also the issue mentioned in > > ??? JDK-8191418 List.of().indexOf(null) doesn't throw > NullPointerException > > Tests should be added for that and related methods. Since many > collections permit null, and I suspect some that disallow null might > permit indexOf(null) and related, it's not clear to me these belong in > MOAT.java. > > But if List.of().indexOf(null) were to throw NPE, I'd expect > List.of().subList(...).indexOf(null) also to throw NPE. > > s'marks > > > On 12/8/17 9:44 AM, Martin Buchholz wrote: >> Should there be changes to general purpose collection testing classes >> like >> test/jdk/java/util/concurrent/tck/CollectionTest.java >> test/jdk/java/util/Collection/MOAT.java >> that would have caught this bug? >> (although those are mostly designed for mutable collections, but I >> think some immutable collections (nCopies?) get tested somewhere. >> >> On Fri, Dec 8, 2017 at 7:13 AM, Claes Redestad >> > wrote: >> >> ??? Hi, >> >> ??? On 2017-12-08 07:54, Andrej Golovnin wrote: >> >> ??????? Hi Claes, >> >> http://cr.openjdk.java.net/~redestad/8193128/open.02/ >> >> >> ??????? I think you should provide specialised implementations of the >> ??????? #indexOf() and #lastIndexOf() methods in the classes List12 >> and ListN. >> ??????? Using an iterator in List12 is an overkill. But even in ListN >> using a >> ??????? simple for loop would be much better. >> >> >> ??? it's somewhat ironic that I started looking at this *because* >> ??? indexOf was slow due use of iterators[1], but then never got >> ??? around to specialize them in this patch. >> >> ??????? In any case please take look at >> ??????? the implementation of the #lastIndexOf() method in the >> ??????? AbstractImmutableList class. It looks like a copy of >> ??????? AbstractImmutableList#indexOf() and this is wrong. >> >> >> ??? Nice catch! Quite the embarrassing copy-paste that... >> >> ??? - Specialized the index/lastIndexOf methods for List12, ListN >> ??? - Fixed implementation of lastIndexOf in AbstractImmutableList. >> ??? - As AbstractImmutableList.indexOf/lastIndexOf are now only used >> ??? by AbstractImmutableList.SubList, I moved them there with some >> ??? commentary since it's not clear JDK-8191418 should add null >> ??? checkson the input for SubList or not. >> ??? - Added sanity tests of indexOf/lastIndexOf that touches all >> ??? the new implementations: >> >> ??? http://cr.openjdk.java.net/~redestad/8193128/open.03/ >> >> >> ??? Thanks! >> >> ??? /Claes >> >> ??? [1] https://bugs.openjdk.java.net/browse/JDK-8191442 >> ??? >> >> From andrej.golovnin at gmail.com Mon Jan 8 13:25:51 2018 From: andrej.golovnin at gmail.com (Andrej Golovnin) Date: Mon, 8 Jan 2018 14:25:51 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: Hi Claes, > http://cr.openjdk.java.net/~redestad/8193128/open.05/ src/java.base/share/classes/java/util/ImmutableCollections.java 599 public boolean contains(Object o) { 600 Objects.requireNonNull(o); 601 return size > 0 && probe(o) >= 0; // implicit nullcheck of o 602 } I think that the comment about the implicit null check in the line 601 is not needed anymore. Best regards, Andrej Golovnin From claes.redestad at oracle.com Mon Jan 8 15:00:51 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 8 Jan 2018 16:00:51 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: Hi Andrej, thanks for catching this. Webrev updated in-place. Regards /Claes On 2018-01-08 14:25, Andrej Golovnin wrote: > Hi Claes, > >> http://cr.openjdk.java.net/~redestad/8193128/open.05/ > src/java.base/share/classes/java/util/ImmutableCollections.java > > 599 public boolean contains(Object o) { > 600 Objects.requireNonNull(o); > 601 return size > 0 && probe(o) >= 0; // implicit nullcheck of o > 602 } > > I think that the comment about the implicit null check in the line 601 > is not needed anymore. > > Best regards, > Andrej Golovnin From peter.levart at gmail.com Mon Jan 8 19:09:02 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 8 Jan 2018 20:09:02 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> Message-ID: <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> Hi Claes, Are you sure that AbstractImmutableSet.equals(Object) is correct? ??????? @Override ??????? public boolean equals(Object o) { ??????????? if (o == this) ??????????????? return true; ??????????? if (!(o instanceof Set)) ??????????????? return false; ??????????? Collection c = (Collection) o; ??????????? try { ??????????????? return containsAll(c); ??????????? } catch (ClassCastException | NullPointerException unused) { ??????????????? return false; ??????????? } ??????? } It seems to me that this implementation returns true when passing in a sub-set of this. Should the method also check that the size(s) of both sets are the same? Regards, Peter On 01/08/18 12:37, Claes Redestad wrote: > Hi Stuart et al, > > I've gone through the feedback (thanks, everyone!) and made the > following adjustments which I > hope covers most, if not all, concerns: > > - Per John's suggestion I've moved most of the methods in List12/ListN > to AbstractImmutableList, > using for-loops rather than iterators etc. I've not found evidence > that the suggested > @ForceInline + trivial @Override trick is necessary for the methods > implemented here to match > ArrayList, so left this out for now (examining this in-depth might be > a good follow-up). > > - Extracted common code between immutable List and Set implementations to > AbstractImmutableCollection and addedAbstractImmutableSet (the only > method that was pulled > in and not overridden from AbstractSet was equals, so this seems like > a cleaner abstraction with > minimal copy-paste) > > - Rolled up the changes requested by JDK-8191418 into this patch, > i.e., the following calls > now throw a NPE as expected: > > ? List.of(..).subList(..).indexOf(null); > ? List.of(..).subList(..).lastIndexOf(null); > ? List.of(..).subList(..).contains(null); > > Additionally Set.of(..).contains(null) and > Map.of(..).containsValue/-Key(null) had an inconsistency > discovered when adding tests to this. > > This is going against offline suggestions to break this patch into > smaller parts, e.g. implement > JDK-8191418[1] separately, merge List0 into ListN in a standalone > patch etc.. I agree, but > think teasing the constituent pieces at play *now* would be a large > effort for small gain. > > If faced with another round of overwhelming feedback/criticism I'm > open to but not enthusiastic > about splitting this apart. :-) > > - Per Jason Mehren's suggestion, I've moved EMPTY_* static finals to > their respective > ListN/SetN/MapN class. I think they might all be initialized early in > the bootstrap, but felt > cleaner regardless. > > - I've NOT changed the subList() to return a Serializable view for now. > > Experimentally verified that having List.of().subList() return a > List12 or ListN as appropriate > *could* give a speedup at call-sites that observe List12, ListN and > SubList, but to achieve this > we would either have to redesign ListN to keep track of an offset + > size (which has a small > performance and footprint cost), or have subList() actually copy the > backing array. > > Either alternative seems somewhat unappealing, and take the > conservative approach for now. And > making SubList Serializable would be a simple change at a later date > if deemed appropriate.. > > - Added a set of indexOf / lastIndexOf sanity tests for all(?) List > implementations in java.util to > MOAT > > - Added tests to ListFactories ensuring indexOf(null) / > lastIndexOf(null) / contains(null) are > consistent across immutable list and subList implementations > > - Added a test to SetFactories ensuring contains(null) throw NPE for > all implementations > > - Added tests to MapFactories ensuring > containsKey(null)/containsValue(null) consistently throw > NPE for all implementations > > - Verified micro performance numbers hold up, and added micros for > hashCode, indexOf.. [2] > > http://cr.openjdk.java.net/~redestad/8193128/open.05/ > > Regards > > /Claes > > [1] https://bugs.openjdk.java.net/browse/JDK-8191418 > [2] > > http://cr.openjdk.java.net/~redestad/8193128/ListMorphism.java > > Baseline: > Benchmark??????????????????????????? Mode? Cnt ?? Score?? Error Units > ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.985 ? 0.550 ns/op > ListMorphism.finalGetFromList??????? avgt?? 25?? 15.081 ? 0.016 ns/op > ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.474 ? 0.343 ns/op > ListMorphism.finalSumSizesList?????? avgt?? 25?? 13.817 ? 0.009 ns/op > ListMorphism.getFromArrayList??????? avgt?? 25?? 46.473 ? 0.049 ns/op > ListMorphism.getFromArrayList12????? avgt?? 25?? 34.711 ? 1.680 ns/op > ListMorphism.getFromList???????????? avgt?? 25?? 88.202 ? 0.500 ns/op > ListMorphism.getFromList12?????????? avgt?? 25?? 28.904 ? 0.041 ns/op > ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.603 ? 0.012 ns/op > ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.584 ? 0.019 ns/op > ListMorphism.sumSizesList??????????? avgt?? 25?? 59.781 ? 2.666 ns/op > ListMorphism.sumSizesList12????????? avgt?? 25?? 17.589 ? 0.036 ns/op > ListMorphism.arrayListHash?????????? avgt?? 25? 121.163 ? 7.160 ns/op > ListMorphism.listHash??????????????? avgt?? 25? 104.532 ? 0.783 ns/op > ListMorphism.arrayListIndexOf??????? avgt?? 25 ? 74.204 ? 0.515 ns/op > ListMorphism.listIndexOf???????????? avgt?? 25? 529.105 ? 8.281 ns/op > > Patch: > Benchmark??????????????????????????? Mode? Cnt??? Score?? Error Units > ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.744 ? 0.110 ns/op > ListMorphism.finalGetFromList??????? avgt?? 25?? 13.862 ? 0.085 ns/op > ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.326 ? 0.012 ns/op > ListMorphism.finalSumSizesList?????? avgt?? 25?? 15.249 ? 0.658 ns/op > ListMorphism.getFromArrayList??????? avgt?? 25?? 46.520 ? 0.220 ns/op > ListMorphism.getFromArrayList12????? avgt?? 25?? 33.922 ? 0.051 ns/op > ListMorphism.getFromList???????????? avgt?? 25?? 40.186 ? 0.019 ns/op > ListMorphism.getFromList12?????????? avgt?? 25?? 27.741 ? 0.241 ns/op > ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.908 ? 1.051 ns/op > ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.656 ? 0.121 ns/op > ListMorphism.sumSizesList??????????? avgt?? 25?? 29.342 ? 1.410 ns/op > ListMorphism.sumSizesList12????????? avgt?? 25?? 20.180 ? 0.148 ns/op > ListMorphism.arrayListHash?????????? avgt?? 25? 117.427 ? 7.805 ns/op > ListMorphism.listHash ? ? ? ?? ????? avgt?? 25? 110.268 ? 7.485 ns/op > ListMorphism.arrayListIndexOf??????? avgt?? 25?? 75.051 ? 2.539 ns/op > ListMorphism.listIndexOf????? ? ? ?? avgt?? 25?? 76.609 ? 0.121 ns/op > > The arrayListHash/listHash results are inconclusive due to a bimodal > run-to-run > distribution in my micro but are close enough for comfort. The ~7x > listIndexOf improvement is large and stable, and stem mainly from moving > from the inefficient use of iterators previously inherited from > AbstractList. > > On 2017-12-22 02:04, Stuart Marks wrote: >> Finally catching up with this thread.... >> >> Yes, there should be some additional test cases added. When I added >> the immutable collection classes in JDK 9, I did modify MOAT.java so >> that its test cases would apply to the new implementations. >> >> A few more cases could be added that apply not only to the immutable >> lists but also to lists in general. >> >> I think the bugs mentioned here are with indexOf() and lastIndexOf(), >> with the latter accidentally being a copy of the former. Later in the >> thread John pointed out an off-by-one error with lastIndexOf(), so >> edge cases should be added as well. These would apply to all lists, >> not just the immutable ones. >> >> There is also the issue mentioned in >> >> ??? JDK-8191418 List.of().indexOf(null) doesn't throw >> NullPointerException >> >> Tests should be added for that and related methods. Since many >> collections permit null, and I suspect some that disallow null might >> permit indexOf(null) and related, it's not clear to me these belong >> in MOAT.java. >> >> But if List.of().indexOf(null) were to throw NPE, I'd expect >> List.of().subList(...).indexOf(null) also to throw NPE. >> >> s'marks >> >> >> On 12/8/17 9:44 AM, Martin Buchholz wrote: >>> Should there be changes to general purpose collection testing >>> classes like >>> test/jdk/java/util/concurrent/tck/CollectionTest.java >>> test/jdk/java/util/Collection/MOAT.java >>> that would have caught this bug? >>> (although those are mostly designed for mutable collections, but I >>> think some immutable collections (nCopies?) get tested somewhere. >>> >>> On Fri, Dec 8, 2017 at 7:13 AM, Claes Redestad >>> > wrote: >>> >>> ??? Hi, >>> >>> ??? On 2017-12-08 07:54, Andrej Golovnin wrote: >>> >>> ??????? Hi Claes, >>> >>> http://cr.openjdk.java.net/~redestad/8193128/open.02/ >>> >>> >>> ??????? I think you should provide specialised implementations of the >>> ??????? #indexOf() and #lastIndexOf() methods in the classes List12 >>> and ListN. >>> ??????? Using an iterator in List12 is an overkill. But even in >>> ListN using a >>> ??????? simple for loop would be much better. >>> >>> >>> ??? it's somewhat ironic that I started looking at this *because* >>> ??? indexOf was slow due use of iterators[1], but then never got >>> ??? around to specialize them in this patch. >>> >>> ??????? In any case please take look at >>> ??????? the implementation of the #lastIndexOf() method in the >>> ??????? AbstractImmutableList class. It looks like a copy of >>> ??????? AbstractImmutableList#indexOf() and this is wrong. >>> >>> >>> ??? Nice catch! Quite the embarrassing copy-paste that... >>> >>> ??? - Specialized the index/lastIndexOf methods for List12, ListN >>> ??? - Fixed implementation of lastIndexOf in AbstractImmutableList. >>> ??? - As AbstractImmutableList.indexOf/lastIndexOf are now only used >>> ??? by AbstractImmutableList.SubList, I moved them there with some >>> ??? commentary since it's not clear JDK-8191418 should add null >>> ??? checkson the input for SubList or not. >>> ??? - Added sanity tests of indexOf/lastIndexOf that touches all >>> ??? the new implementations: >>> >>> ??? http://cr.openjdk.java.net/~redestad/8193128/open.03/ >>> >>> >>> ??? Thanks! >>> >>> ??? /Claes >>> >>> ??? [1] https://bugs.openjdk.java.net/browse/JDK-8191442 >>> ??? >>> >>> > From peter.levart at gmail.com Mon Jan 8 19:38:20 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 8 Jan 2018 20:38:20 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> Message-ID: Also, I don't think that ClassCastException should be caught here. It should never be thrown by containsAll(c) anyway. On 01/08/18 20:09, Peter Levart wrote: > Hi Claes, > > Are you sure that AbstractImmutableSet.equals(Object) is correct? > > ??????? @Override > ??????? public boolean equals(Object o) { > ??????????? if (o == this) > ??????????????? return true; > > ??????????? if (!(o instanceof Set)) > ??????????????? return false; > ??????????? Collection c = (Collection) o; > ??????????? try { > ??????????????? return containsAll(c); > ??????????? } catch (ClassCastException | NullPointerException unused) { > ??????????????? return false; > ??????????? } > ??????? } > > > It seems to me that this implementation returns true when passing in a > sub-set of this. Should the method also check that the size(s) of both > sets are the same? > > Regards, Peter > > On 01/08/18 12:37, Claes Redestad wrote: >> Hi Stuart et al, >> >> I've gone through the feedback (thanks, everyone!) and made the >> following adjustments which I >> hope covers most, if not all, concerns: >> >> - Per John's suggestion I've moved most of the methods in >> List12/ListN to AbstractImmutableList, >> using for-loops rather than iterators etc. I've not found evidence >> that the suggested >> @ForceInline + trivial @Override trick is necessary for the methods >> implemented here to match >> ArrayList, so left this out for now (examining this in-depth might be >> a good follow-up). >> >> - Extracted common code between immutable List and Set >> implementations to >> AbstractImmutableCollection and addedAbstractImmutableSet (the only >> method that was pulled >> in and not overridden from AbstractSet was equals, so this seems like >> a cleaner abstraction with >> minimal copy-paste) >> >> - Rolled up the changes requested by JDK-8191418 into this patch, >> i.e., the following calls >> now throw a NPE as expected: >> >> ? List.of(..).subList(..).indexOf(null); >> ? List.of(..).subList(..).lastIndexOf(null); >> ? List.of(..).subList(..).contains(null); >> >> Additionally Set.of(..).contains(null) and >> Map.of(..).containsValue/-Key(null) had an inconsistency >> discovered when adding tests to this. >> >> This is going against offline suggestions to break this patch into >> smaller parts, e.g. implement >> JDK-8191418[1] separately, merge List0 into ListN in a standalone >> patch etc.. I agree, but >> think teasing the constituent pieces at play *now* would be a large >> effort for small gain. >> >> If faced with another round of overwhelming feedback/criticism I'm >> open to but not enthusiastic >> about splitting this apart. :-) >> >> - Per Jason Mehren's suggestion, I've moved EMPTY_* static finals to >> their respective >> ListN/SetN/MapN class. I think they might all be initialized early in >> the bootstrap, but felt >> cleaner regardless. >> >> - I've NOT changed the subList() to return a Serializable view for now. >> >> Experimentally verified that having List.of().subList() return a >> List12 or ListN as appropriate >> *could* give a speedup at call-sites that observe List12, ListN and >> SubList, but to achieve this >> we would either have to redesign ListN to keep track of an offset + >> size (which has a small >> performance and footprint cost), or have subList() actually copy the >> backing array. >> >> Either alternative seems somewhat unappealing, and take the >> conservative approach for now. And >> making SubList Serializable would be a simple change at a later date >> if deemed appropriate.. >> >> - Added a set of indexOf / lastIndexOf sanity tests for all(?) List >> implementations in java.util to >> MOAT >> >> - Added tests to ListFactories ensuring indexOf(null) / >> lastIndexOf(null) / contains(null) are >> consistent across immutable list and subList implementations >> >> - Added a test to SetFactories ensuring contains(null) throw NPE for >> all implementations >> >> - Added tests to MapFactories ensuring >> containsKey(null)/containsValue(null) consistently throw >> NPE for all implementations >> >> - Verified micro performance numbers hold up, and added micros for >> hashCode, indexOf.. [2] >> >> http://cr.openjdk.java.net/~redestad/8193128/open.05/ >> >> Regards >> >> /Claes >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8191418 >> [2] >> >> http://cr.openjdk.java.net/~redestad/8193128/ListMorphism.java >> >> Baseline: >> Benchmark??????????????????????????? Mode? Cnt ?? Score?? Error Units >> ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.985 ? 0.550 ns/op >> ListMorphism.finalGetFromList??????? avgt?? 25?? 15.081 ? 0.016 ns/op >> ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.474 ? 0.343 ns/op >> ListMorphism.finalSumSizesList?????? avgt?? 25?? 13.817 ? 0.009 ns/op >> ListMorphism.getFromArrayList??????? avgt?? 25?? 46.473 ? 0.049 ns/op >> ListMorphism.getFromArrayList12????? avgt?? 25?? 34.711 ? 1.680 ns/op >> ListMorphism.getFromList???????????? avgt?? 25?? 88.202 ? 0.500 ns/op >> ListMorphism.getFromList12?????????? avgt?? 25?? 28.904 ? 0.041 ns/op >> ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.603 ? 0.012 ns/op >> ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.584 ? 0.019 ns/op >> ListMorphism.sumSizesList??????????? avgt?? 25?? 59.781 ? 2.666 ns/op >> ListMorphism.sumSizesList12????????? avgt?? 25?? 17.589 ? 0.036 ns/op >> ListMorphism.arrayListHash?????????? avgt?? 25? 121.163 ? 7.160 ns/op >> ListMorphism.listHash??????????????? avgt?? 25? 104.532 ? 0.783 ns/op >> ListMorphism.arrayListIndexOf??????? avgt?? 25 ? 74.204 ? 0.515 ns/op >> ListMorphism.listIndexOf???????????? avgt?? 25? 529.105 ? 8.281 ns/op >> >> Patch: >> Benchmark??????????????????????????? Mode? Cnt??? Score?? Error Units >> ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.744 ? 0.110 ns/op >> ListMorphism.finalGetFromList??????? avgt?? 25?? 13.862 ? 0.085 ns/op >> ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.326 ? 0.012 ns/op >> ListMorphism.finalSumSizesList?????? avgt?? 25?? 15.249 ? 0.658 ns/op >> ListMorphism.getFromArrayList??????? avgt?? 25?? 46.520 ? 0.220 ns/op >> ListMorphism.getFromArrayList12????? avgt?? 25?? 33.922 ? 0.051 ns/op >> ListMorphism.getFromList???????????? avgt?? 25?? 40.186 ? 0.019 ns/op >> ListMorphism.getFromList12?????????? avgt?? 25?? 27.741 ? 0.241 ns/op >> ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.908 ? 1.051 ns/op >> ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.656 ? 0.121 ns/op >> ListMorphism.sumSizesList??????????? avgt?? 25?? 29.342 ? 1.410 ns/op >> ListMorphism.sumSizesList12????????? avgt?? 25?? 20.180 ? 0.148 ns/op >> ListMorphism.arrayListHash?????????? avgt?? 25? 117.427 ? 7.805 ns/op >> ListMorphism.listHash ? ? ? ?? ????? avgt?? 25? 110.268 ? 7.485 ns/op >> ListMorphism.arrayListIndexOf??????? avgt?? 25?? 75.051 ? 2.539 ns/op >> ListMorphism.listIndexOf????? ? ? ?? avgt?? 25?? 76.609 ? 0.121 ns/op >> >> The arrayListHash/listHash results are inconclusive due to a bimodal >> run-to-run >> distribution in my micro but are close enough for comfort. The ~7x >> listIndexOf improvement is large and stable, and stem mainly from moving >> from the inefficient use of iterators previously inherited from >> AbstractList. >> >> On 2017-12-22 02:04, Stuart Marks wrote: >>> Finally catching up with this thread.... >>> >>> Yes, there should be some additional test cases added. When I added >>> the immutable collection classes in JDK 9, I did modify MOAT.java so >>> that its test cases would apply to the new implementations. >>> >>> A few more cases could be added that apply not only to the immutable >>> lists but also to lists in general. >>> >>> I think the bugs mentioned here are with indexOf() and >>> lastIndexOf(), with the latter accidentally being a copy of the >>> former. Later in the thread John pointed out an off-by-one error >>> with lastIndexOf(), so edge cases should be added as well. These >>> would apply to all lists, not just the immutable ones. >>> >>> There is also the issue mentioned in >>> >>> ??? JDK-8191418 List.of().indexOf(null) doesn't throw >>> NullPointerException >>> >>> Tests should be added for that and related methods. Since many >>> collections permit null, and I suspect some that disallow null might >>> permit indexOf(null) and related, it's not clear to me these belong >>> in MOAT.java. >>> >>> But if List.of().indexOf(null) were to throw NPE, I'd expect >>> List.of().subList(...).indexOf(null) also to throw NPE. >>> >>> s'marks >>> >>> >>> On 12/8/17 9:44 AM, Martin Buchholz wrote: >>>> Should there be changes to general purpose collection testing >>>> classes like >>>> test/jdk/java/util/concurrent/tck/CollectionTest.java >>>> test/jdk/java/util/Collection/MOAT.java >>>> that would have caught this bug? >>>> (although those are mostly designed for mutable collections, but I >>>> think some immutable collections (nCopies?) get tested somewhere. >>>> >>>> On Fri, Dec 8, 2017 at 7:13 AM, Claes Redestad >>>> > wrote: >>>> >>>> ??? Hi, >>>> >>>> ??? On 2017-12-08 07:54, Andrej Golovnin wrote: >>>> >>>> ??????? Hi Claes, >>>> >>>> http://cr.openjdk.java.net/~redestad/8193128/open.02/ >>>> >>>> >>>> ??????? I think you should provide specialised implementations of the >>>> ??????? #indexOf() and #lastIndexOf() methods in the classes List12 >>>> and ListN. >>>> ??????? Using an iterator in List12 is an overkill. But even in >>>> ListN using a >>>> ??????? simple for loop would be much better. >>>> >>>> >>>> ??? it's somewhat ironic that I started looking at this *because* >>>> ??? indexOf was slow due use of iterators[1], but then never got >>>> ??? around to specialize them in this patch. >>>> >>>> ??????? In any case please take look at >>>> ??????? the implementation of the #lastIndexOf() method in the >>>> ??????? AbstractImmutableList class. It looks like a copy of >>>> ??????? AbstractImmutableList#indexOf() and this is wrong. >>>> >>>> >>>> ??? Nice catch! Quite the embarrassing copy-paste that... >>>> >>>> ??? - Specialized the index/lastIndexOf methods for List12, ListN >>>> ??? - Fixed implementation of lastIndexOf in AbstractImmutableList. >>>> ??? - As AbstractImmutableList.indexOf/lastIndexOf are now only used >>>> ??? by AbstractImmutableList.SubList, I moved them there with some >>>> ??? commentary since it's not clear JDK-8191418 should add null >>>> ??? checkson the input for SubList or not. >>>> ??? - Added sanity tests of indexOf/lastIndexOf that touches all >>>> ??? the new implementations: >>>> >>>> http://cr.openjdk.java.net/~redestad/8193128/open.03/ >>>> >>>> >>>> ??? Thanks! >>>> >>>> ??? /Claes >>>> >>>> ??? [1] https://bugs.openjdk.java.net/browse/JDK-8191442 >>>> >>>> >>>> >> > From peter.levart at gmail.com Mon Jan 8 19:46:36 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 8 Jan 2018 20:46:36 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> Message-ID: On 01/08/18 20:09, Peter Levart wrote: > > Should the method also check that the size(s) of both sets are the same? Or better yet, compute the size of the other set as you iterate the elements. Like this: ??????? public boolean equals(Object o) { ??????????? if (o == this) ??????????????? return true; ??????????? if (!(o instanceof Set)) ??????????????? return false; ??????????? int osize = 0; ??????????? for (Object e : (Iterable) o) { ??????????????? if (!contains(e)) { ??????????????????? return false; ??????????????? } ??????????????? osize++; ??????????? } ??????????? return size() == osize; ??????? } ...since calling .size() on the passed-in Set might not be free. Regards, Peter > > Regards, Peter > > On 01/08/18 12:37, Claes Redestad wrote: >> Hi Stuart et al, >> >> I've gone through the feedback (thanks, everyone!) and made the >> following adjustments which I >> hope covers most, if not all, concerns: >> >> - Per John's suggestion I've moved most of the methods in >> List12/ListN to AbstractImmutableList, >> using for-loops rather than iterators etc. I've not found evidence >> that the suggested >> @ForceInline + trivial @Override trick is necessary for the methods >> implemented here to match >> ArrayList, so left this out for now (examining this in-depth might be >> a good follow-up). >> >> - Extracted common code between immutable List and Set >> implementations to >> AbstractImmutableCollection and addedAbstractImmutableSet (the only >> method that was pulled >> in and not overridden from AbstractSet was equals, so this seems like >> a cleaner abstraction with >> minimal copy-paste) >> >> - Rolled up the changes requested by JDK-8191418 into this patch, >> i.e., the following calls >> now throw a NPE as expected: >> >> ? List.of(..).subList(..).indexOf(null); >> ? List.of(..).subList(..).lastIndexOf(null); >> ? List.of(..).subList(..).contains(null); >> >> Additionally Set.of(..).contains(null) and >> Map.of(..).containsValue/-Key(null) had an inconsistency >> discovered when adding tests to this. >> >> This is going against offline suggestions to break this patch into >> smaller parts, e.g. implement >> JDK-8191418[1] separately, merge List0 into ListN in a standalone >> patch etc.. I agree, but >> think teasing the constituent pieces at play *now* would be a large >> effort for small gain. >> >> If faced with another round of overwhelming feedback/criticism I'm >> open to but not enthusiastic >> about splitting this apart. :-) >> >> - Per Jason Mehren's suggestion, I've moved EMPTY_* static finals to >> their respective >> ListN/SetN/MapN class. I think they might all be initialized early in >> the bootstrap, but felt >> cleaner regardless. >> >> - I've NOT changed the subList() to return a Serializable view for now. >> >> Experimentally verified that having List.of().subList() return a >> List12 or ListN as appropriate >> *could* give a speedup at call-sites that observe List12, ListN and >> SubList, but to achieve this >> we would either have to redesign ListN to keep track of an offset + >> size (which has a small >> performance and footprint cost), or have subList() actually copy the >> backing array. >> >> Either alternative seems somewhat unappealing, and take the >> conservative approach for now. And >> making SubList Serializable would be a simple change at a later date >> if deemed appropriate.. >> >> - Added a set of indexOf / lastIndexOf sanity tests for all(?) List >> implementations in java.util to >> MOAT >> >> - Added tests to ListFactories ensuring indexOf(null) / >> lastIndexOf(null) / contains(null) are >> consistent across immutable list and subList implementations >> >> - Added a test to SetFactories ensuring contains(null) throw NPE for >> all implementations >> >> - Added tests to MapFactories ensuring >> containsKey(null)/containsValue(null) consistently throw >> NPE for all implementations >> >> - Verified micro performance numbers hold up, and added micros for >> hashCode, indexOf.. [2] >> >> http://cr.openjdk.java.net/~redestad/8193128/open.05/ >> >> Regards >> >> /Claes >> >> [1] https://bugs.openjdk.java.net/browse/JDK-8191418 >> [2] >> >> http://cr.openjdk.java.net/~redestad/8193128/ListMorphism.java >> >> Baseline: >> Benchmark??????????????????????????? Mode? Cnt ?? Score?? Error Units >> ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.985 ? 0.550 ns/op >> ListMorphism.finalGetFromList??????? avgt?? 25?? 15.081 ? 0.016 ns/op >> ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.474 ? 0.343 ns/op >> ListMorphism.finalSumSizesList?????? avgt?? 25?? 13.817 ? 0.009 ns/op >> ListMorphism.getFromArrayList??????? avgt?? 25?? 46.473 ? 0.049 ns/op >> ListMorphism.getFromArrayList12????? avgt?? 25?? 34.711 ? 1.680 ns/op >> ListMorphism.getFromList???????????? avgt?? 25?? 88.202 ? 0.500 ns/op >> ListMorphism.getFromList12?????????? avgt?? 25?? 28.904 ? 0.041 ns/op >> ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.603 ? 0.012 ns/op >> ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.584 ? 0.019 ns/op >> ListMorphism.sumSizesList??????????? avgt?? 25?? 59.781 ? 2.666 ns/op >> ListMorphism.sumSizesList12????????? avgt?? 25?? 17.589 ? 0.036 ns/op >> ListMorphism.arrayListHash?????????? avgt?? 25? 121.163 ? 7.160 ns/op >> ListMorphism.listHash??????????????? avgt?? 25? 104.532 ? 0.783 ns/op >> ListMorphism.arrayListIndexOf??????? avgt?? 25 ? 74.204 ? 0.515 ns/op >> ListMorphism.listIndexOf???????????? avgt?? 25? 529.105 ? 8.281 ns/op >> >> Patch: >> Benchmark??????????????????????????? Mode? Cnt??? Score?? Error Units >> ListMorphism.finalGetFromArrayList?? avgt?? 25?? 37.744 ? 0.110 ns/op >> ListMorphism.finalGetFromList??????? avgt?? 25?? 13.862 ? 0.085 ns/op >> ListMorphism.finalSumSizesArrayList? avgt?? 25?? 16.326 ? 0.012 ns/op >> ListMorphism.finalSumSizesList?????? avgt?? 25?? 15.249 ? 0.658 ns/op >> ListMorphism.getFromArrayList??????? avgt?? 25?? 46.520 ? 0.220 ns/op >> ListMorphism.getFromArrayList12????? avgt?? 25?? 33.922 ? 0.051 ns/op >> ListMorphism.getFromList???????????? avgt?? 25?? 40.186 ? 0.019 ns/op >> ListMorphism.getFromList12?????????? avgt?? 25?? 27.741 ? 0.241 ns/op >> ListMorphism.sumSizesArrayList?????? avgt?? 25?? 22.908 ? 1.051 ns/op >> ListMorphism.sumSizesArrayList12???? avgt?? 25?? 17.656 ? 0.121 ns/op >> ListMorphism.sumSizesList??????????? avgt?? 25?? 29.342 ? 1.410 ns/op >> ListMorphism.sumSizesList12????????? avgt?? 25?? 20.180 ? 0.148 ns/op >> ListMorphism.arrayListHash?????????? avgt?? 25? 117.427 ? 7.805 ns/op >> ListMorphism.listHash ? ? ? ?? ????? avgt?? 25? 110.268 ? 7.485 ns/op >> ListMorphism.arrayListIndexOf??????? avgt?? 25?? 75.051 ? 2.539 ns/op >> ListMorphism.listIndexOf????? ? ? ?? avgt?? 25?? 76.609 ? 0.121 ns/op >> >> The arrayListHash/listHash results are inconclusive due to a bimodal >> run-to-run >> distribution in my micro but are close enough for comfort. The ~7x >> listIndexOf improvement is large and stable, and stem mainly from moving >> from the inefficient use of iterators previously inherited from >> AbstractList. >> >> On 2017-12-22 02:04, Stuart Marks wrote: >>> Finally catching up with this thread.... >>> >>> Yes, there should be some additional test cases added. When I added >>> the immutable collection classes in JDK 9, I did modify MOAT.java so >>> that its test cases would apply to the new implementations. >>> >>> A few more cases could be added that apply not only to the immutable >>> lists but also to lists in general. >>> >>> I think the bugs mentioned here are with indexOf() and >>> lastIndexOf(), with the latter accidentally being a copy of the >>> former. Later in the thread John pointed out an off-by-one error >>> with lastIndexOf(), so edge cases should be added as well. These >>> would apply to all lists, not just the immutable ones. >>> >>> There is also the issue mentioned in >>> >>> ??? JDK-8191418 List.of().indexOf(null) doesn't throw >>> NullPointerException >>> >>> Tests should be added for that and related methods. Since many >>> collections permit null, and I suspect some that disallow null might >>> permit indexOf(null) and related, it's not clear to me these belong >>> in MOAT.java. >>> >>> But if List.of().indexOf(null) were to throw NPE, I'd expect >>> List.of().subList(...).indexOf(null) also to throw NPE. >>> >>> s'marks >>> >>> >>> On 12/8/17 9:44 AM, Martin Buchholz wrote: >>>> Should there be changes to general purpose collection testing >>>> classes like >>>> test/jdk/java/util/concurrent/tck/CollectionTest.java >>>> test/jdk/java/util/Collection/MOAT.java >>>> that would have caught this bug? >>>> (although those are mostly designed for mutable collections, but I >>>> think some immutable collections (nCopies?) get tested somewhere. >>>> >>>> On Fri, Dec 8, 2017 at 7:13 AM, Claes Redestad >>>> > wrote: >>>> >>>> ??? Hi, >>>> >>>> ??? On 2017-12-08 07:54, Andrej Golovnin wrote: >>>> >>>> ??????? Hi Claes, >>>> >>>> http://cr.openjdk.java.net/~redestad/8193128/open.02/ >>>> >>>> >>>> ??????? I think you should provide specialised implementations of the >>>> ??????? #indexOf() and #lastIndexOf() methods in the classes List12 >>>> and ListN. >>>> ??????? Using an iterator in List12 is an overkill. But even in >>>> ListN using a >>>> ??????? simple for loop would be much better. >>>> >>>> >>>> ??? it's somewhat ironic that I started looking at this *because* >>>> ??? indexOf was slow due use of iterators[1], but then never got >>>> ??? around to specialize them in this patch. >>>> >>>> ??????? In any case please take look at >>>> ??????? the implementation of the #lastIndexOf() method in the >>>> ??????? AbstractImmutableList class. It looks like a copy of >>>> ??????? AbstractImmutableList#indexOf() and this is wrong. >>>> >>>> >>>> ??? Nice catch! Quite the embarrassing copy-paste that... >>>> >>>> ??? - Specialized the index/lastIndexOf methods for List12, ListN >>>> ??? - Fixed implementation of lastIndexOf in AbstractImmutableList. >>>> ??? - As AbstractImmutableList.indexOf/lastIndexOf are now only used >>>> ??? by AbstractImmutableList.SubList, I moved them there with some >>>> ??? commentary since it's not clear JDK-8191418 should add null >>>> ??? checkson the input for SubList or not. >>>> ??? - Added sanity tests of indexOf/lastIndexOf that touches all >>>> ??? the new implementations: >>>> >>>> http://cr.openjdk.java.net/~redestad/8193128/open.03/ >>>> >>>> >>>> ??? Thanks! >>>> >>>> ??? /Claes >>>> >>>> ??? [1] https://bugs.openjdk.java.net/browse/JDK-8191442 >>>> >>>> >>>> >> > From peter.levart at gmail.com Mon Jan 8 19:49:22 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 8 Jan 2018 20:49:22 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> Message-ID: <144cee4c-e671-1620-c60a-e9457a709d2c@gmail.com> On 01/08/18 20:46, Peter Levart wrote: > > Or better yet, compute the size of the other set as you iterate the > elements. Like this: > > ??????? public boolean equals(Object o) { > ??????????? if (o == this) > ??????????????? return true; > > ??????????? if (!(o instanceof Set)) > ??????????????? return false; > > ??????????? int osize = 0; > ??????????? for (Object e : (Iterable) o) { > ??????????????? if (!contains(e)) { > ??????????????????? return false; > ??????????????? } > ??????????????? osize++; > ??????????? } > ??????????? return size() == osize; > ??????? } > > ...since calling .size() on the passed-in Set might not be free. > Ops, forgot to check for null 'e': ??????? @Override ??????? public boolean equals(Object o) { ??????????? if (o == this) ??????????????? return true; ??????????? if (!(o instanceof Set)) ??????????????? return false; ??????????? int osize = 0; ??????????? for (Object e : (Iterable) o) { ??????????????? if (e == null || !contains(e)) { ??????????????????? return false; ??????????????? } ??????????????? osize++; ??????????? } ??????????? return size() == osize; ??????? } Regards, Peter From claes.redestad at oracle.com Mon Jan 8 20:57:00 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 8 Jan 2018 21:57:00 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> Message-ID: <14910571-3cc8-75a5-9f80-8d633b23ac4b@oracle.com> Gosh! My intent was to carry over AbstractSet::equals verbatim, so I have no idea how the size check got lost in translation and passed through testing! Appears to be another hole in the test coverage, or I didn't run the right ones. As to calculating o.size() as we go then I'm not sure: most Set::size() impls in the JDK are O(1) (including the ones here), and you lose a very fast fast path in common cases for a hypothetical improvement on some collection types whose size() isn't. Leaving it as is is also sticking with the status quo, which seems favorable for the scope of this RFE. CCE is specified to be thrown by Set::contains/containsAll, so I think we should play it safe and leave it unchanged. Thanks! /Claes On 2018-01-08 20:38, Peter Levart wrote: > Also, I don't think that ClassCastException should be caught here. It > should never be thrown by containsAll(c) anyway. > > On 01/08/18 20:09, Peter Levart wrote: >> Hi Claes, >> >> Are you sure that AbstractImmutableSet.equals(Object) is correct? >> >> ??????? @Override >> ??????? public boolean equals(Object o) { >> ??????????? if (o == this) >> ??????????????? return true; >> >> ??????????? if (!(o instanceof Set)) >> ??????????????? return false; >> ??????????? Collection c = (Collection) o; >> ??????????? try { >> ??????????????? return containsAll(c); >> ??????????? } catch (ClassCastException | NullPointerException unused) { >> ??????????????? return false; >> ??????????? } >> ??????? } >> >> >> It seems to me that this implementation returns true when passing in >> a sub-set of this. Should the method also check that the size(s) of >> both sets are the same? >> >> Regards, Peter From peter.levart at gmail.com Mon Jan 8 22:41:11 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 8 Jan 2018 23:41:11 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <14910571-3cc8-75a5-9f80-8d633b23ac4b@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> <14910571-3cc8-75a5-9f80-8d633b23ac4b@oracle.com> Message-ID: <94ea8e2e-418c-4408-2dc2-7bea1ca099d4@gmail.com> Hi Claes, On 01/08/18 21:57, Claes Redestad wrote: > Gosh! My intent was to carry over AbstractSet::equals verbatim, so I > have no idea how the size > check got lost in translation and passed through testing! Appears to > be another hole in the test > coverage, or I didn't run the right ones. > > As to calculating o.size() as we go then I'm not sure: most > Set::size() impls in the JDK are O(1) > (including the ones here), and you lose a very fast fast path in > common cases for a hypothetical > improvement on some collection types whose size() isn't. Leaving it as > is is also sticking with > the status quo, which seems favorable for the scope of this RFE. > > CCE is specified to be thrown by Set::contains/containsAll, so I think > we should play it safe and > leave it unchanged. It's probably specified so to accommodate for SortedSet(s) which don't use equals but compareTo / Comparator. But here you are calling containsAll on an abstract Set with known implementation(s) which are hash-based and CCE will never be thrown. If CCE is thrown from element's equals() then it should probably propagate out of AbstractImmutableSet.equals() and not silently translate to non-equality of sets. What do you think? Regards, Peter > > Thanks! > > /Claes > > On 2018-01-08 20:38, Peter Levart wrote: >> Also, I don't think that ClassCastException should be caught here. It >> should never be thrown by containsAll(c) anyway. >> >> On 01/08/18 20:09, Peter Levart wrote: >>> Hi Claes, >>> >>> Are you sure that AbstractImmutableSet.equals(Object) is correct? >>> >>> ??????? @Override >>> ??????? public boolean equals(Object o) { >>> ??????????? if (o == this) >>> ??????????????? return true; >>> >>> ??????????? if (!(o instanceof Set)) >>> ??????????????? return false; >>> ??????????? Collection c = (Collection) o; >>> ??????????? try { >>> ??????????????? return containsAll(c); >>> ??????????? } catch (ClassCastException | NullPointerException >>> unused) { >>> ??????????????? return false; >>> ??????????? } >>> ??????? } >>> >>> >>> It seems to me that this implementation returns true when passing in >>> a sub-set of this. Should the method also check that the size(s) of >>> both sets are the same? >>> >>> Regards, Peter > From peter.levart at gmail.com Mon Jan 8 23:04:30 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 9 Jan 2018 00:04:30 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <94ea8e2e-418c-4408-2dc2-7bea1ca099d4@gmail.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> <14910571-3cc8-75a5-9f80-8d633b23ac4b@oracle.com> <94ea8e2e-418c-4408-2dc2-7bea1ca099d4@gmail.com> Message-ID: Hi Claes, On 01/08/18 23:41, Peter Levart wrote: > Hi Claes, > > On 01/08/18 21:57, Claes Redestad wrote: >> Gosh! My intent was to carry over AbstractSet::equals verbatim, so I >> have no idea how the size >> check got lost in translation and passed through testing! Appears to >> be another hole in the test >> coverage, or I didn't run the right ones. >> >> As to calculating o.size() as we go then I'm not sure: most >> Set::size() impls in the JDK are O(1) >> (including the ones here), and you lose a very fast fast path in >> common cases for a hypothetical >> improvement on some collection types whose size() isn't. Leaving it >> as is is also sticking with >> the status quo, which seems favorable for the scope of this RFE. >> >> CCE is specified to be thrown by Set::contains/containsAll, so I >> think we should play it safe and >> leave it unchanged. > > It's probably specified so to accommodate for SortedSet(s) which don't > use equals but compareTo / Comparator. But here you are calling > containsAll on an abstract Set with known implementation(s) which are > hash-based and CCE will never be thrown. If CCE is thrown from > element's equals() then it should probably propagate out of > AbstractImmutableSet.equals() and not silently translate to > non-equality of sets. > > What do you think? I just noticed that AbstractSet does the same (catches CCE). That's probably because of SortedSet subclasses too. So if you think AbstractImmutableSet might be used for some immutable SortedSet implementations in the future, you should probably leave it unchanged. If elemen't equals() throws CCE it is violating the spec so it does not matter what the outcome is (although probably propagating CCE might help catching the bug). If you envision other AbstractImmutableSet subclasses in the future, then perhaps it would be good to declare an abstract hashCode() method in it. So implementations can't forget to define it. When one sees that equals() is already defined, he might falsely assume that hashCode() is defined too. Regards, Peter > > Regards, Peter > >> >> Thanks! >> >> /Claes >> >> On 2018-01-08 20:38, Peter Levart wrote: >>> Also, I don't think that ClassCastException should be caught here. >>> It should never be thrown by containsAll(c) anyway. >>> >>> On 01/08/18 20:09, Peter Levart wrote: >>>> Hi Claes, >>>> >>>> Are you sure that AbstractImmutableSet.equals(Object) is correct? >>>> >>>> ??????? @Override >>>> ??????? public boolean equals(Object o) { >>>> ??????????? if (o == this) >>>> ??????????????? return true; >>>> >>>> ??????????? if (!(o instanceof Set)) >>>> ??????????????? return false; >>>> ??????????? Collection c = (Collection) o; >>>> ??????????? try { >>>> ??????????????? return containsAll(c); >>>> ??????????? } catch (ClassCastException | NullPointerException >>>> unused) { >>>> ??????????????? return false; >>>> ??????????? } >>>> ??????? } >>>> >>>> >>>> It seems to me that this implementation returns true when passing >>>> in a sub-set of this. Should the method also check that the size(s) >>>> of both sets are the same? >>>> >>>> Regards, Peter >> > From martinrb at google.com Tue Jan 9 00:52:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 8 Jan 2018 16:52:24 -0800 Subject: RFR: jsr166 jdk integration 2018-01 Message-ID: http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html 8191483: AbstractQueuedSynchronizer cancel/cancel race http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/AQS-cancel-cancel-race/index.html https://bugs.openjdk.java.net/browse/JDK-8191483 Lots of minor changes inspired by code analysis tools: 8193300: Miscellaneous changes imported from jsr166 CVS 2018-01 http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/miscellaneous/index.html https://bugs.openjdk.java.net/browse/JDK-8193300 From claes.redestad at oracle.com Tue Jan 9 01:17:52 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 9 Jan 2018 02:17:52 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> <14910571-3cc8-75a5-9f80-8d633b23ac4b@oracle.com> <94ea8e2e-418c-4408-2dc2-7bea1ca099d4@gmail.com> Message-ID: <2845d01f-742a-bcfe-330c-5fbd586eaf88@oracle.com> Hi, On 2018-01-09 00:04, Peter Levart wrote: > Hi Claes, > > On 01/08/18 23:41, Peter Levart wrote: >> Hi Claes, >> >> On 01/08/18 21:57, Claes Redestad wrote: >>> CCE is specified to be thrown by Set::contains/containsAll, so I >>> think we should play it safe and >>> leave it unchanged. >> >> It's probably specified so to accommodate for SortedSet(s) which >> don't use equals but compareTo / Comparator. But here you are calling >> containsAll on an abstract Set with known implementation(s) which are >> hash-based and CCE will never be thrown. If CCE is thrown from >> element's equals() then it should probably propagate out of >> AbstractImmutableSet.equals() and not silently translate to >> non-equality of sets. >> >> What do you think? > > I just noticed that AbstractSet does the same (catches CCE). That's > probably because of SortedSet subclasses too. So if you think > AbstractImmutableSet might be used for some immutable SortedSet > implementations in the future, you should probably leave it unchanged. > If elemen't equals() throws CCE it is violating the spec so it does > not matter what the outcome is (although probably propagating CCE > might help catching the bug). If you envision other > AbstractImmutableSet subclasses in the future, then perhaps it would > be good to declare an abstract hashCode() method in it. So > implementations can't forget to define it. When one sees that equals() > is already defined, he might falsely assume that hashCode() is defined > too. > > Regards, Peter you're right that for this closed world of AbstractImmutableSets we can deduce that CCE can't be thrown and ignore that possibility for now. Not sure it matters much. Adding a public abstract int hashCode(); might be a kindness. I'll add that unless there are objections. :-) If we took it a step further we could save us the trouble of throwing and swallowing NPEs in AbstractImmutableSet::equals when the other set contains null, which might be an optimization in that case: ??????? @Override ??????? public boolean equals(Object o) { ??????????? if (o == this) ??????????????? return true; ??????????? if (!(o instanceof Set)) ??????????????? return false; ??????????? Collection c = (Collection) o; ??????????? if (c.size() != size()) ??????????????? return false; ??????????? for (Object e : c) ??????????????? if (e == null || !contains(e)) ??????????????????? return false; ??????????? return true; ??????? } It might be rare enough that it's not worth bothering with, though. /Claes From martinrb at google.com Tue Jan 9 02:41:04 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 8 Jan 2018 18:41:04 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: No takers? ... let's try again. Despite uniform failure to reproduce this except under a debugger, let's apply the simple obviously correct fix, namely: /** Craft a LambdaForm customized for this particular MethodHandle */ /*non-public*/ void customize() { + final LambdaForm form = this.form; if (form.customized == null) { LambdaForm newForm = form.customize(this); updateForm(newForm); On Wed, Jan 3, 2018 at 1:51 PM, Martin Buchholz wrote: > Let's try again without mail bounces ... > > > On Wed, Jan 3, 2018 at 1:40 PM, Martin Buchholz > wrote: > >> Here at Google we tried to fix JDK-8145371 because it looked like it was >> causing rare problems in production. But after looking at the code, I'm >> not sure... Anyways, >> >> http://cr.openjdk.java.net/~martin/webrevs/jdk/MethodHandle.form/ >> https://bugs.openjdk.java.net/browse/JDK-8145371 >> Copying to a local in customize() must be an improvement. >> The UNSAFE publication in updateForm looks fishy, as does that comment >> in MethodHandleImpl.java. Is there something the fullFence() is supposed >> to do that isn't taken care of by putObjectVolatile ? >> >> Feel free to take ownership of this bug from me. >> >> --- a/src/java.base/share/classes/java/lang/invoke/MethodHandle.java >> +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandle.java >> @@ -1660,13 +1660,13 @@ >> assert(newForm.customized == null || newForm.customized == this); >> if (form == newForm) return; >> newForm.prepare(); // as in MethodHandle. >> - UNSAFE.putObject(this, FORM_OFFSET, newForm); >> - UNSAFE.fullFence(); >> + UNSAFE.putObjectVolatile(this, FORM_OFFSET, newForm); >> } >> >> /** Craft a LambdaForm customized for this particular MethodHandle */ >> /*non-public*/ >> void customize() { >> + final LambdaForm form = this.form; >> if (form.customized == null) { >> LambdaForm newForm = form.customize(this); >> updateForm(newForm); >> diff --git a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java >> --- a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java >> +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java >> @@ -909,7 +909,7 @@ >> int c = count; >> maybeCustomizeTarget(); >> if (c <= 1) { >> - // Try to limit number of updates. MethodHandle.updateForm() doesn't guarantee LF update visibility. >> + // Try to limit number of updates. >> if (isCounting) { >> isCounting = false; >> return true; >> >> >> > From john.r.rose at oracle.com Tue Jan 9 06:49:19 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 8 Jan 2018 22:49:19 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: That looks good to me. Vladimir Ivanov should take a look. Christmas vacation is just ending in Russia, so he should be around soon. On Jan 8, 2018, at 6:41 PM, Martin Buchholz wrote: > > No takers? ... let's try again. Despite uniform failure to reproduce this except under a debugger, let's apply the simple obviously correct fix, namely: > /** Craft a LambdaForm customized for this particular MethodHandle */ > /*non-public*/ > void customize() { > + final LambdaForm form = this.form; > if (form.customized == null) { > LambdaForm newForm = form.customize(this); > updateForm(newForm); > > > > On Wed, Jan 3, 2018 at 1:51 PM, Martin Buchholz > wrote: > Let's try again without mail bounces ... > > > On Wed, Jan 3, 2018 at 1:40 PM, Martin Buchholz > wrote: > Here at Google we tried to fix JDK-8145371 because it looked like it was causing rare problems in production. But after looking at the code, I'm not sure... Anyways, > > http://cr.openjdk.java.net/~martin/webrevs/jdk/MethodHandle.form/ > https://bugs.openjdk.java.net/browse/JDK-8145371 > Copying to a local in customize() must be an improvement. > The UNSAFE publication in updateForm looks fishy, as does that comment in MethodHandleImpl.java. Is there something the fullFence() is supposed to do that isn't taken care of by putObjectVolatile ? > > Feel free to take ownership of this bug from me. > > --- a/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > @@ -1660,13 +1660,13 @@ > assert(newForm.customized == null || newForm.customized == this); > if (form == newForm) return; > newForm.prepare(); // as in MethodHandle. > - UNSAFE.putObject(this, FORM_OFFSET, newForm); > - UNSAFE.fullFence(); > + UNSAFE.putObjectVolatile(this, FORM_OFFSET, newForm); > } > > /** Craft a LambdaForm customized for this particular MethodHandle */ > /*non-public*/ > void customize() { > + final LambdaForm form = this.form; > if (form.customized == null) { > LambdaForm newForm = form.customize(this); > updateForm(newForm); > diff --git a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > --- a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > @@ -909,7 +909,7 @@ > int c = count; > maybeCustomizeTarget(); > if (c <= 1) { > - // Try to limit number of updates. MethodHandle.updateForm() doesn't guarantee LF update visibility. > + // Try to limit number of updates. > if (isCounting) { > isCounting = false; > return true; > > > From paul.sandoz at oracle.com Tue Jan 9 06:50:09 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 8 Jan 2018 22:50:09 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: Hi Martin [Back from holiday] Can you reproduce using the debugger? If so do you have a more up to date stack trace? That change looks ok. The form field is final, and in the j.l.invoke package such fields are also stable so once C2 gets at it it will likely treat it as a constant. In general the updates to the field will only be visible to interpreted code or recompiled code. I suspect the problematic field may well be that of LambdaForm.transformCache. Paul. > On 8 Jan 2018, at 18:41, Martin Buchholz wrote: > > No takers? ... let's try again. Despite uniform failure to reproduce this > except under a debugger, let's apply the simple obviously correct fix, > namely: > > /** Craft a LambdaForm customized for this particular MethodHandle */ > /*non-public*/ > void customize() { > + final LambdaForm form = this.form; > if (form.customized == null) { > LambdaForm newForm = form.customize(this); > updateForm(newForm); > > > > > On Wed, Jan 3, 2018 at 1:51 PM, Martin Buchholz wrote: > >> Let's try again without mail bounces ... >> >> >> On Wed, Jan 3, 2018 at 1:40 PM, Martin Buchholz >> wrote: >> >>> Here at Google we tried to fix JDK-8145371 because it looked like it was >>> causing rare problems in production. But after looking at the code, I'm >>> not sure... Anyways, >>> >>> http://cr.openjdk.java.net/~martin/webrevs/jdk/MethodHandle.form/ >>> https://bugs.openjdk.java.net/browse/JDK-8145371 >>> Copying to a local in customize() must be an improvement. >>> The UNSAFE publication in updateForm looks fishy, as does that comment >>> in MethodHandleImpl.java. Is there something the fullFence() is supposed >>> to do that isn't taken care of by putObjectVolatile ? >>> >>> Feel free to take ownership of this bug from me. >>> >>> --- a/src/java.base/share/classes/java/lang/invoke/MethodHandle.java >>> +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandle.java >>> @@ -1660,13 +1660,13 @@ >>> assert(newForm.customized == null || newForm.customized == this); >>> if (form == newForm) return; >>> newForm.prepare(); // as in MethodHandle. >>> - UNSAFE.putObject(this, FORM_OFFSET, newForm); >>> - UNSAFE.fullFence(); >>> + UNSAFE.putObjectVolatile(this, FORM_OFFSET, newForm); >>> } >>> >>> /** Craft a LambdaForm customized for this particular MethodHandle */ >>> /*non-public*/ >>> void customize() { >>> + final LambdaForm form = this.form; >>> if (form.customized == null) { >>> LambdaForm newForm = form.customize(this); >>> updateForm(newForm); >>> diff --git a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java >>> --- a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java >>> +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java >>> @@ -909,7 +909,7 @@ >>> int c = count; >>> maybeCustomizeTarget(); >>> if (c <= 1) { >>> - // Try to limit number of updates. MethodHandle.updateForm() doesn't guarantee LF update visibility. >>> + // Try to limit number of updates. >>> if (isCounting) { >>> isCounting = false; >>> return true; >>> >>> >>> >> From martinrb at google.com Tue Jan 9 07:24:45 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 8 Jan 2018 23:24:45 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: In the hotspot sources I find // An instance field can be constant if it's a final static field or if // it's a final non-static field of a trusted class (classes in // java.lang.invoke and sun.invoke packages and subpackages). BUT you can't trust MethodHandle.form! Yes, it's final LambdaForm form; but it's also frobbed via Unsafe, and in an unusual unsafe way: UNSAFE.putObject(this, FORM_OFFSET, newForm); UNSAFE.fullFence(); On Mon, Jan 8, 2018 at 10:50 PM, Paul Sandoz wrote: > Hi Martin > > [Back from holiday] > > Can you reproduce using the debugger? If so do you have a more up to date > stack trace? > > That change looks ok. The form field is final, and in the j.l.invoke > package such fields are also stable so once C2 gets at it it will likely > treat it as a constant. In general the updates to the field will only be > visible to interpreted code or recompiled code. > > I suspect the problematic field may well be that of > LambdaForm.transformCache. > > Paul. > > > On 8 Jan 2018, at 18:41, Martin Buchholz wrote: > > > > No takers? ... let's try again. Despite uniform failure to reproduce > this > > except under a debugger, let's apply the simple obviously correct fix, > > namely: > > > > /** Craft a LambdaForm customized for this particular MethodHandle */ > > /*non-public*/ > > void customize() { > > + final LambdaForm form = this.form; > > if (form.customized == null) { > > LambdaForm newForm = form.customize(this); > > updateForm(newForm); > > > > > > > > > > On Wed, Jan 3, 2018 at 1:51 PM, Martin Buchholz > wrote: > > > >> Let's try again without mail bounces ... > >> > >> > >> On Wed, Jan 3, 2018 at 1:40 PM, Martin Buchholz > >> wrote: > >> > >>> Here at Google we tried to fix JDK-8145371 because it looked like it > was > >>> causing rare problems in production. But after looking at the code, > I'm > >>> not sure... Anyways, > >>> > >>> http://cr.openjdk.java.net/~martin/webrevs/jdk/MethodHandle.form/ > >>> https://bugs.openjdk.java.net/browse/JDK-8145371 > >>> Copying to a local in customize() must be an improvement. > >>> The UNSAFE publication in updateForm looks fishy, as does that comment > >>> in MethodHandleImpl.java. Is there something the fullFence() is > supposed > >>> to do that isn't taken care of by putObjectVolatile ? > >>> > >>> Feel free to take ownership of this bug from me. > >>> > >>> --- a/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > >>> +++ b/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > >>> @@ -1660,13 +1660,13 @@ > >>> assert(newForm.customized == null || newForm.customized == > this); > >>> if (form == newForm) return; > >>> newForm.prepare(); // as in MethodHandle. > >>> - UNSAFE.putObject(this, FORM_OFFSET, newForm); > >>> - UNSAFE.fullFence(); > >>> + UNSAFE.putObjectVolatile(this, FORM_OFFSET, newForm); > >>> } > >>> > >>> /** Craft a LambdaForm customized for this particular MethodHandle > */ > >>> /*non-public*/ > >>> void customize() { > >>> + final LambdaForm form = this.form; > >>> if (form.customized == null) { > >>> LambdaForm newForm = form.customize(this); > >>> updateForm(newForm); > >>> diff --git a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > >>> --- a/src/java.base/share/classes/java/lang/invoke/ > MethodHandleImpl.java > >>> +++ b/src/java.base/share/classes/java/lang/invoke/ > MethodHandleImpl.java > >>> @@ -909,7 +909,7 @@ > >>> int c = count; > >>> maybeCustomizeTarget(); > >>> if (c <= 1) { > >>> - // Try to limit number of updates. > MethodHandle.updateForm() doesn't guarantee LF update visibility. > >>> + // Try to limit number of updates. > >>> if (isCounting) { > >>> isCounting = false; > >>> return true; > >>> > >>> > >>> > >> > > From vladimir.x.ivanov at oracle.com Tue Jan 9 12:12:57 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Tue, 9 Jan 2018 15:12:57 +0300 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: Martin, The fix (MH.customize() part) looks good. Thanks for taking care of it! If there's a concurrent customization taking place, repeated read of MH.form can observe customized LF and repeat customization thus leading to a customized LF pointing to another customized LF (by LF.transformCache). Then LambdaFormEditor expects uncustomized LF (in factory method LFE.lambdaFormEditor(LF)) and calls LF.uncustomize(), but the latter doesn't expect nested customized LFs and just returns the immediate value from LF.transformCache which is a customized LF. And that's why probing LF cache fails later with a CCE. So, caching MH.form value and redoing LF customization (in the worst case) is safe and fixes the bug. It would be nice to have a regression test (or an assert), but I'm fine with it as is. > In the hotspot sources I find > ? ? ? // An instance field can be constant if it's a final static field > or if > ? ? ? // it's a final non-static field of a trusted class (classes in > ? ? ? // java.lang.invoke and sun.invoke packages and subpackages). > BUT > you can't trust MethodHandle.form! > Yes, it's > ? ? ? ? final LambdaForm form; > but it's also frobbed via Unsafe, and in an unusual unsafe way: > ? ? ? ? UNSAFE.putObject(this, FORM_OFFSET, newForm); > ? ? ? ? UNSAFE.fullFence(); Constant folding of MH.form is crucial for performance: without it no inlining through MethodHandle chains would happen. It is expected that MH.form updates aren't guaranteed to be visible (and it's not a theoretical possibility, but a pretty common scenario in JIT-compiled code (both by C1 & C2)). So, observing old MH.form values should be benign. Here's the relevant part of MH.updateForm() contract: /** * Replace the old lambda form of this method handle with a new one. * The new one must be functionally equivalent to the old one. ... void updateForm(LambdaForm newForm) { It is used to improve performance (remove class init check, customize LF for a particular MH instance) and not to arbitrarily change behavior of existing MH instance. Actual code on writer side (in MH.updateForm) doesn't matter much (it can be a volatile or even a plain field write) while readers treat the field as a plain final. There's an option to invalidate all compiled code where old MH.form value is used (akin to how CallSite.target is optimized), but it doesn't worth it most of the time: invalidating C1-compiled code isn't needed in tiered mode (it'll be recompiled by C2 anyway), recompilation is expensive (waste of resources), and possible performance improvements are limited. So, having a full fence there (to improve how prompt update is visible to readers w/o affecting them) is as good as other options IMO and I'd leave it as is for now. Best regards, Vladimir Ivanov > On Mon, Jan 8, 2018 at 10:50 PM, Paul Sandoz > wrote: > > Hi Martin > > [Back from holiday] > > Can you reproduce using the debugger? If so do you have a more up to > date stack trace? > > That change looks ok. The form field is final, and in the j.l.invoke > package such fields are also stable so once C2 gets at it it will > likely treat it as a constant. In general the updates to the field > will only be visible to interpreted code or recompiled code. > > I suspect the problematic field may well be that of > LambdaForm.transformCache. > > Paul. > > > On 8 Jan 2018, at 18:41, Martin Buchholz > wrote: > > > > No takers? ... let's try again.? Despite uniform failure to > reproduce this > > except under a debugger, let's apply the simple obviously correct > fix, > > namely: > > > >? ? ?/** Craft a LambdaForm customized for this particular > MethodHandle */ > >? ? ?/*non-public*/ > >? ? ?void customize() { > > +? ? ? ? final LambdaForm form = this.form; > >? ? ? ? ?if (form.customized == null) { > >? ? ? ? ? ? ?LambdaForm newForm = form.customize(this); > >? ? ? ? ? ? ?updateForm(newForm); > > > > > > > > > > On Wed, Jan 3, 2018 at 1:51 PM, Martin Buchholz > > wrote: > > > >> Let's try again without mail bounces ... > >> > >> > >> On Wed, Jan 3, 2018 at 1:40 PM, Martin Buchholz > > > >> wrote: > >> > >>> Here at Google we tried to fix JDK-8145371 because it looked > like it was > >>> causing rare problems in production.? But after looking at the > code, I'm > >>> not sure...? Anyways, > >>> > >>> > http://cr.openjdk.java.net/~martin/webrevs/jdk/MethodHandle.form/ > > >>> https://bugs.openjdk.java.net/browse/JDK-8145371 > > >>> Copying to a local in customize() must be an improvement. > >>> The UNSAFE publication in updateForm looks fishy, as does that > comment > >>> in MethodHandleImpl.java.? Is there something the fullFence() > is supposed > >>> to do that isn't taken care of by putObjectVolatile ? > >>> > >>> Feel free to take ownership of this bug from me. > >>> > >>> --- > a/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > >>> +++ > b/src/java.base/share/classes/java/lang/invoke/MethodHandle.java > >>> @@ -1660,13 +1660,13 @@ > >>>? ? ? ? ?assert(newForm.customized == null || newForm.customized > == this); > >>>? ? ? ? ?if (form == newForm)? return; > >>>? ? ? ? ?newForm.prepare();? // as in MethodHandle. > >>> -? ? ? ? UNSAFE.putObject(this, FORM_OFFSET, newForm); > >>> -? ? ? ? UNSAFE.fullFence(); > >>> +? ? ? ? UNSAFE.putObjectVolatile(this, FORM_OFFSET, newForm); > >>>? ? ?} > >>> > >>>? ? ?/** Craft a LambdaForm customized for this particular > MethodHandle */ > >>>? ? ?/*non-public*/ > >>>? ? ?void customize() { > >>> +? ? ? ? final LambdaForm form = this.form; > >>>? ? ? ? ?if (form.customized == null) { > >>>? ? ? ? ? ? ?LambdaForm newForm = form.customize(this); > >>>? ? ? ? ? ? ?updateForm(newForm); > >>> diff --git > a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > >>> --- > a/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > >>> +++ > b/src/java.base/share/classes/java/lang/invoke/MethodHandleImpl.java > >>> @@ -909,7 +909,7 @@ > >>>? ? ? ? ? ? ?int c = count; > >>>? ? ? ? ? ? ?maybeCustomizeTarget(); > >>>? ? ? ? ? ? ?if (c <= 1) { > >>> -? ? ? ? ? ? ? ? // Try to limit number of updates. > MethodHandle.updateForm() doesn't guarantee LF update visibility. > >>> +? ? ? ? ? ? ? ? // Try to limit number of updates. > >>>? ? ? ? ? ? ? ? ?if (isCounting) { > >>>? ? ? ? ? ? ? ? ? ? ?isCounting = false; > >>>? ? ? ? ? ? ? ? ? ? ?return true; > >>> > >>> > >>> > >> > > From peter.levart at gmail.com Tue Jan 9 12:27:19 2018 From: peter.levart at gmail.com (Peter Levart) Date: Tue, 9 Jan 2018 13:27:19 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <2845d01f-742a-bcfe-330c-5fbd586eaf88@oracle.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> <14910571-3cc8-75a5-9f80-8d633b23ac4b@oracle.com> <94ea8e2e-418c-4408-2dc2-7bea1ca099d4@gmail.com> <2845d01f-742a-bcfe-330c-5fbd586eaf88@oracle.com> Message-ID: <1bcb7c02-5ce0-be4f-e28c-842f229004bc@gmail.com> Hi Claes, On 01/09/2018 02:17 AM, Claes Redestad wrote: > If we took it a step further we could save us the trouble of throwing > and swallowing NPEs in > AbstractImmutableSet::equals when the other set contains null, which > might be an optimization > in that case: > > ??????? @Override > ??????? public boolean equals(Object o) { > ??????????? if (o == this) > ??????????????? return true; > > ??????????? if (!(o instanceof Set)) > ??????????????? return false; > > ??????????? Collection c = (Collection) o; > ??????????? if (c.size() != size()) > ??????????????? return false; > > ??????????? for (Object e : c) > ??????????????? if (e == null || !contains(e)) > ??????????????????? return false; > ??????????? return true; > ??????? } > > It might be rare enough that it's not worth bothering with, though. That's nicer in my opinion. What to do with ClassCastException from contains(e)? Perhaps just document that any future implementation that might throw it from contains() should also modify the equals() method and catch it there. And if you already do explicit iteration over elements, you could as well count them and do the size check with counted size at the end. Imagine the following scenario: ConcurrentHashMap m = new ChocurrentHashMap(); m.put(1, 1); m.put(2, 2); m.put(3, 3); Set s = Set.of("a", "b", "c"); Thread 1: s.equals(m.keySet()) Thread 2: m.clear(); It is surprising when Thread 1 encounters 'true' as the result of the equals() above which can happen when code checks for sizes separately. Iterating CHM.keySet() does not provide a snapshot of the content either, but counting iterated and compared elements nevertheless exhibits less surprising behavior. I don't know if it matters though. Regards, Peter From claes.redestad at oracle.com Tue Jan 9 21:15:59 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 9 Jan 2018 22:15:59 +0100 Subject: RFR: 8134459: java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java timed out Message-ID: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> Hi, the java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java has started timing out locallty on my machine, and analyzing why it seems it simply has added enough test cases recently to hit the default 120s timeout. Quickly analyzing what is taking so much time I ran into an inefficiency in TestNGs assertEquals(Iterator, Iterator) implementation, where at least one error message string is created unconditionally in an inner loop, leading to quite a bit of allocation pressure in a test like WhileOpTest. On my machine, avoiding this extra work speeds up the test from taking around ~130s to ~100s. A good improvement, but still somewhat close to the default 120s timeout. So as a fix I propose: - work around this by providing simplified assert methods in the LambdaTestHelper that avoid the excessively allocating methods in TestNG - increasing the timeout of WhileOpTest to 240s http://cr.openjdk.java.net/~redestad/8134459/open.00/ Regards /Claes From Roger.Riggs at Oracle.com Tue Jan 9 21:30:40 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 9 Jan 2018 16:30:40 -0500 Subject: RFR: 8134459: java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java timed out In-Reply-To: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> References: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> Message-ID: <6b5a1bff-d883-6306-2405-9459974dbfbb@Oracle.com> Hi Claes, Looks ok, a couple of nits. LambdaTestHelpers: line 57: Please break the long string to make future side-by-side reviews easier. line 322:? Perhaps "Iterator contents differ" instead of "iterators differ" so its clear the contents of the iterators are different, not the instances. Having the indices might be useful for debugging, but they aren't ever supposed to be different. Regards, Roger On 1/9/2018 4:15 PM, Claes Redestad wrote: > Hi, > > the > java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java > has > started timing out locallty on my machine, and analyzing why it seems > it simply has > added enough test cases recently to hit the default 120s timeout. > > Quickly analyzing what is taking so much time I ran into an > inefficiency in TestNGs > assertEquals(Iterator, Iterator) implementation, where at least > one error message > string is created unconditionally in an inner loop, leading to quite a > bit of allocation > pressure in a test like WhileOpTest. > > On my machine, avoiding this extra work speeds up the test from taking > around ~130s > to ~100s. A good improvement, but still somewhat close to the default > 120s timeout. > > So as a fix I propose: > > - work around this by providing simplified assert methods in the > LambdaTestHelper that > avoid the excessively allocating methods in TestNG > - increasing the timeout of WhileOpTest to 240s > > http://cr.openjdk.java.net/~redestad/8134459/open.00/ > > Regards > > /Claes From martinrb at google.com Tue Jan 9 22:20:08 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jan 2018 14:20:08 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: On Tue, Jan 9, 2018 at 4:12 AM, Vladimir Ivanov < vladimir.x.ivanov at oracle.com> wrote: > > In the hotspot sources I find >> // An instance field can be constant if it's a final static field >> or if >> // it's a final non-static field of a trusted class (classes in >> // java.lang.invoke and sun.invoke packages and subpackages). >> BUT >> you can't trust MethodHandle.form! >> Yes, it's >> final LambdaForm form; >> but it's also frobbed via Unsafe, and in an unusual unsafe way: >> UNSAFE.putObject(this, FORM_OFFSET, newForm); >> UNSAFE.fullFence(); >> > > Constant folding of MH.form is crucial for performance: without it no > inlining through MethodHandle chains would happen. > > It is expected that MH.form updates aren't guaranteed to be visible (and > it's not a theoretical possibility, but a pretty common scenario in > JIT-compiled code (both by C1 & C2)). So, observing old MH.form values > should be benign. > > Here's the relevant part of MH.updateForm() contract: > /** > * Replace the old lambda form of this method handle with a new one. > * The new one must be functionally equivalent to the old one. > ... > void updateForm(LambdaForm newForm) { > > It is used to improve performance (remove class init check, customize LF > for a particular MH instance) and not to arbitrarily change behavior of > existing MH instance. > > Actual code on writer side (in MH.updateForm) doesn't matter much (it can > be a volatile or even a plain field write) while readers treat the field as > a plain final. > > There's an option to invalidate all compiled code where old MH.form value > is used (akin to how CallSite.target is optimized), but it doesn't worth it > most of the time: invalidating C1-compiled code isn't needed in tiered mode > (it'll be recompiled by C2 anyway), recompilation is expensive (waste of > resources), and possible performance improvements are limited. > > So, having a full fence there (to improve how prompt update is visible to > readers w/o affecting them) is as good as other options IMO and I'd leave > it as is for now. > Thanks for the explanation. Simple fix was submitted. I'm still hoping we can later reduce the magicalness of j.l.invoke. The memory model is already too hard to reason about, but here the VM can assume that the final fields will never be mutated - yet they are! With the special j.l.invoke guarantees, there should be no need for final fields to additionally have @Stable annotations, yet many do! LambdaForm is a mutable class, so publishing it via a plain Unsafe write is a (tiny, hard to detect) data race. I would feel much more comfortable replacing the Unsafe put with a putVolatile and dropping the fence. Whenever the form field is read, perhaps it should be explicitly read via a volatile or acquire read for safety. From paul.sandoz at oracle.com Tue Jan 9 22:23:10 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 9 Jan 2018 14:23:10 -0800 Subject: RFR: 8134459: java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java timed out In-Reply-To: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> References: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> Message-ID: This might also improve other tests using the same assert. LambdaTestHelpers -- s/if(/if ( + int i = -1; + while (actual.hasNext() && expected.hasNext()) { + i++; + Object e = expected.next(); + Object a = actual.next(); + assertEquals(a, e, "Iterators differ"); + + } ?i' is redundant. I presume might have wanted to use it to print out the index on element assertion failure, but it would of course induce garbage. Paul. > On 9 Jan 2018, at 13:15, Claes Redestad wrote: > > Hi, > > the java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java has > started timing out locallty on my machine, and analyzing why it seems it simply has > added enough test cases recently to hit the default 120s timeout. > > Quickly analyzing what is taking so much time I ran into an inefficiency in TestNGs > assertEquals(Iterator, Iterator) implementation, where at least one error message > string is created unconditionally in an inner loop, leading to quite a bit of allocation > pressure in a test like WhileOpTest. > > On my machine, avoiding this extra work speeds up the test from taking around ~130s > to ~100s. A good improvement, but still somewhat close to the default 120s timeout. > > So as a fix I propose: > > - work around this by providing simplified assert methods in the LambdaTestHelper that > avoid the excessively allocating methods in TestNG > - increasing the timeout of WhileOpTest to 240s > > http://cr.openjdk.java.net/~redestad/8134459/open.00/ > > Regards > > /Claes From paul.sandoz at oracle.com Tue Jan 9 22:42:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 9 Jan 2018 14:42:17 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: Message-ID: <22408C16-27D4-482A-8A31-6BB14EB15EC0@oracle.com> > On 9 Jan 2018, at 14:20, Martin Buchholz wrote: > > The memory model is already too hard to reason about, but here the VM can assume that the final fields will never be mutated - yet they are! Because of reflection and Field/AccessibleObject.setAccessible the VM is conservative and does not in general assume final fields are really final. Because of that we miss out on some juicy optimisations. We have made some inroads into limiting the use of setAccessible. If we can deprecate and remove it we can make progress on generally applying "final means final? to all Java code (which also means tackling the case of deserialisation). > With the special j.l.invoke guarantees, there should be no need for final fields to additionally have @Stable annotations, yet many do! In j.l.invoke I only found one redundant @Stable annotated field on MethodType: private final @Stable Class rtype; All the others are on non-final fields or array fields where the stable semantics are propagated to array elements. > LambdaForm is a mutable class, so publishing it via a plain Unsafe write is a (tiny, hard to detect) data race. I would feel much more comfortable replacing the Unsafe put with a putVolatile and dropping the fence. Whenever the form field is read, perhaps it should be explicitly read via a volatile or acquire read for safety. That would incur a cost. j.l.invoke contains code that has carefully arranged interactions with the runtime compilers, this is one of those cases. Paul. From claes.redestad at oracle.com Tue Jan 9 22:55:37 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 9 Jan 2018 23:55:37 +0100 Subject: RFR: 8134459: java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java timed out In-Reply-To: References: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> Message-ID: <67da3d03-7bcc-0543-4c06-dcd5bf5bc1d1@oracle.com> Hi Paul, On 2018-01-09 23:23, Paul Sandoz wrote: > This might also improve other tests using the same assert. Not unlikely. > > LambdaTestHelpers > -- > > s/if(/if ( > > + int i = -1; > + while (actual.hasNext() && expected.hasNext()) { > + i++; > + Object e = expected.next(); > + Object a = actual.next(); > + assertEquals(a, e, "Iterators differ"); > + > + } > > > ?i' is redundant. I presume might have wanted to use it to print out the index on element assertion failure, but it would of course induce garbage. Fixed in place /Claes From joe.darcy at oracle.com Tue Jan 9 22:58:49 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Tue, 09 Jan 2018 14:58:49 -0800 Subject: RFR: 8134459: java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java timed out In-Reply-To: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> References: <310482cb-3872-464e-ce01-dd91ba904c93@oracle.com> Message-ID: <5A554929.2010700@oracle.com> Hello, Making tests run faster is all good of course. However, I'm leery of bumping up the timeouts; I'd prefer the tests be split into multiple parts if there is a chance it will take many minutes to run. (Having individual regression tests run in at most a minute or two helps avoid Amdahl's law problems when running the regression tests with concurrency.) Thanks, -Joe On 1/9/2018 1:15 PM, Claes Redestad wrote: > Hi, > > the > java/util/stream/test/org/openjdk/tests/java/util/stream/WhileOpTest.java > has > started timing out locallty on my machine, and analyzing why it seems > it simply has > added enough test cases recently to hit the default 120s timeout. > > Quickly analyzing what is taking so much time I ran into an > inefficiency in TestNGs > assertEquals(Iterator, Iterator) implementation, where at least > one error message > string is created unconditionally in an inner loop, leading to quite a > bit of allocation > pressure in a test like WhileOpTest. > > On my machine, avoiding this extra work speeds up the test from taking > around ~130s > to ~100s. A good improvement, but still somewhat close to the default > 120s timeout. > > So as a fix I propose: > > - work around this by providing simplified assert methods in the > LambdaTestHelper that > avoid the excessively allocating methods in TestNG > - increasing the timeout of WhileOpTest to 240s > > http://cr.openjdk.java.net/~redestad/8134459/open.00/ > > Regards > > /Claes From martinrb at google.com Tue Jan 9 23:11:24 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jan 2018 15:11:24 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: <22408C16-27D4-482A-8A31-6BB14EB15EC0@oracle.com> References: <22408C16-27D4-482A-8A31-6BB14EB15EC0@oracle.com> Message-ID: On Tue, Jan 9, 2018 at 2:42 PM, Paul Sandoz wrote: > > > > On 9 Jan 2018, at 14:20, Martin Buchholz wrote: > > > > The memory model is already too hard to reason about, but here the VM > can assume that the final fields will never be mutated - yet they are! > > Because of reflection and Field/AccessibleObject.setAccessible the VM is > conservative and does not in general assume final fields are really final. > Because of that we miss out on some juicy optimisations. We have made some > inroads into limiting the use of setAccessible. If we can deprecate and > remove it we can make progress on generally applying "final means final? to > all Java code (which also means tackling the case of deserialisation). > > Will there be a principled way to handle "final means final" for MethodHandle.form? Maybe a new annotation that means "mutable BUT compile as if final" ? > > > With the special j.l.invoke guarantees, there should be no need for > final fields to additionally have @Stable annotations, yet many do! > > In j.l.invoke I only found one redundant @Stable annotated field on > MethodType: > > private final @Stable Class rtype; > > All the others are on non-final fields or array fields where the stable > semantics are propagated to array elements. > Yes, I had forgotten that subtlety with @Stable. > > > LambdaForm is a mutable class, so publishing it via a plain Unsafe write > is a (tiny, hard to detect) data race. I would feel much more comfortable > replacing the Unsafe put with a putVolatile and dropping the fence. > Whenever the form field is read, perhaps it should be explicitly read via a > volatile or acquire read for safety. > > That would incur a cost. j.l.invoke contains code that has carefully > arranged interactions with the runtime compilers, this is one of those > cases. > (we already have a full fence on writes!) As it stands, MethodHandle.form is published via a data race which seems dangerous to me, but may be safe in practice because no one is running Java on an early Alpha machine. From paul.sandoz at oracle.com Wed Jan 10 00:50:28 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 9 Jan 2018 16:50:28 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: <22408C16-27D4-482A-8A31-6BB14EB15EC0@oracle.com> Message-ID: <80B537BB-734A-40DD-AFC0-8533299C894C@oracle.com> > On 9 Jan 2018, at 15:11, Martin Buchholz wrote: > > > > On Tue, Jan 9, 2018 at 2:42 PM, Paul Sandoz > wrote: > > > > On 9 Jan 2018, at 14:20, Martin Buchholz > wrote: > > > > The memory model is already too hard to reason about, but here the VM can assume that the final fields will never be mutated - yet they are! > > Because of reflection and Field/AccessibleObject.setAccessible the VM is conservative and does not in general assume final fields are really final. Because of that we miss out on some juicy optimisations. We have made some inroads into limiting the use of setAccessible. If we can deprecate and remove it we can make progress on generally applying "final means final? to all Java code (which also means tackling the case of deserialisation). > > > Will there be a principled way to handle "final means final" for MethodHandle.form? Maybe a new annotation that means "mutable BUT compile as if final" ? > Yes, i could imagine in the future removing the final modifier and adding the @Stable annotation or some variant of for the more ?relaxed" form of finality that is currently supported. In fact it might be clearer if we do that now (make non-final and add @Stable). > > > With the special j.l.invoke guarantees, there should be no need for final fields to additionally have @Stable annotations, yet many do! > > In j.l.invoke I only found one redundant @Stable annotated field on MethodType: > > private final @Stable Class rtype; > > All the others are on non-final fields or array fields where the stable semantics are propagated to array elements. > > Yes, I had forgotten that subtlety with @Stable. > > > > LambdaForm is a mutable class, so publishing it via a plain Unsafe write is a (tiny, hard to detect) data race. I would feel much more comfortable replacing the Unsafe put with a putVolatile and dropping the fence. Whenever the form field is read, perhaps it should be explicitly read via a volatile or acquire read for safety. > > That would incur a cost. j.l.invoke contains code that has carefully arranged interactions with the runtime compilers, this is one of those cases. > > (we already have a full fence on writes!) > True, but the writes should be rare compared to the reads and as Vladimir points out the customisation should not change the functionality. Paul. > As it stands, MethodHandle.form is published via a data race which seems dangerous to me, but may be safe in practice because no one is running Java on an early Alpha machine. From ralph.goers at dslextreme.com Wed Jan 10 01:11:03 2018 From: ralph.goers at dslextreme.com (Ralph Goers) Date: Tue, 9 Jan 2018 18:11:03 -0700 Subject: Android and Log4j In-Reply-To: References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Message-ID: Martin, I don?t think I got a reply to this question. I created https://issuetracker.google.com/issues/70118537 for this but I have not gotten any response. Is there a better mailing list where I should follow up? Ralph > On Dec 6, 2017, at 10:25 PM, Ralph Goers wrote: > > Martin, > > Do they also ignore the class files in META-INF/versions? > > Ralph > >> On Dec 6, 2017, at 2:35 PM, Martin Buchholz wrote: >> >> Very latest Android Studio comes with two compilers, dx and d8. >> https://developer.android.com/studio/preview/features/index.html >> I hear that both compilers know to skip module-info files. >> >> On Mon, Dec 4, 2017 at 12:03 PM, Martin Buchholz > wrote: >> >> >> On Sun, Dec 3, 2017 at 11:02 AM, Remi Forax > wrote: >> Ask Google to fix dx, >> dx should ignore the module-info.class and everything inside META-INF/versions (at least it's a first simple patch). >> >> Hi, I'm "Google", sort of. Friends tell me that dx is getting fixed. >> >> > > From martinrb at google.com Wed Jan 10 03:24:26 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jan 2018 19:24:26 -0800 Subject: Android and Log4j In-Reply-To: References: <42D0ACD0-C54E-4D93-A26E-7E36D5499C9E@dslextreme.com> <1337090690.3302668.1512327739247.JavaMail.zimbra@u-pem.fr> Message-ID: On Tue, Jan 9, 2018 at 5:11 PM, Ralph Goers wrote: > > I don?t think I got a reply to this question. I created > https://issuetracker.google.com/issues/70118537 for this but I have not > gotten any response. Is there a better mailing list where I should follow > up? > Sorry Ralph, I don't know. From martinrb at google.com Wed Jan 10 07:28:26 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 9 Jan 2018 23:28:26 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: <80B537BB-734A-40DD-AFC0-8533299C894C@oracle.com> References: <22408C16-27D4-482A-8A31-6BB14EB15EC0@oracle.com> <80B537BB-734A-40DD-AFC0-8533299C894C@oracle.com> Message-ID: On Tue, Jan 9, 2018 at 4:50 PM, Paul Sandoz wrote: > > > On 9 Jan 2018, at 15:11, Martin Buchholz wrote: > > > > On Tue, Jan 9, 2018 at 2:42 PM, Paul Sandoz > wrote: > >> >> >> > On 9 Jan 2018, at 14:20, Martin Buchholz wrote: >> > >> > The memory model is already too hard to reason about, but here the VM >> can assume that the final fields will never be mutated - yet they are! >> >> Because of reflection and Field/AccessibleObject.setAccessible the VM is >> conservative and does not in general assume final fields are really final. >> Because of that we miss out on some juicy optimisations. We have made some >> inroads into limiting the use of setAccessible. If we can deprecate and >> remove it we can make progress on generally applying "final means final? to >> all Java code (which also means tackling the case of deserialisation). >> >> > Will there be a principled way to handle "final means final" for > MethodHandle.form? Maybe a new annotation that means "mutable BUT compile > as if final" ? > > > > Yes, i could imagine in the future removing the final modifier and adding > the @Stable annotation or some variant of for the more ?relaxed" form of > finality that is currently supported. In fact it might be clearer if we do > that now (make non-final and add @Stable). > > Well, using @Stable here would be very much undefined * It is (currently) undefined what happens if a field annotated as stable * is given a third value (by explicitly updating a stable field, a component of * a stable array, or a final stable field via reflection or other means). * Since the HotSpot VM promotes a non-null component value to constant, it may * be that the Java memory model would appear to be broken, if such a constant * (the second value of the field) is used as the value of the field even after * the field value has changed (to a third value). >> > LambdaForm is a mutable class, so publishing it via a plain Unsafe >> write is a (tiny, hard to detect) data race. I would feel much more >> comfortable replacing the Unsafe put with a putVolatile and dropping the >> fence. Whenever the form field is read, perhaps it should be explicitly >> read via a volatile or acquire read for safety. >> >> That would incur a cost. j.l.invoke contains code that has carefully >> arranged interactions with the runtime compilers, this is one of those >> cases. >> > > (we already have a full fence on writes!) > > > True, but the writes should be rare compared to the reads and as Vladimir > points out the customisation should not change the functionality. > We're changing a "truly" final field holding a mutable object via a data race. Subsequently we may randomly mix the cached old object fields and the new object, and perhaps even uninitialized fields of the latter. In theory. > Paul. > > As it stands, MethodHandle.form is published via a data race which seems > dangerous to me, but may be safe in practice because no one is running Java > on an early Alpha machine. > > From goetz.lindenmaier at sap.com Wed Jan 10 08:23:12 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 10 Jan 2018 08:23:12 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. Message-ID: Hi, this change adapts a row of jtreg tests to AIX. Shell tests often switch over all OSes, here the case for AIX was missing. PKCSTest needs to know where the libs are on s390. Please review. http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ Best regards, Goetz. From naoto.sato at oracle.com Wed Jan 10 19:04:30 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Wed, 10 Jan 2018 11:04:30 -0800 Subject: [11] RFR 8194148: bcp47u/SymbolsTests.java and FormatTests.java fail on th_TH locale Message-ID: <2ac0bd96-3fb4-67a4-aa8d-ad38f13b1c20@oracle.com> Hello, Please review a simple test fix to the following issue: https://bugs.openjdk.java.net/browse/JDK-8194148 The proposed changeset is located at: http://cr.openjdk.java.net/~naoto/8194148/webrev.00/ The gist of the issue is that on some test machine, Buddhist calendar is the default for th-TH locale. The test case should explicitly specify Gregorian for its purpose. Also, fixed an error in SymbolsTests where the test class was erroneously set to FormatTests. Naoto From paul.sandoz at oracle.com Wed Jan 10 23:58:00 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 10 Jan 2018 15:58:00 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: References: <22408C16-27D4-482A-8A31-6BB14EB15EC0@oracle.com> <80B537BB-734A-40DD-AFC0-8533299C894C@oracle.com> Message-ID: <68664FBC-AD43-45E1-BFD1-EA3D8576C225@oracle.com> > On 9 Jan 2018, at 23:28, Martin Buchholz wrote: > > > > On Tue, Jan 9, 2018 at 4:50 PM, Paul Sandoz > wrote: > > >> On 9 Jan 2018, at 15:11, Martin Buchholz > wrote: >> >> >> >> On Tue, Jan 9, 2018 at 2:42 PM, Paul Sandoz > wrote: >> >> >> > On 9 Jan 2018, at 14:20, Martin Buchholz > wrote: >> > >> > The memory model is already too hard to reason about, but here the VM can assume that the final fields will never be mutated - yet they are! >> >> Because of reflection and Field/AccessibleObject.setAccessible the VM is conservative and does not in general assume final fields are really final. Because of that we miss out on some juicy optimisations. We have made some inroads into limiting the use of setAccessible. If we can deprecate and remove it we can make progress on generally applying "final means final? to all Java code (which also means tackling the case of deserialisation). >> >> >> Will there be a principled way to handle "final means final" for MethodHandle.form? Maybe a new annotation that means "mutable BUT compile as if final" ? >> > > Yes, i could imagine in the future removing the final modifier and adding the @Stable annotation or some variant of for the more ?relaxed" form of finality that is currently supported. In fact it might be clearer if we do that now (make non-final and add @Stable). > > > Well, using @Stable here would be very much undefined > > * It is (currently) undefined what happens if a field annotated as stable > * is given a third value (by explicitly updating a stable field, a component of > * a stable array, or a final stable field via reflection or other means). > * Since the HotSpot VM promotes a non-null component value to constant, it may > * be that the Java memory model would appear to be broken, if such a constant > * (the second value of the field) is used as the value of the field even after > * the field value has changed (to a third value). Yes :-) In this case the form field is updated with the knowledge that visibility but not functionality may be affected. Note that final fields of classes in java.lang.invoke are in effect implicitly annotated with @Stable (that?s why i referred to the notion of ?relaxed? finality). > >> >> > LambdaForm is a mutable class, so publishing it via a plain Unsafe write is a (tiny, hard to detect) data race. I would feel much more comfortable replacing the Unsafe put with a putVolatile and dropping the fence. Whenever the form field is read, perhaps it should be explicitly read via a volatile or acquire read for safety. >> >> That would incur a cost. j.l.invoke contains code that has carefully arranged interactions with the runtime compilers, this is one of those cases. >> >> (we already have a full fence on writes!) >> > > True, but the writes should be rare compared to the reads and as Vladimir points out the customisation should not change the functionality. > > We're changing a "truly" final field holding a mutable object via a data race. Subsequently we may randomly mix the cached old object fields and the new object, and perhaps even uninitialized fields of the latter. In theory. > Given how the compiler consumes values of these fields i am unsure how that can happen. The code is skating close to the edge and is definitely making you uncomfortable :-) so I proposed making the field non-final but annotated with @Stable, the same optimisations should still apply. Paul. From martinrb at google.com Thu Jan 11 03:41:26 2018 From: martinrb at google.com (Martin Buchholz) Date: Wed, 10 Jan 2018 19:41:26 -0800 Subject: JDK-8145371 ClassCastException thrown in LambdaFormEditor.getInCache In-Reply-To: <68664FBC-AD43-45E1-BFD1-EA3D8576C225@oracle.com> References: <22408C16-27D4-482A-8A31-6BB14EB15EC0@oracle.com> <80B537BB-734A-40DD-AFC0-8533299C894C@oracle.com> <68664FBC-AD43-45E1-BFD1-EA3D8576C225@oracle.com> Message-ID: On Wed, Jan 10, 2018 at 3:58 PM, Paul Sandoz wrote: > > The code is skating close to the edge and is definitely making you > uncomfortable :-) so I proposed making the field non-final but annotated > with @Stable, the same optimisations should still apply. > Sorry, @Stable doesn't increase my comfort. It is common for unsafe publication bugs to persist for years because they don't manifest as user-visible behavior very often. Like https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html From rachna.goel at oracle.com Thu Jan 11 08:15:07 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Thu, 11 Jan 2018 13:45:07 +0530 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: <46a67401-92e4-09a2-dfdf-2c10a977a71e@oracle.com> References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> <46a67401-92e4-09a2-dfdf-2c10a977a71e@oracle.com> Message-ID: Hi Joe, I have revised this webrev to include your feedback and also updated this CSR. Kindly have a look at : CSR: https://bugs.openjdk.java.net/browse/JDK-8191414 webrev: http://cr.openjdk.java.net/~rgoel/8146656/webrev.01/ Thanks, Rachna On 20/12/17 10:44 PM, joe darcy wrote: > > Hi Rachna, > > I think the revised version with the @implSpec tag switch is > acceptable, but also think providing more text to describe this > situation would be helpful to programmers unaware of a 13 month > possibility. > > Cheers, > > -Joe > > > On 12/19/2017 2:08 AM, Rachna Goel wrote: >> >> Hi Joe, >> >> Thanks for the comments. >> >> I have updated the CSR to have @implSpec in place of @implNote. >> >> https://bugs.openjdk.java.net/browse/JDK-8191414 >> >> Regarding "An array with either 12 or 13 elements will be returned >> depending on whether or {@link Calendar.UNDECIMBER} is supported." , >> I would like to go with existing statement as this method always >> returns 13 elements where the 13th element may be empty string or may >> contain Calendar.UNDECIMBER, depending upon whether its supported by >> the Calendar instance. >> >> kindly suggest whether this looks fine! >> >> Thanks, >> Rachna >> >> >> On 19/12/17 2:55 PM, joe darcy wrote: >>> Hi Rachna, >>> >>> On 12/19/2017 1:13 AM, Rachna Goel wrote: >>>> >>>> Hello Joe, >>>> >>>> Thanks for the review. >>>> >>>> Reason I added @implNote is that it's the case for the default >>>> implementation. Not added as a part of spec, as some implementation >>>> can just return 12 element array for same methods through the >>>> "java.text.spi.DateFormatSymbolsProvider" SPI. >>>> >>>> >>> >>> That is precisely the sort of situation the @implSpec tag is >>> intended for. It allows the specification to say DateFormatSymbols >>> must behave this way while allowing subclasses to behave differently. >>> >>> Perhaps some general text can be added as normal specification, >>> something like >>> >>> "An array with either 12 or 13 elements will be returned depending >>> on whether or {@link Calendar.UNDECIMBER} is supported." >>> >>> paired with >>> >>> @implSpec This method returns 13 elements since @link >>> Calendar.UNDECIMBER} is supported. >>> >>> HTH, >>> >>> -Joe >>> >> >> -- >> Thanks, >> Rachna > -- Thanks, Rachna From Roger.Riggs at Oracle.com Thu Jan 11 15:19:16 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Jan 2018 10:19:16 -0500 Subject: RFR 8194929 : Unreferenced FileDescriptors not closed Message-ID: <9588ce1f-52c9-8387-2cd3-3b55c9cbf0db@Oracle.com> Please review a fix for a missing entry in the mapfiles.? The missing entry point caused the closing of unreferenced FileDescriptors to be skipped. The tests for FileInputStream, FileOutputStream, and RandomAccess file unreferenced FileDescriptors have been updated to verify the raw fds have been closed. This fix applies to JDK 10+. Webrev: http://cr.openjdk.java.net/~rriggs/webrev-fd-cleanup-8194929/index.html Issue: ? https://bugs.openjdk.java.net/browse/JDK-8194929 Thanks, Roger From Alan.Bateman at oracle.com Thu Jan 11 16:14:56 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Thu, 11 Jan 2018 16:14:56 +0000 Subject: RFR 8194929 : Unreferenced FileDescriptors not closed In-Reply-To: <9588ce1f-52c9-8387-2cd3-3b55c9cbf0db@Oracle.com> References: <9588ce1f-52c9-8387-2cd3-3b55c9cbf0db@Oracle.com> Message-ID: On 11/01/2018 15:19, Roger Riggs wrote: > Please review a fix for a missing entry in the mapfiles.? The missing > entry point > caused the closing of unreferenced FileDescriptors to be skipped. Looks good. > > The tests for FileInputStream, FileOutputStream, and RandomAccess file > unreferenced > FileDescriptors have been updated to verify the raw fds have been closed. closeCounter is incremented before super.close(). That is good enough for the original test as it's just checking that close is called. I'm just wondering if the test methods now need to poll FileDescriptor::valid or wait until ffd is -1 to ensure to avoid a race with the getOpenFileDescriptorCount. -Alan From Roger.Riggs at Oracle.com Thu Jan 11 16:37:04 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Jan 2018 11:37:04 -0500 Subject: RFR 8194929 : Unreferenced FileDescriptors not closed In-Reply-To: References: <9588ce1f-52c9-8387-2cd3-3b55c9cbf0db@Oracle.com> Message-ID: Hi? Alan, On 1/11/2018 11:14 AM, Alan Bateman wrote: > > > On 11/01/2018 15:19, Roger Riggs wrote: >> Please review a fix for a missing entry in the mapfiles.? The missing >> entry point >> caused the closing of unreferenced FileDescriptors to be skipped. > Looks good. > >> >> The tests for FileInputStream, FileOutputStream, and RandomAccess >> file unreferenced >> FileDescriptors have been updated to verify the raw fds have been >> closed. > closeCounter is incremented before super.close(). That is good enough > for the original test as it's just checking that close is called. I'm > just wondering if the test methods now need to poll > FileDescriptor::valid or wait until ffd is -1 to ensure to avoid a > race with the getOpenFileDescriptorCount. Holding a reference to the FileDescriptor would prevent it from being reclaimed and the test would never complete. Each test waits until the Cleanup object has been reclaimed; if it has been reclaimed then the closeCleanup0 has already been called to close the raw fd. Thanks, Roger From brian.burkhalter at oracle.com Thu Jan 11 16:59:15 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 11 Jan 2018 08:59:15 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> Message-ID: <60A17F10-D01C-4B02-825F-DEA7C31360FF@oracle.com> While there appears now to be agreement on using naming option C below, formal Reviewer approval appears still to be lacking. It would be good if there were at least two approvals forthcoming, if appropriate. http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ Thanks, Brian On Jan 4, 2018, at 3:42 PM, Brian Burkhalter wrote: > Re-reading the discussion [1] of last year regarding [2], it seems that there was convergence on all points raised except the names of the new methods. The alternatives are: > > A) InputStream.nullStream() and OutputStream.nullStream() as in the most recent version, webrev.03, > B) InputStream.nullInput() and OutputStream.nullOutput() as shown above, and > C) InputStream.nullInputStream() and OutputStream.nullOutputStream(). > > Does anyone have any more comments on this point? I would be inclined to choose option C. > > Thanks, > > Brian > > [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050367.html > [2] https://bugs.openjdk.java.net/browse/JDK-4358774 From Roger.Riggs at Oracle.com Thu Jan 11 19:16:44 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Jan 2018 14:16:44 -0500 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <60A17F10-D01C-4B02-825F-DEA7C31360FF@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> <60A17F10-D01C-4B02-825F-DEA7C31360FF@oracle.com> Message-ID: <79e79fe7-eb4e-fdfd-1f0c-36ffdbf95eac@Oracle.com> +1 looks fine On 1/11/2018 11:59 AM, Brian Burkhalter wrote: > While there appears now to be agreement on using naming option C below, formal Reviewer approval appears still to be lacking. It would be good if there were at least two approvals forthcoming, if appropriate. > > http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ > > Thanks, > > Brian > > On Jan 4, 2018, at 3:42 PM, Brian Burkhalter wrote: > >> Re-reading the discussion [1] of last year regarding [2], it seems that there was convergence on all points raised except the names of the new methods. The alternatives are: >> >> A) InputStream.nullStream() and OutputStream.nullStream() as in the most recent version, webrev.03, >> B) InputStream.nullInput() and OutputStream.nullOutput() as shown above, and >> C) InputStream.nullInputStream() and OutputStream.nullOutputStream(). >> >> Does anyone have any more comments on this point? I would be inclined to choose option C. >> >> Thanks, >> >> Brian >> >> [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050367.html >> [2] https://bugs.openjdk.java.net/browse/JDK-4358774 From Roger.Riggs at Oracle.com Thu Jan 11 20:30:08 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Thu, 11 Jan 2018 15:30:08 -0500 Subject: [11] RFR 8194148: bcp47u/SymbolsTests.java and FormatTests.java fail on th_TH locale In-Reply-To: <2ac0bd96-3fb4-67a4-aa8d-ad38f13b1c20@oracle.com> References: <2ac0bd96-3fb4-67a4-aa8d-ad38f13b1c20@oracle.com> Message-ID: <30ccdfa3-976b-3644-b199-5ce21321196c@Oracle.com> Hi Naoto, Looks fine. Thanks On 1/10/2018 2:04 PM, Naoto Sato wrote: > Hello, > > Please review a simple test fix to the following issue: > > https://bugs.openjdk.java.net/browse/JDK-8194148 > > The proposed changeset is located at: > > http://cr.openjdk.java.net/~naoto/8194148/webrev.00/ > > The gist of the issue is that on some test machine, Buddhist calendar > is the default for th-TH locale. The test case should explicitly > specify Gregorian for its purpose. Also, fixed an error in > SymbolsTests where the test class was erroneously set to FormatTests. > > Naoto From brian.burkhalter at oracle.com Thu Jan 11 22:14:01 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 11 Jan 2018 14:14:01 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <79e79fe7-eb4e-fdfd-1f0c-36ffdbf95eac@Oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> <60A17F10-D01C-4B02-825F-DEA7C31360FF@oracle.com> <79e79fe7-eb4e-fdfd-1f0c-36ffdbf95eac@Oracle.com> Message-ID: <811FCB65-0C8B-41D4-BD46-20127D2F072A@oracle.com> Thanks. The final version is updated in place under webrev.03. Pushing the change is pending CSR-reapproval. Brian On Jan 11, 2018, at 11:16 AM, Roger Riggs wrote: > +1 looks fine > > On 1/11/2018 11:59 AM, Brian Burkhalter wrote: >> While there appears now to be agreement on using naming option C below, formal Reviewer approval appears still to be lacking. It would be good if there were at least two approvals forthcoming, if appropriate. >> >> http://cr.openjdk.java.net/~bpb/4358774/webrev.03/ From kumar.x.srinivasan at oracle.com Thu Jan 11 23:00:16 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Thu, 11 Jan 2018 15:00:16 -0800 Subject: RFR: 8194287 and 8194286: simple test changes for the execution of these tests in a docket container Message-ID: <5A57EC80.2050909@oracle.com> Hi, Please review bug fixes for 2 test bugs, which fails with a base Linux installation in a docker container. Thanks Kumar [1] http://cr.openjdk.java.net/~ksrini/8194287/webrev-00/ [2] http://cr.openjdk.java.net/~ksrini/8194287/cmt.txt From jason.greene at redhat.com Fri Jan 12 04:33:17 2018 From: jason.greene at redhat.com (Jason Greene) Date: Thu, 11 Jan 2018 22:33:17 -0600 Subject: Can @Stable (or something similar) be made accessible? Message-ID: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> MethodHandle.invokeExact() can match the performance of direct invocation, but it requires constant folding to do so. Otherwise results are similar to Java reflection(e.g [1]). While TrustFinalNonStaticFields can be used, it?s a sledgehammer where a scalpel is really be more appropriate. The internal @Stable facility provides the desired semantics and precision, but it is heavily locked down, with privileged code checks and export restrictions. Could this be made more accessible (or perhaps a variant restricted to just final fields)? Informing the compiler that a final field is a true lazy initialized constant, with no store-to-final seems a pretty useful construct in general. I can understand that the long term desire is that this shouldn?t be necessary, and should be inferred [3], but at that point the annotation is still useful as documentation and legacy compatibility. If nothing else could it be allowed in non-privileged code via some flag? It seems odd that the much more aggressive facility (TrustFinalNonStaticFields) is simpler to use than a more targeted one. Thanks! [1] https://www.optaplanner.org/blog/2018/01/09/JavaReflectionButMuchFaster.html [2] http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/65464a307408/src/java.base/share/classes/jdk/internal/vm/annotation/Stable.java [3] https://bugs.openjdk.java.net/browse/JDK-8058164 From amy.lu at oracle.com Fri Jan 12 04:51:53 2018 From: amy.lu at oracle.com (Amy Lu) Date: Fri, 12 Jan 2018 12:51:53 +0800 Subject: [10] RFR 8194959: Correct test tag to move bugid from @test to @bug Message-ID: Please review this minor test-tag-only change to move bugid from @test to @bug bug: https://bugs.openjdk.java.net/browse/JDK-8194959 webrev: http://cr.openjdk.java.net/~amlu/8194959/webrev.00/ Thanks, Amy From sundararajan.athijegannathan at oracle.com Fri Jan 12 05:08:55 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Fri, 12 Jan 2018 10:38:55 +0530 Subject: [10] RFR 8194959: Correct test tag to move bugid from @test to @bug In-Reply-To: References: Message-ID: <5A5842E7.4050804@oracle.com> +1 -Sundar On 12/01/18, 10:21 AM, Amy Lu wrote: > Please review this minor test-tag-only change to move bugid from @test > to @bug > > bug: https://bugs.openjdk.java.net/browse/JDK-8194959 > webrev: http://cr.openjdk.java.net/~amlu/8194959/webrev.00/ > > Thanks, > Amy From aph at redhat.com Fri Jan 12 08:47:46 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 12 Jan 2018 08:47:46 +0000 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: On 12/01/18 04:33, Jason Greene wrote: > The internal @Stable facility provides the desired semantics and > precision, but it is heavily locked down, with privileged code > checks and export restrictions. Could this be made more accessible > (or perhaps a variant restricted to just final fields)? Informing > the compiler that a final field is a true lazy initialized constant, > with no store-to-final seems a pretty useful construct in general. I > can understand that the long term desire is that this shouldn?t be > necessary, and should be inferred [3], but at that point the > annotation is still useful as documentation and legacy > compatibility. If nothing else could it be allowed in non-privileged > code via some flag? I don't know of any way to do that without compromising the integrity of the JVM. All that anybody would have to do to break the VM is to define a field as @Stable and then change the field. I suppose you could say that Unsafe is just as bad and people still are allowed to use it, but there's understandable reluctance any further to transform a safe runtime into an unsafe one. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From Alan.Bateman at oracle.com Fri Jan 12 08:51:06 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Jan 2018 08:51:06 +0000 Subject: [10] RFR 8194959: Correct test tag to move bugid from @test to @bug In-Reply-To: References: Message-ID: On 12/01/2018 04:51, Amy Lu wrote: > Please review this minor test-tag-only change to move bugid from @test > to @bug > > bug: https://bugs.openjdk.java.net/browse/JDK-8194959 > webrev: http://cr.openjdk.java.net/~amlu/8194959/webrev.00/ Looks good. From amy.lu at oracle.com Fri Jan 12 08:56:10 2018 From: amy.lu at oracle.com (Amy Lu) Date: Fri, 12 Jan 2018 16:56:10 +0800 Subject: [10] RFR 8194959: Correct test tag to move bugid from @test to @bug In-Reply-To: References: Message-ID: <17bf235e-88c1-76fe-89f4-99ea7303ec06@oracle.com> Thank you Alan and Sundar for your quick review. Changes pushed. Thanks, Amy On 12/01/2018 4:51 PM, Alan Bateman wrote: > > > On 12/01/2018 04:51, Amy Lu wrote: >> Please review this minor test-tag-only change to move bugid from >> @test to @bug >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8194959 >> webrev: http://cr.openjdk.java.net/~amlu/8194959/webrev.00/ > Looks good. From bourges.laurent at gmail.com Fri Jan 12 09:06:41 2018 From: bourges.laurent at gmail.com (=?UTF-8?Q?Laurent_Bourg=C3=A8s?=) Date: Fri, 12 Jan 2018 10:06:41 +0100 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: Hi, I am the author of the Marlin renderer, integrated in both java.desktop & javafx.graphics modules. I wonder if such core annotations @Stable @ForceInline ... could be allowed to such openjdk modules in order to improve performance. Marlin already uses jdk.internal.Unsafe but @Stable looks promising as Marlin uses (and recycles) lots of int/double arrays. I already made my fields (& constants) final but it is not enough to let hotspot trust totally my code, isn't it ? I think these annotations are confined to java.lang for now. I will check thatasap. Cheers, Laurent Le 12 janv. 2018 9:50 AM, "Andrew Haley" a ?crit : > On 12/01/18 04:33, Jason Greene wrote: > > > The internal @Stable facility provides the desired semantics and > > precision, but it is heavily locked down, with privileged code > > checks and export restrictions. Could this be made more accessible > > (or perhaps a variant restricted to just final fields)? Informing > > the compiler that a final field is a true lazy initialized constant, > > with no store-to-final seems a pretty useful construct in general. I > > can understand that the long term desire is that this shouldn?t be > > necessary, and should be inferred [3], but at that point the > > annotation is still useful as documentation and legacy > > compatibility. If nothing else could it be allowed in non-privileged > > code via some flag? > > I don't know of any way to do that without compromising the integrity > of the JVM. All that anybody would have to do to break the VM is to > define a field as @Stable and then change the field. I suppose you > could say that Unsafe is just as bad and people still are allowed to > use it, but there's understandable reluctance any further to transform > a safe runtime into an unsafe one. > > -- > Andrew Haley > Java Platform Lead Engineer > Red Hat UK Ltd. > EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 > From aph at redhat.com Fri Jan 12 10:08:18 2018 From: aph at redhat.com (Andrew Haley) Date: Fri, 12 Jan 2018 10:08:18 +0000 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: On 12/01/18 09:06, Laurent Bourg?s wrote: > I already made my fields (& constants) final but it is not enough to let > hotspot trust totally my code, isn't it ? To some extent yes, to some extent no. HotSpot really does trust static final constants not to change, so @Stable won't help you with those. It's hard to optimize much of anything with fields that are not static final. -- Andrew Haley Java Platform Lead Engineer Red Hat UK Ltd. EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671 From dawid.weiss at gmail.com Fri Jan 12 10:08:48 2018 From: dawid.weiss at gmail.com (Dawid Weiss) Date: Fri, 12 Jan 2018 11:08:48 +0100 Subject: Inflater nowrap and padding byte requirement Message-ID: Hello, The javadoc for Inflater states that: * Note: When using the 'nowrap' option it is also necessary to provide * an extra "dummy" byte as input. This is required by the ZLIB native * library in order to support certain optimizations. This doesn't seem to be a requirement anymore in newer zlib versions, as clarified by Mark Adler himself [1]. The changelog for zlib 1.2.0 indeed mentions this "zero byte" padding isn't required anymore [2]: Changes in 1.2.0 (9 March 2003) [...] - Raw inflate no longer needs an extra dummy byte at end Perhaps it'd make sense to remove it from the JavaDoc, so that there's no need to complicate downstream code? Dawid [1] https://stackoverflow.com/a/9779941 [2] https://www.zlib.net/ChangeLog.txt From shade at redhat.com Fri Jan 12 10:37:37 2018 From: shade at redhat.com (Aleksey Shipilev) Date: Fri, 12 Jan 2018 11:37:37 +0100 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: <41070dfe-2d63-3b58-fbd4-3eaa6b820ff8@redhat.com> On 01/12/2018 05:33 AM, Jason Greene wrote: > MethodHandle.invokeExact() can match the performance of direct invocation, but it requires > constant folding to do so. Otherwise results are similar to Java reflection(e.g [1]). > > While TrustFinalNonStaticFields can be used, it?s a sledgehammer where a scalpel is really be > more appropriate. So, the easier way out would be enable TrustFinalNonStaticFields to be used with finer-grained control, that JEP 165 (Compiler Control) provides. Then you can tell which user-side classes/packages are safe for trusting, and which are not. IIRC, the reason why TrustFinalNonStaticFields is not on by default is because we cannot rely that everyone is not writing to final fields and expecting update to be visible. ...which makes this one partially go away: > It seems odd that the much more aggressive facility (TrustFinalNonStaticFields) is simpler to > use than a more targeted one. Thanks, -Aleksey From peter.levart at gmail.com Fri Jan 12 11:35:39 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 12 Jan 2018 12:35:39 +0100 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: Hi Andrew, On 01/12/2018 09:47 AM, Andrew Haley wrote: > On 12/01/18 04:33, Jason Greene wrote: > >> The internal @Stable facility provides the desired semantics and >> precision, but it is heavily locked down, with privileged code >> checks and export restrictions. Could this be made more accessible >> (or perhaps a variant restricted to just final fields)? Informing >> the compiler that a final field is a true lazy initialized constant, >> with no store-to-final seems a pretty useful construct in general. I >> can understand that the long term desire is that this shouldn?t be >> necessary, and should be inferred [3], but at that point the >> annotation is still useful as documentation and legacy >> compatibility. If nothing else could it be allowed in non-privileged >> code via some flag? > I don't know of any way to do that without compromising the integrity > of the JVM. All that anybody would have to do to break the VM is to > define a field as @Stable and then change the field. Would you be so kind to explain how this breakage occurs? I can understand that improper use of @Stable annotation may break the intended semantics of a program, but to break the integrity of VM? I'm trying to imagine the scenario, but can't. Regards, Peter From vitalyd at gmail.com Fri Jan 12 12:11:37 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 12 Jan 2018 12:11:37 +0000 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: On Fri, Jan 12, 2018 at 6:36 AM Peter Levart wrote: > Hi Andrew, > > On 01/12/2018 09:47 AM, Andrew Haley wrote: > > On 12/01/18 04:33, Jason Greene wrote: > > > >> The internal @Stable facility provides the desired semantics and > >> precision, but it is heavily locked down, with privileged code > >> checks and export restrictions. Could this be made more accessible > >> (or perhaps a variant restricted to just final fields)? Informing > >> the compiler that a final field is a true lazy initialized constant, > >> with no store-to-final seems a pretty useful construct in general. I > >> can understand that the long term desire is that this shouldn?t be > >> necessary, and should be inferred [3], but at that point the > >> annotation is still useful as documentation and legacy > >> compatibility. If nothing else could it be allowed in non-privileged > >> code via some flag? > > I don't know of any way to do that without compromising the integrity > > of the JVM. All that anybody would have to do to break the VM is to > > define a field as @Stable and then change the field. > > Would you be so kind to explain how this breakage occurs? I can > understand that improper use of @Stable annotation may break the > intended semantics of a program, but to break the integrity of VM? I'm > trying to imagine the scenario, but can't. One example might be a @Stable array field used with a constant index to write a value into it. If the JIT trusts the field to be final it could elide a range check when storing into the slot. If the field is actually modified to be a smaller length array, you?d end up with a write to an out of bounds memory area. I suppose something similar can be done with a non-array field - make the field type Object, store a Foo initially. The JIT can assume the type is always Foo and generate read/writes using its layout. Then change the field to another type. > > > Regards, Peter > > -- Sent from my phone From peter.levart at gmail.com Fri Jan 12 12:37:30 2018 From: peter.levart at gmail.com (Peter Levart) Date: Fri, 12 Jan 2018 13:37:30 +0100 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: <6380f289-bfae-896c-6a44-bc7fcfca6998@gmail.com> On 01/12/2018 01:11 PM, Vitaly Davidovich wrote: > On Fri, Jan 12, 2018 at 6:36 AM Peter Levart wrote: > >> Hi Andrew, >> >> On 01/12/2018 09:47 AM, Andrew Haley wrote: >>> On 12/01/18 04:33, Jason Greene wrote: >>> >>>> The internal @Stable facility provides the desired semantics and >>>> precision, but it is heavily locked down, with privileged code >>>> checks and export restrictions. Could this be made more accessible >>>> (or perhaps a variant restricted to just final fields)? Informing >>>> the compiler that a final field is a true lazy initialized constant, >>>> with no store-to-final seems a pretty useful construct in general. I >>>> can understand that the long term desire is that this shouldn?t be >>>> necessary, and should be inferred [3], but at that point the >>>> annotation is still useful as documentation and legacy >>>> compatibility. If nothing else could it be allowed in non-privileged >>>> code via some flag? >>> I don't know of any way to do that without compromising the integrity >>> of the JVM. All that anybody would have to do to break the VM is to >>> define a field as @Stable and then change the field. >> Would you be so kind to explain how this breakage occurs? I can >> understand that improper use of @Stable annotation may break the >> intended semantics of a program, but to break the integrity of VM? I'm >> trying to imagine the scenario, but can't. > One example might be a @Stable array field used with a constant index to > write a value into it. If the JIT trusts the field to be final it could > elide a range check when storing into the slot. If the field is actually > modified to be a smaller length array, you?d end up with a write to an out > of bounds memory area. > > I suppose something similar can be done with a non-array field - make the > field type Object, store a Foo initially. The JIT can assume the type is > always Foo and generate read/writes using its layout. Then change the > field to another type. So you are saying that JIT might optimize some aspects of code assuming the "constantness" of the value of the field (for example inline code for a particular type of object or skip index range check), but not optimize other aspects - for example still read the value of the field from the field's primary location in containing object to obtain 'this' pointer or index value. Is this really they way JIT does compilation/optimization? I would assume that if JIT optimizes code assuming some value is constant, it also uses this same constant value in the optimized code and not emit code that actually reads the value from the primary field location or assume that that value already resides in some register which was established in non-optimized part of code. I'm probably wrong in assuming that. So one that uses @Stable annotation on fields in JDK internals must be very careful not to break the rules which are very subtle or he may create situations that compromise stability of VM? There are places in JDK using @Stable annotations where such marked fields can be written to more than once with different values (racy caching). Are you saying that all those values that are written to a @Stable field must be equivalent not only in terms of Java logic but in case they are reference values, they must also point to the object of the same runtime class or else the stability of VM may be compromised? Thanks, Peter > >> >> Regards, Peter >> >> -- > Sent from my phone > From vladimir.x.ivanov at oracle.com Fri Jan 12 12:50:02 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Fri, 12 Jan 2018 15:50:02 +0300 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: >> Would you be so kind to explain how this breakage occurs? I can >> understand that improper use of @Stable annotation may break the >> intended semantics of a program, but to break the integrity of VM? I'm >> trying to imagine the scenario, but can't. > > One example might be a @Stable array field used with a constant index to > write a value into it. If the JIT trusts the field to be final it could > elide a range check when storing into the slot. If the field is actually > modified to be a smaller length array, you?d end up with a write to an out > of bounds memory area. > > I suppose something similar can be done with a non-array field - make the > field type Object, store a Foo initially. The JIT can assume the type is > always Foo and generate read/writes using its layout. Then change the > field to another type. No, it doesn't work that way. @Stable enables JIT to constant fold loads from fields/arrays (holding non-default values) whenever possible. After that JIT has a constant value at its hands. All other optimizations follows from that: range check before array store can be elided since array instance is known and its length is known as well, type check can be elided since constant type is known. But the value won't be reloaded from memory, all optimizations happen on a constant which was loaded from memory once during compilation. So, the worst case scenario is: a value written into @Stable field/array is never visible in some code no matter what you do. It can lead to nasty bugs when different parts of program don't agree on observed value. It can happen when user doesn't obey @Stable contract and performs multiple writes into a @Stable field/array. Current implementation in HotSpot doesn't forbid that. Best regards, Vladimir Ivanov From vitalyd at gmail.com Fri Jan 12 13:41:06 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 12 Jan 2018 08:41:06 -0500 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: On Fri, Jan 12, 2018 at 7:50 AM, Vladimir Ivanov < vladimir.x.ivanov at oracle.com> wrote: > Would you be so kind to explain how this breakage occurs? I can >>> understand that improper use of @Stable annotation may break the >>> intended semantics of a program, but to break the integrity of VM? I'm >>> trying to imagine the scenario, but can't. >>> >> >> One example might be a @Stable array field used with a constant index to >> write a value into it. If the JIT trusts the field to be final it could >> elide a range check when storing into the slot. If the field is actually >> modified to be a smaller length array, you?d end up with a write to an out >> of bounds memory area. >> >> I suppose something similar can be done with a non-array field - make the >> field type Object, store a Foo initially. The JIT can assume the type is >> always Foo and generate read/writes using its layout. Then change the >> field to another type. >> > > No, it doesn't work that way. @Stable enables JIT to constant fold loads > from fields/arrays (holding non-default values) whenever possible. After > that JIT has a constant value at its hands. All other optimizations follows > from that: range check before array store can be elided since array > instance is known and its length is known as well, type check can be elided > since constant type is known. But the value won't be reloaded from memory, > all optimizations happen on a constant which was loaded from memory once > during compilation. > > So, the worst case scenario is: a value written into @Stable field/array > is never visible in some code no matter what you do. It can lead to nasty > bugs when different parts of program don't agree on observed value. It can > happen when user doesn't obey @Stable contract and performs multiple writes > into a @Stable field/array. Current implementation in HotSpot doesn't > forbid that. > Thanks for clarifying Vladimir - that makes sense. So back to the integrity of the VM question - what can be compromised then? I suppose if one was to use Unsafe access in combination with relying on a @Stable field to feed it info (e.g. index into some memory accessed via Unsafe), then things can go south. But Unsafe is already, well, unsafe. What extra VM integrity issues would a misbehaving @Stable cause that Andrew was alluding to? > > Best regards, > Vladimir Ivanov > From Roger.Riggs at Oracle.com Fri Jan 12 14:37:16 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Jan 2018 09:37:16 -0500 Subject: RFR: 8194287 and 8194286: simple test changes for the execution of these tests in a docket container In-Reply-To: <5A57EC80.2050909@oracle.com> References: <5A57EC80.2050909@oracle.com> Message-ID: <3bb07b33-1336-9a89-1d78-13850e7e4307@Oracle.com> Hi Kumar, RunPathTest should say *what* is missing from the environment to skip the test. Otherwise, the curious has to go back a read the code to figure it out. Thanks, Roger On 1/11/2018 6:00 PM, Kumar Srinivasan wrote: > Hi, > > Please review bug fixes for 2 test bugs, which fails with a? base > Linux installation > in a docker container. > > Thanks > Kumar > > [1] http://cr.openjdk.java.net/~ksrini/8194287/webrev-00/ > [2] http://cr.openjdk.java.net/~ksrini/8194287/cmt.txt From bob.vandette at oracle.com Fri Jan 12 15:02:28 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Fri, 12 Jan 2018 10:02:28 -0500 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers In-Reply-To: <9FEEDEC9-2BF5-4CEC-A6D4-E11273573204@oracle.com> References: <39C5B4B1-6EEF-4E07-90BA-5EC77B83195B@oracle.com> <9FEEDEC9-2BF5-4CEC-A6D4-E11273573204@oracle.com> Message-ID: <88DDE9EA-91B7-4249-B45B-27C8D550DD8C@oracle.com> Looping in core-libs. The following fix alters the module-info for java.base. Can someone from the core-libs comment on these changes? I?d also like to remove the two PerfDataFile.getFile methods? Since these are in jdk.internal.jvmstat, can I just pull them or do they need to go through deprecation/CSR? Thanks, Bob. > On Jan 12, 2018, at 8:59 AM, Bob Vandette wrote: > > > >> On Jan 11, 2018, at 9:44 PM, David Holmes wrote: >> >> Hi Bob, >> >> Some initial discussion and commentary. I'm a bit concerned by how much accommodating docker penalises code that is not using containers. > > True but querying the number of Java processes is at least not in a performance critical area. > >> >> On 12/01/2018 4:41 AM, Bob Vandette wrote: >>> Bug: >>> https://bugs.openjdk.java.net/browse/JDK-8193710 >>> Webrev: >>> http://cr.openjdk.java.net/~bobv/8193710/webrev.00/ >>> Summary: >>> This changeset enables the ability to use jcmd and jps running on a Host to >>> list the java processes that are running in docker (cgroup based) containers. >> >> Can you clarify this for me please. I assume we are still only listing processes for the current user - correct? > > No, it will report any Java processes that the current user has permission to see (hsperfdata* files are readable). > If you run as root, you see all processes. > >> So the issue is if I am running mutiple JVMs and also running a Docker process that itself contains JVMs then jps outside of Docker won't show the JVMs that are inside Docker. > > Yes, this is what is being fixed. > >> And presumably the opposite is also true? >> > > Yes, this is also true but is not being fixed by this change. I am only fixing the case where you run jps on a host and > want to see all java processes running (in and out of containers). > > >> If we change jps to find all JVMs on the host for the current user, whether in a container or not (and whether jps is run from in a container or not?) are the process id's that get returned directly useable from where jps was run? > > Given the above constraints, yes. > > The results jps can passed to jcmd to query things like VM.version > >> >>> I?ve tested this change by examining processes running as root on both host and in >>> docker containers as well as under my userid using ?jps and jcmd -l?. >>> I?ve also tested the getFile functions with a small example program that I wrote. >>> From what I can tell, these two getFile functions are not used in the JDK sources >>> and in fact the PerfDataFile.getFile(int lvmid) method doesn?t appear to have never worked! >>> I?d really like to remove these two methods. >> >> If they are not used by anything then by all means drop them. > I saw some Javadocs on the web documenting these methods leading me to believe that we would need > to deprecate these even though they are internal APIs. I?ll ask core-libs. > > http://openjdk.java.net/groups/serviceability/jvmstat/sun/jvmstat/perfdata/monitor/protocol/local/PerfDataFile.html#getFile(int) > >> >>> 140 candidate = new File(f.getName(), name); <>> Here are some implementation details that I?ve added added to one of the Unix >>> specific source files (src/java.base/unix/classes/jdk/internal/vm/VMSupportImpl.java) >> >> I have an issue with this. This is not "unix" (ie anything other than windows) support code, it is Linux support code. I'm also unclear why this needs to be lifted out of the PerfMemory subsystem. > You are correct, I?ll move this into a linux specific directory. I factored these functions in order to isolate the cgroup specific functionality > in an OS specific tree. > >> >> And if you're going to mess with the jdk.internal module then I think core-libs folk will want to know before hand. > > Will do. > >> >> Other comments ... >> >> src/hotspot/os/linux/perfMemory_linux.cpp >> >> I'm concerned about the MAXPATHLEN stack buffers and the extra overhead being added to the normal non-container path. > > I believe the impacted paths are only used when attaching to the VM and should not impact normal startup. > >> We've had issues in the past where a newly added stack buffer caused StackOverflowError. We don't need a 4K buffer because os::get_temp_directory() just returns "/tmp" and we're dealing with all Linux code here. I don't like making assumptions like this but ... >> >> #define TMP_BUFFER_LEN (4 + 21) >> static char* get_user_tmp_dir(const char* user, int vmid, int nspid) { >> char buffer[TMP_BUFFER_LEN]; >> char* tmpdir = os::get_temp_directory(); >> assert(strlen(tmpdir) == 4, "No longer using /tmp - update buffer size"); >> if (nspid != -1) { >> jio_snprintf(buffer, TMP_BUFFER_LEN, "/proc/%d/root%s", vmid, tmpdir); >> tmpdir = buffer; >> } >> ... >> >> >> 657 char fname[64]; >> >> Maximum length of "/proc/%d/status" is 23. > > Ok, I?ll reduce the buffer sizes I use. > > Thanks, > Bob. > > >> >> Thanks, >> David > From kumar.x.srinivasan at oracle.com Fri Jan 12 15:42:00 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Fri, 12 Jan 2018 07:42:00 -0800 Subject: RFR: 8194287 and 8194286: simple test changes for the execution of these tests in a docket container In-Reply-To: <3bb07b33-1336-9a89-1d78-13850e7e4307@Oracle.com> References: <5A57EC80.2050909@oracle.com> <3bb07b33-1336-9a89-1d78-13850e7e4307@Oracle.com> Message-ID: <5A58D748.2040907@oracle.com> Hi Roger, Thanks for looking at this. > Hi Kumar, > > RunPathTest should say *what* is missing from the environment to skip > the test. > Otherwise, the curious has to go back a read the code to figure it out. Actually it *does* say *what* is missing, however it will be printed before this message. webrev does not quite make this clear as it gobbles up areas of the source files which have are not changed, please see this line, it will become apparent. http://cr.openjdk.java.net/~ksrini/8194287/webrev-00/test/jdk/tools/launcher/RunpathTest.java.html line 52 Thanks Kumar > > Thanks, Roger > > > On 1/11/2018 6:00 PM, Kumar Srinivasan wrote: >> Hi, >> >> Please review bug fixes for 2 test bugs, which fails with a base >> Linux installation >> in a docker container. >> >> Thanks >> Kumar >> >> [1] http://cr.openjdk.java.net/~ksrini/8194287/webrev-00/ >> [2] http://cr.openjdk.java.net/~ksrini/8194287/cmt.txt > From Roger.Riggs at Oracle.com Fri Jan 12 15:54:34 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Jan 2018 10:54:34 -0500 Subject: RFR: 8194287 and 8194286: simple test changes for the execution of these tests in a docket container In-Reply-To: <5A58D748.2040907@oracle.com> References: <5A57EC80.2050909@oracle.com> <3bb07b33-1336-9a89-1d78-13850e7e4307@Oracle.com> <5A58D748.2040907@oracle.com> Message-ID: ok, in that case, no additional Warning message is needed, the useful warning has already been printed. Thanks, Roger On 1/12/2018 10:42 AM, Kumar Srinivasan wrote: > Hi Roger, > > Thanks for looking at this. > >> Hi Kumar, >> >> RunPathTest should say *what* is missing from the environment to skip >> the test. >> Otherwise, the curious has to go back a read the code to figure it out. > > > Actually it? *does* say *what* is missing, however it will be printed > before this message. > > webrev does not quite make this clear as it gobbles up areas of the > source files which have > are not changed, please see this line, it will become apparent. > > http://cr.openjdk.java.net/~ksrini/8194287/webrev-00/test/jdk/tools/launcher/RunpathTest.java.html > > line 52 > > Thanks > Kumar >> >> Thanks, Roger >> >> >> On 1/11/2018 6:00 PM, Kumar Srinivasan wrote: >>> Hi, >>> >>> Please review bug fixes for 2 test bugs, which fails with a base >>> Linux installation >>> in a docker container. >>> >>> Thanks >>> Kumar >>> >>> [1] http://cr.openjdk.java.net/~ksrini/8194287/webrev-00/ >>> [2] http://cr.openjdk.java.net/~ksrini/8194287/cmt.txt >> > From jason.greene at redhat.com Fri Jan 12 16:28:07 2018 From: jason.greene at redhat.com (Jason Greene) Date: Fri, 12 Jan 2018 10:28:07 -0600 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: <0DAD4412-5E7E-4992-92CF-27CD08F9E237@redhat.com> > On Jan 12, 2018, at 6:50 AM, Vladimir Ivanov wrote: > -snip- > So, the worst case scenario is: a value written into @Stable field/array is never visible in some code no matter what you do. It can lead to nasty bugs when different parts of program don't agree on observed value. It can happen when user doesn't obey @Stable contract and performs multiple writes into a @Stable field/array. Current implementation in HotSpot doesn?t > forbid that. > This isn?t much different from the already allowed fine tuning of memory model effects (e.g., observing stale values from a plain write). In those scenarios, as is the case with @Stable, developers have to be aware of and honor the respective contract. It could be argued that bugs arising from contract violations of @Stable are easier to catch, due to their persistent nature. -Jason From xueming.shen at oracle.com Fri Jan 12 16:48:37 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 12 Jan 2018 08:48:37 -0800 Subject: RFR JDK-8194667: Regex: Serialization doesn't work with match flags Message-ID: <5A58E6E5.80206@oracle.com> Hi, Please help review the change for JDK-8194667 issue: https://bugs.openjdk.java.net/browse/JDK-8194667 webrev: http://cr.openjdk.java.net/~sherman/8194667/webrev The bits of "flags" are being updated on and off during the pattern compiling by the possible embedded match flag(s) in old implementation. The proposed fix is to add a temporary field "flags0" for this purpose and save/store "flags" for the purpose of deserialization (same as the field "pattern"). For the compatibility concern I'm leaving the behavior of method flags() unchanged to return the muted "flags". Arguably it might be desired to return the original "flags" instead. We can leave this "issue" for future RFE. 1145 public int flags() { 1146 return flags0; 1147 } Thanks, Sherman From vitalyd at gmail.com Fri Jan 12 17:18:42 2018 From: vitalyd at gmail.com (Vitaly Davidovich) Date: Fri, 12 Jan 2018 17:18:42 +0000 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <0DAD4412-5E7E-4992-92CF-27CD08F9E237@redhat.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> <0DAD4412-5E7E-4992-92CF-27CD08F9E237@redhat.com> Message-ID: On Fri, Jan 12, 2018 at 11:28 AM Jason Greene wrote: > > > On Jan 12, 2018, at 6:50 AM, Vladimir Ivanov < > vladimir.x.ivanov at oracle.com> wrote: > > > > -snip- > > > So, the worst case scenario is: a value written into @Stable > field/array is never visible in some code no matter what you do. It can > lead to nasty bugs when different parts of program don't agree on observed > value. It can happen when user doesn't obey @Stable contract and performs > multiple writes into a @Stable field/array. Current implementation in > HotSpot doesn?t > > forbid that. > > > > > This isn?t much different from the already allowed fine tuning of memory > model effects (e.g., observing stale values from a plain write). In those > scenarios, as is the case with @Stable, developers have to be aware of and > honor the respective contract. It could be argued that bugs arising from > contract violations of @Stable are easier to catch, due to their persistent > nature. Theyre only sort of persistent - they?re persistent only after the code is JIT?d and also stays JIT?d (ie no deopt back to interpreter). But JIT itself is non-deterministic so it could create some really funky behavior, and maybe even heisenbugs. > > > -Jason -- Sent from my phone From forax at univ-mlv.fr Fri Jan 12 17:51:12 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 12 Jan 2018 18:51:12 +0100 (CET) Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <41070dfe-2d63-3b58-fbd4-3eaa6b820ff8@redhat.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> <41070dfe-2d63-3b58-fbd4-3eaa6b820ff8@redhat.com> Message-ID: <938719012.2858446.1515779472284.JavaMail.zimbra@u-pem.fr> Or you can use method handles ... I've not tested but this should work: https://gist.github.com/forax/e49d63d50e5973f602f720ba3b6ea1b8 I have implemented two semantics, either do a call to get field value the first time (FIRST_WIN) and then for the subsequent calls always return this value as a constant or put a guard (if) that checks that the reference never change so instead of getting the field value, i always return the same constant (STABLE). You may think that it doesn't work because the method handle is a captured value of a lambda that is stored in a field of the lambda proxy but the VM has a special case for that (fields of VM anonynous classes (as the lambda proxies) are always trusted, see [1]). So it's safe, from the perf POV to use lambdas has a mechanism to hidden the method handle incantations. cheers, R?mi [1] https://shipilev.net/jvm-anatomy-park/17-trust-nonstatic-final-fields/ ----- Mail original ----- > De: "Aleksey Shipilev" > ?: "Jason Greene" , "core-libs-dev" > Envoy?: Vendredi 12 Janvier 2018 11:37:37 > Objet: Re: Can @Stable (or something similar) be made accessible? > On 01/12/2018 05:33 AM, Jason Greene wrote: >> MethodHandle.invokeExact() can match the performance of direct invocation, but >> it requires >> constant folding to do so. Otherwise results are similar to Java reflection(e.g >> [1]). >> >> While TrustFinalNonStaticFields can be used, it?s a sledgehammer where a scalpel >> is really be >> more appropriate. > > So, the easier way out would be enable TrustFinalNonStaticFields to be used with > finer-grained > control, that JEP 165 (Compiler Control) provides. Then you can tell which > user-side > classes/packages are safe for trusting, and which are not. IIRC, the reason why > TrustFinalNonStaticFields is not on by default is because we cannot rely that > everyone is not > writing to final fields and expecting update to be visible. > > ...which makes this one partially go away: > >> It seems odd that the much more aggressive facility (TrustFinalNonStaticFields) >> is simpler to >> use than a more targeted one. > Thanks, > -Aleksey From paul.sandoz at oracle.com Fri Jan 12 18:09:27 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 12 Jan 2018 10:09:27 -0800 Subject: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations In-Reply-To: <391296167.1979044.1513941333703.JavaMail.zimbra@u-pem.fr> References: <85218D91-BE9C-458C-BDF6-8E9FEAEC9647@oracle.com> <1907603756.1806400.1513900017261.JavaMail.zimbra@u-pem.fr> <391296167.1979044.1513941333703.JavaMail.zimbra@u-pem.fr> Message-ID: Hi Remi, Catching up after the holidays. I would prefer to leave both things as they are. Thanks, Paul. > On 22 Dec 2017, at 03:15, forax at univ-mlv.fr wrote: > > > > De: "Paul Sandoz" > ?: "Remi Forax" > Cc: "core-libs-dev" > Envoy?: Vendredi 22 D?cembre 2017 01:38:37 > Objet: Re: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations > > > On 21 Dec 2017, at 15:46, Remi Forax wrote: > > Hi Paul, > three things: > - I think you should add a comment to explain why you have chosen to create a the field downstream* in the primitive implementations, > I suppose it's to avoid to allocate a lambda proxy at each call. > > Yes, i included this comment: > // cache the consumer to avoid creation on every accepted element > > - the fields in the inner classes cancellationRequested and downstream* should be private. > > Does it make any difference for such inner classes? > > Usually, a stronger encapsulation is better than a weaker one. Being an anonymous class only means that you can not access the type, not that the fields are not accessible. > > (I lean towards package private as the default.) > > I tend to think that before, but with nestmate around the corner, private now really means accessible in the same java file, so i do think that private should be the new default in nested classes. > > > > - if you use var, you should use a meaningful name, here, 's' can be replaced by 'spliterator', making the code more readable. > > > Don?t agree in this case, given the locality of use and given the method name on the RHS. > > It means that when you want to know what a variable is you have to read its initialization part, it's a bit of a stretch compare to what Brian said about var, we can use var because the variable name is meaningful enough, so the type is redundant. I suppose it's not a big deal if you read the code sequentially but i tend to read the meat part of the code and extend above and below, so i was focus on the code that uses downstreamAsInt when i ask myself what 's' was. > > > Thanks, > Paul. > > cheers, > R?mi > From Alan.Bateman at oracle.com Fri Jan 12 18:12:19 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 12 Jan 2018 18:12:19 +0000 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <811FCB65-0C8B-41D4-BD46-20127D2F072A@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> <60A17F10-D01C-4B02-825F-DEA7C31360FF@oracle.com> <79e79fe7-eb4e-fdfd-1f0c-36ffdbf95eac@Oracle.com> <811FCB65-0C8B-41D4-BD46-20127D2F072A@oracle.com> Message-ID: <68a0ea4c-42d2-e544-39a0-87c06e55621d@oracle.com> On 11/01/2018 22:14, Brian Burkhalter wrote: > Thanks. The final version is updated in place under webrev.03. Pushing the change is pending CSR-reapproval. > > Looks okay to me. Just a minor nit in the first line of nullInputStream where the javadoc has "contains no bytes". I assume you mean to put "reads" rather than "contains". -Alan From forax at univ-mlv.fr Fri Jan 12 18:18:35 2018 From: forax at univ-mlv.fr (Remi Forax) Date: Fri, 12 Jan 2018 18:18:35 +0000 Subject: [10] RFR 8075939: Stream.flatMap() causes breaking of short-circuiting of terminal operations In-Reply-To: References: <85218D91-BE9C-458C-BDF6-8E9FEAEC9647@oracle.com> <1907603756.1806400.1513900017261.JavaMail.zimbra@u-pem.fr> <391296167.1979044.1513941333703.JavaMail.zimbra@u-pem.fr> Message-ID: sure, if you prefer, it's fine for me. Remi On January 12, 2018 6:09:27 PM UTC, Paul Sandoz wrote: >Hi Remi, > >Catching up after the holidays. I would prefer to leave both things as >they are. > >Thanks, >Paul. > >> On 22 Dec 2017, at 03:15, forax at univ-mlv.fr wrote: >> >> >> >> De: "Paul Sandoz" >> ?: "Remi Forax" >> Cc: "core-libs-dev" >> Envoy?: Vendredi 22 D?cembre 2017 01:38:37 >> Objet: Re: [10] RFR 8075939: Stream.flatMap() causes breaking of >short-circuiting of terminal operations >> >> >> On 21 Dec 2017, at 15:46, Remi Forax wrote: >> >> Hi Paul, >> three things: >> - I think you should add a comment to explain why you have chosen to >create a the field downstream* in the primitive implementations, >> I suppose it's to avoid to allocate a lambda proxy at each call. >> >> Yes, i included this comment: >> // cache the consumer to avoid creation on every accepted element >> >> - the fields in the inner classes cancellationRequested and >downstream* should be private. >> >> Does it make any difference for such inner classes? >> >> Usually, a stronger encapsulation is better than a weaker one. Being >an anonymous class only means that you can not access the type, not >that the fields are not accessible. >> >> (I lean towards package private as the default.) >> >> I tend to think that before, but with nestmate around the corner, >private now really means accessible in the same java file, so i do >think that private should be the new default in nested classes. >> >> >> >> - if you use var, you should use a meaningful name, here, 's' can be >replaced by 'spliterator', making the code more readable. >> >> >> Don?t agree in this case, given the locality of use and given the >method name on the RHS. >> >> It means that when you want to know what a variable is you have to >read its initialization part, it's a bit of a stretch compare to what >Brian said about var, we can use var because the variable name is >meaningful enough, so the type is redundant. I suppose it's not a big >deal if you read the code sequentially but i tend to read the meat part >of the code and extend above and below, so i was focus on the code that >uses downstreamAsInt when i ask myself what 's' was. >> >> >> Thanks, >> Paul. >> >> cheers, >> R?mi >> -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From Roger.Riggs at Oracle.com Fri Jan 12 18:22:43 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 12 Jan 2018 13:22:43 -0500 Subject: RFR JDK-8194667: Regex: Serialization doesn't work with match flags In-Reply-To: <5A58E6E5.80206@oracle.com> References: <5A58E6E5.80206@oracle.com> Message-ID: Hi Sherman, flag0 javadoc: /** 957 * The pattern flags used during compiling. The flags might be turn*ed* on and 958 * off by*an *embedded flag. 959 */ 2946-3047: Can an exception happen between save and restore? Would the value of flag0 matter if so? +Copyright year update Regards, Roger On 1/12/2018 11:48 AM, Xueming Shen wrote: > Hi, > > Please help review the change for JDK-8194667 > > issue: https://bugs.openjdk.java.net/browse/JDK-8194667 > webrev: http://cr.openjdk.java.net/~sherman/8194667/webrev > > The bits of "flags" are being updated on and off during the pattern > compiling by > the possible embedded match flag(s) in old implementation. The > proposed fix is > to add a temporary field "flags0" for this purpose and save/store > "flags" for the > purpose of deserialization (same as the field "pattern"). > > For the compatibility concern I'm leaving the behavior of method > flags() unchanged > to return the muted "flags". Arguably it might be desired to return > the original "flags" > instead. We can leave this "issue" for future RFE. > > 1145???? public int flags() { > 1146???????? return flags0; > 1147???? } > > > Thanks, > Sherman From xueming.shen at oracle.com Fri Jan 12 18:36:55 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 12 Jan 2018 10:36:55 -0800 Subject: RFR JDK-8194667: Regex: Serialization doesn't work with match flags In-Reply-To: References: <5A58E6E5.80206@oracle.com> Message-ID: <5A590047.4040808@oracle.com> On 1/12/18, 10:22 AM, Roger Riggs wrote: > Hi Sherman, > > flag0 javadoc: > > /** > 957 * The pattern flags used during compiling. The flags might > be turn*ed* on and > 958 * off by*an *embedded flag. > 959 */ > > 2946-3047: Can an exception happen between save and restore? Would the > value of flag0 matter if so? if there is any exception thrown (in the middle of pattern compiling) you don't even get a Pattern object. So the status of "flag0" does not matter at all in this circumstance. > > +Copyright year update will be updated accordingly. thanks, sherman > > Regards, Roger > > On 1/12/2018 11:48 AM, Xueming Shen wrote: >> Hi, >> >> Please help review the change for JDK-8194667 >> >> issue: https://bugs.openjdk.java.net/browse/JDK-8194667 >> webrev: http://cr.openjdk.java.net/~sherman/8194667/webrev >> >> The bits of "flags" are being updated on and off during the pattern >> compiling by >> the possible embedded match flag(s) in old implementation. The >> proposed fix is >> to add a temporary field "flags0" for this purpose and save/store >> "flags" for the >> purpose of deserialization (same as the field "pattern"). >> >> For the compatibility concern I'm leaving the behavior of method >> flags() unchanged >> to return the muted "flags". Arguably it might be desired to return >> the original "flags" >> instead. We can leave this "issue" for future RFE. >> >> 1145 public int flags() { >> 1146 return flags0; >> 1147 } >> >> >> Thanks, >> Sherman > From brian.burkhalter at oracle.com Fri Jan 12 18:59:25 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Fri, 12 Jan 2018 10:59:25 -0800 Subject: RFR 4358774: Add null InputStream and OutputStream In-Reply-To: <68a0ea4c-42d2-e544-39a0-87c06e55621d@oracle.com> References: <09B7A146-48BC-4600-9C2B-01517F0A2232@oracle.com> <71749D2C-7B1F-4D2E-86E7-90A54533B882@oracle.com> <41D04E64-ACE4-40C7-B9A9-F484265363FF@oracle.com> <60A17F10-D01C-4B02-825F-DEA7C31360FF@oracle.com> <79e79fe7-eb4e-fdfd-1f0c-36ffdbf95eac@Oracle.com> <811FCB65-0C8B-41D4-BD46-20127D2F072A@oracle.com> <68a0ea4c-42d2-e544-39a0-87c06e55621d@oracle.com> Message-ID: On Jan 12, 2018, at 10:12 AM, Alan Bateman wrote: > On 11/01/2018 22:14, Brian Burkhalter wrote: >> Thanks. The final version is updated in place under webrev.03. Pushing the change is pending CSR-reapproval. >> >> > Looks okay to me. Just a minor nit in the first line of nullInputStream where the javadoc has "contains no bytes". I assume you mean to put "reads" rather than "contains". I have not pushed it yet so I will change that in place before doing so. Thanks, Brian From huizhe.wang at oracle.com Fri Jan 12 19:40:39 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 12 Jan 2018 11:40:39 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8181047: Add comment to technical terms that shall not be translated Message-ID: <56467db4-88b1-c19e-7a7d-6c6b033b3995@oracle.com> Hi, Please review a doc-only change that adds a few notes to message files that serves as a notification to translation teams that some technical terms shall not be translated. JBS: https://bugs.openjdk.java.net/browse/JDK-8181047 webres: http://cr.openjdk.java.net/~joehw/jdk10/8181047/webrev/ Thanks, Joe From lance.andersen at oracle.com Fri Jan 12 20:13:02 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Fri, 12 Jan 2018 15:13:02 -0500 Subject: RFR (JDK10/JAXP Doc-only) 8181047: Add comment to technical terms that shall not be translated In-Reply-To: <56467db4-88b1-c19e-7a7d-6c6b033b3995@oracle.com> References: <56467db4-88b1-c19e-7a7d-6c6b033b3995@oracle.com> Message-ID: <9D3957AA-2D13-4729-B405-AA5C64D840BA@oracle.com> +1 > On Jan 12, 2018, at 2:40 PM, huizhe wang wrote: > > Hi, > > Please review a doc-only change that adds a few notes to message files that serves as a notification to translation teams that some technical terms shall not be translated. > > JBS: > https://bugs.openjdk.java.net/browse/JDK-8181047 > webres: > http://cr.openjdk.java.net/~joehw/jdk10/8181047/webrev/ > > Thanks, > Joe Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From jonathan.gibbons at oracle.com Fri Jan 12 22:31:39 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Fri, 12 Jan 2018 14:31:39 -0800 Subject: RFR(XXS) : 8195067 : problem list tools/javac/jvm/VerboseOutTest.java In-Reply-To: References: Message-ID: <5A59374B.2040002@oracle.com> OK. Sorry for the bad test. -- Jon On 01/12/2018 02:29 PM, Igor Ignatyev wrote: > http://cr.openjdk.java.net/~iignatyev//8195067/webrev.00/index.html >> 2 lines changed: 1 ins; 0 del; 1 mod; > Hi all, > > could you please review this tiny fix which put tools/javac/jvm/VerboseOutTest.java into the problem list till 8194968 is fixed? > > webrev: http://cr.openjdk.java.net/~iignatyev//8195067/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8195067 > > Thanks, > -- Igor From igor.ignatyev at oracle.com Fri Jan 12 22:29:02 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 12 Jan 2018 14:29:02 -0800 Subject: RFR(XXS) : 8195067 : problem list tools/javac/jvm/VerboseOutTest.java Message-ID: http://cr.openjdk.java.net/~iignatyev//8195067/webrev.00/index.html > 2 lines changed: 1 ins; 0 del; 1 mod; Hi all, could you please review this tiny fix which put tools/javac/jvm/VerboseOutTest.java into the problem list till 8194968 is fixed? webrev: http://cr.openjdk.java.net/~iignatyev//8195067/webrev.00/index.html JBS: https://bugs.openjdk.java.net/browse/JDK-8195067 Thanks, -- Igor From jesper.wilhelmsson at oracle.com Fri Jan 12 22:34:17 2018 From: jesper.wilhelmsson at oracle.com (jesper.wilhelmsson at oracle.com) Date: Fri, 12 Jan 2018 23:34:17 +0100 Subject: RFR(XXS) : 8195067 : problem list tools/javac/jvm/VerboseOutTest.java In-Reply-To: References: Message-ID: <571FBCDC-BBDD-4DD1-99DD-15D7E50895E4@oracle.com> Looks good! Thanks for problem listing this test! /Jesper > On 12 Jan 2018, at 23:29, Igor Ignatyev wrote: > > http://cr.openjdk.java.net/~iignatyev//8195067/webrev.00/index.html >> 2 lines changed: 1 ins; 0 del; 1 mod; > > Hi all, > > could you please review this tiny fix which put tools/javac/jvm/VerboseOutTest.java into the problem list till 8194968 is fixed? > > webrev: http://cr.openjdk.java.net/~iignatyev//8195067/webrev.00/index.html > JBS: https://bugs.openjdk.java.net/browse/JDK-8195067 > > Thanks, > -- Igor From kumar.x.srinivasan at oracle.com Sat Jan 13 00:31:22 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Fri, 12 Jan 2018 16:31:22 -0800 Subject: RFR: 8195072: Update ASM 3rd party legal (TPL) copyright to 6.0 Message-ID: <5A59535A.1060701@oracle.com> Hello, Could I get a review for this simple update to the 3rd party legal (TPL) information. Thanks Webrev: http://cr.openjdk.java.net/~ksrini/8195072/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8195072 Current ASM copyright: http://hg.openjdk.java.net/jdk/jdk10/file/c674ff28c69d/src/java.base/share/classes/jdk/internal/org/objectweb/asm/AnnotationVisitor.java Current ASM version: http://hg.openjdk.java.net/jdk/jdk10/file/c674ff28c69d/src/java.base/share/classes/jdk/internal/org/objectweb/asm/version.txt From li.jiang at oracle.com Sat Jan 13 01:03:29 2018 From: li.jiang at oracle.com (Leo Jiang) Date: Sat, 13 Jan 2018 09:03:29 +0800 Subject: RFR (JDK10/JAXP Doc-only) 8181047: Add comment to technical terms that shall not be translated In-Reply-To: <56467db4-88b1-c19e-7a7d-6c6b033b3995@oracle.com> References: <56467db4-88b1-c19e-7a7d-6c6b033b3995@oracle.com> Message-ID: <22095d26-a0fa-17e6-1b96-272054a2da82@oracle.com> Maybe you should also add a comment for this line: NoCatalogFound = JAXP09030003: No Catalog is specified. Thanks, Leo On 01/13/2018 03:40 AM, huizhe wang wrote: > Hi, > > Please review a doc-only change that adds a few notes to message files that serves as a notification to translation > teams that some technical terms shall not be translated. > > JBS: > https://bugs.openjdk.java.net/browse/JDK-8181047 > webres: > http://cr.openjdk.java.net/~joehw/jdk10/8181047/webrev/ > > Thanks, > Joe From vicente.romero at oracle.com Sat Jan 13 02:15:14 2018 From: vicente.romero at oracle.com (Vicente Romero) Date: Fri, 12 Jan 2018 21:15:14 -0500 Subject: RFR: 8195072: Update ASM 3rd party legal (TPL) copyright to 6.0 In-Reply-To: <5A59535A.1060701@oracle.com> References: <5A59535A.1060701@oracle.com> Message-ID: <6020e60d-b1c8-7dcf-9bdd-f1fa72953665@oracle.com> looks good, Vicente On 01/12/2018 07:31 PM, Kumar Srinivasan wrote: > Hello, > > Could I get a review for this simple update to the 3rd party legal > (TPL) information. > > Thanks > > Webrev: http://cr.openjdk.java.net/~ksrini/8195072/webrev.00/ > JBS: https://bugs.openjdk.java.net/browse/JDK-8195072 > Current ASM copyright: > http://hg.openjdk.java.net/jdk/jdk10/file/c674ff28c69d/src/java.base/share/classes/jdk/internal/org/objectweb/asm/AnnotationVisitor.java > Current ASM version: > http://hg.openjdk.java.net/jdk/jdk10/file/c674ff28c69d/src/java.base/share/classes/jdk/internal/org/objectweb/asm/version.txt From mandy.chung at oracle.com Sat Jan 13 19:55:32 2018 From: mandy.chung at oracle.com (mandy chung) Date: Sat, 13 Jan 2018 11:55:32 -0800 Subject: [10] RFR: JDK-8191350: jdk/internal/reflect/CallerSensitive/CheckCSMs.java test fails when -Xmx512m set Message-ID: <347fb325-95cb-9e07-e346-856056ac8e9f@oracle.com> Test-only fix. The CheckCSMs.java and CallerSensitiveFinder.java tests fail with OOME when running with -Xmx512m.? The test intends to parse class files, one per task in a thread pool but each task keeps a ClassFile that has done the parsing and occupies the space.? This patch fixes the tests to delay reading of the class file when the task is scheduled (i.e. the task takes the path to the class file instead of ClassFile instance). Webrev at: http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8191350/webrev.00/ Mandy From Alan.Bateman at oracle.com Sun Jan 14 09:54:26 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Sun, 14 Jan 2018 09:54:26 +0000 Subject: [10] RFR: JDK-8191350: jdk/internal/reflect/CallerSensitive/CheckCSMs.java test fails when -Xmx512m set In-Reply-To: <347fb325-95cb-9e07-e346-856056ac8e9f@oracle.com> References: <347fb325-95cb-9e07-e346-856056ac8e9f@oracle.com> Message-ID: On 13/01/2018 19:55, mandy chung wrote: > Test-only fix. > > The CheckCSMs.java and CallerSensitiveFinder.java tests fail with OOME > when running with -Xmx512m.? The test intends to parse class files, one > per task in a thread pool but each task keeps a ClassFile that has done > the parsing and occupies the space.? This patch fixes the tests to delay > reading of the class file when the task is scheduled (i.e. the task > takes the path to the class file instead of ClassFile instance). > > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk10/webrevs/8191350/webrev.00/ This looks okay to me. -Alan From matthias.baesken at sap.com Mon Jan 15 08:03:17 2018 From: matthias.baesken at sap.com (Baesken, Matthias) Date: Mon, 15 Jan 2018 08:03:17 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: References: Message-ID: Hi Goetz, change looks good except test/jdk/sun/security/tools/keytool/i18n.sh where the Cygwin PS setting has been changed from ; to : (probably this was not intended) : PS=":" FS="/" ;; - CYGWIN* ) - NULL=/dev/null - PS=";" (not a Reviewer however). Best regards, Matthias > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Mittwoch, 10. Januar 2018 09:23 > To: core-libs-dev at openjdk.java.net > Cc: Baesken, Matthias > Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > > Hi, > > this change adapts a row of jtreg tests to AIX. > Shell tests often switch over all OSes, here the case for AIX was > missing. PKCSTest needs to know where the libs are on s390. > > Please review. > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ > > Best regards, > Goetz. From goetz.lindenmaier at sap.com Mon Jan 15 11:12:52 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Mon, 15 Jan 2018 11:12:52 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: References: Message-ID: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> Hi Matthias, I didin't mean to do any changes for other platforms, so I reverted the part you pointed out: http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ But scanning through all the scripts: Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... Best regards, Goetz. > -----Original Message----- > From: Baesken, Matthias > Sent: Montag, 15. Januar 2018 09:03 > To: Lindenmaier, Goetz ; core-libs- > dev at openjdk.java.net > Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > > Hi Goetz, change looks good except > > test/jdk/sun/security/tools/keytool/i18n.sh > > where the Cygwin PS setting has been changed from ; to : (probably this > was not intended) : > > PS=":" > FS="/" > ;; > - CYGWIN* ) > - NULL=/dev/null > - PS=";" > > > (not a Reviewer however). > > Best regards, Matthias > > > > > -----Original Message----- > > From: Lindenmaier, Goetz > > Sent: Mittwoch, 10. Januar 2018 09:23 > > To: core-libs-dev at openjdk.java.net > > Cc: Baesken, Matthias > > Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > > > > Hi, > > > > this change adapts a row of jtreg tests to AIX. > > Shell tests often switch over all OSes, here the case for AIX was > > missing. PKCSTest needs to know where the libs are on s390. > > > > Please review. > > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ > > > > Best regards, > > Goetz. From vladimir.x.ivanov at oracle.com Mon Jan 15 12:00:31 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 15 Jan 2018 15:00:31 +0300 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <0DAD4412-5E7E-4992-92CF-27CD08F9E237@redhat.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> <0DAD4412-5E7E-4992-92CF-27CD08F9E237@redhat.com> Message-ID: <3f26db3f-24a9-97e8-b205-37e6c3c13acf@oracle.com> > This isn?t much different from the already allowed fine tuning of memory model effects (e.g., observing stale values from a plain write). In those scenarios, as is the case with @Stable, developers have to be aware of and honor the respective contract. It could be argued that bugs arising from contract violations of @Stable are easier to catch, due to their persistent nature. The crucial difference is that @Stable doesn't obey JMM while others do (though you can break that as well using Unsafe or Reflection API), so all the tricks programmers rely on (locks, finals, volatiles, fences) don't work. Considering the contract is not enforced at runtime (in current implementation), it's easy to violate it (especially with concurrent accesses). And, as practice shows, it's very hard to track down bugs causing change in behavior between different execution modes (interpreter vs compiled code vs recompiled code). To sum it up, standardizing something which increases complexity of the area which is already quite complicated (JMM) doesn't look like a wise move in a long term. Best regards, Vladimir Ivanov From vladimir.x.ivanov at oracle.com Mon Jan 15 12:10:46 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 15 Jan 2018 15:10:46 +0300 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: > > No, it doesn't work that way. @Stable enables JIT to constant fold > loads from fields/arrays (holding non-default values) whenever > possible. After that JIT has a constant value at its hands. All > other optimizations follows from that: range check before array > store can be elided since array instance is known and its length is > known as well, type check can be elided since constant type is > known. But the value won't be reloaded from memory, all > optimizations happen on a constant which was loaded from memory once > during compilation. > > So, the worst case scenario is: a value written into @Stable > field/array? is never visible in some code no matter what you do. It > can lead to nasty bugs when different parts of program don't agree > on observed value. It can happen when user doesn't obey @Stable > contract and performs multiple writes into a @Stable field/array. > Current implementation in HotSpot doesn't forbid that. > > Thanks for clarifying Vladimir - that makes sense.? So back to the > integrity of the VM question - what can be compromised then? I suppose > if one was to use Unsafe access in combination with relying on a @Stable > field to feed it info (e.g. index into some memory accessed via Unsafe), > then things can go south.? But Unsafe is already, well, unsafe.? What > extra VM integrity issues would a misbehaving @Stable cause that Andrew > was alluding to? I don't know what exactly Andrew had in mind. Though misuse of @Stable can't be a direct cause of JVM crashes, it can lead to serious bugs in user code which manifest themselves as JIT-compiler bugs. And miscompiled code silently corrupting user data is usually considered much more serious issue than a JVM crash. Best regards, Vladimir Ivanov From vladimir.x.ivanov at oracle.com Mon Jan 15 12:51:03 2018 From: vladimir.x.ivanov at oracle.com (Vladimir Ivanov) Date: Mon, 15 Jan 2018 15:51:03 +0300 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> Message-ID: <15e9c849-fc17-9bdc-49a0-05830f9030e8@oracle.com> Jason, I wouldn't consider @Stable as a scalpel :-) It's heavily locked down for a reason: it wasn't designed for eventual standardization, but as (1) an experiment in _multiple_ directions; and (2) solution for immediate problems in java.lang.invoke. @Stable mixes (incomplete implementations of) 2 independent features (lazy initialization of finals + constant arrays), while trusting instance finals is just a consequence of locking it down to privileged code (JIT-compilers already trust final instance fields in different core packages). Current implementation doesn't enforce at runtime the contract and allows multiple writes to @Stable field. It is useful in rare cases, but overall significantly complicates proper usage of @Stable. All those factors make @Stable unfit for public exposure in it's current form. Both referenced features are independently explored (as lazy finals and frozen arrays). Also, @Stable is not a substitute for final fields: default value is treated specially (not optimized), while "final" covers the whole range of values of a type. For optimizing final fields there are much more promising approaches: (1) optimistic optimizations in JITs: treat finals as constants and track updates invalidating code which relies on that (there were some experiments, e.g. [1]); (2) forbid final field updates at runtime after initialization is over and freely optimize them in JITs. Both approaches still have some roadblocks on their way (deserialization relies on modifying final fields, hard to track final field values of individual objects in JVM, etc), but the platform steadily moves in the direction of treating final fields as truly final by default. Best regards, Vladimir Ivanov [1] http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2015-June/018342.html On 1/12/18 7:33 AM, Jason Greene wrote: > MethodHandle.invokeExact() can match the performance of direct invocation, but it requires constant folding to do so. Otherwise results are similar to Java reflection(e.g [1]). > > While TrustFinalNonStaticFields can be used, it?s a sledgehammer where a scalpel is really be more appropriate. > > The internal @Stable facility provides the desired semantics and precision, but it is heavily locked down, with privileged code checks and export restrictions. Could this be made more accessible (or perhaps a variant restricted to just final fields)? Informing the compiler that a final field is a true lazy initialized constant, with no store-to-final seems a pretty useful construct in general. I can understand that the long term desire is that this shouldn?t be necessary, and should be inferred [3], but at that point the annotation is still useful as documentation and legacy compatibility. If nothing else could it be allowed in non-privileged code via some flag? > It seems odd that the much more aggressive facility (TrustFinalNonStaticFields) is simpler to use than a more targeted one. > > Thanks! > > [1] https://www.optaplanner.org/blog/2018/01/09/JavaReflectionButMuchFaster.html > [2] http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/65464a307408/src/java.base/share/classes/jdk/internal/vm/annotation/Stable.java > [3] https://bugs.openjdk.java.net/browse/JDK-8058164 > From jason.greene at redhat.com Mon Jan 15 23:38:02 2018 From: jason.greene at redhat.com (Jason Greene) Date: Mon, 15 Jan 2018 17:38:02 -0600 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <15e9c849-fc17-9bdc-49a0-05830f9030e8@oracle.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> <15e9c849-fc17-9bdc-49a0-05830f9030e8@oracle.com> Message-ID: > On Jan 15, 2018, at 6:51 AM, Vladimir Ivanov wrote: > > Jason, > > I wouldn't consider @Stable as a scalpel :-) > > It's heavily locked down for a reason: it wasn't designed for eventual standardization, but as (1) an experiment in _multiple_ directions; and (2) solution for immediate problems in java.lang.invoke. > > @Stable mixes (incomplete implementations of) 2 independent features (lazy initialization of finals + constant arrays), while trusting instance finals is just a consequence of locking it down to privileged code (JIT-compilers already trust final instance fields in different core packages). > > Current implementation doesn't enforce at runtime the contract and allows multiple writes to @Stable field. It is useful in rare cases, but overall significantly complicates proper usage of @Stable. > > All those factors make @Stable unfit for public exposure in it?s current form. > > Both referenced features are independently explored (as lazy finals and frozen arrays). > > Also, @Stable is not a substitute for final fields: default value is treated specially (not optimized), while "final" covers the whole range of values of a type. > > For optimizing final fields there are much more promising approaches: > > (1) optimistic optimizations in JITs: treat finals as constants and track updates invalidating code which relies on that (there were some experiments, e.g. [1]); > > (2) forbid final field updates at runtime after initialization is over and freely optimize them in JITs. > > Both approaches still have some roadblocks on their way (deserialization relies on modifying final fields, hard to track final field values of individual objects in JVM, etc), but the platform steadily moves in the direction of treating final fields as truly final by default. Thanks for the explanation Vladimir, I can certainly understand optimistic optimization being the end goal, but, correct me if I am wrong, it seems an improvement in the distant future. I can see the issue is scheduled for 11, but I imagine that is more of a placeholder. Has a narrow variant of (2) in the same vein as @Stable been considered, as it seems to overcome the roadblocks mentioned? An annotation placed on a final field (something like @Constant, @StrictFinal, @ImmutableAfterPublish, whatever) could indicate opting in to the contract of this field as a truly final field, freeing the JVM to optimize and validate as desired (e.g. special case serialization, preventing setAcessible/JNI set field/VarHandle/etc), and without introducing compatibility issues. I realize validating access via Unsafe is non-trivial, but I argue there is limited value in doing so, since the effects are not worse than other actions one might take with Unsafe. Further, I argue any validation at all is going above and beyond, as ?undefined? is a perfectly reasonable result for contract violations. -Jason From jason.greene at redhat.com Tue Jan 16 00:09:39 2018 From: jason.greene at redhat.com (Jason Greene) Date: Mon, 15 Jan 2018 18:09:39 -0600 Subject: Can @Stable (or something similar) be made accessible? In-Reply-To: <3f26db3f-24a9-97e8-b205-37e6c3c13acf@oracle.com> References: <4CEBC6E6-03E9-46AF-A5ED-7C083B7B70D9@redhat.com> <0DAD4412-5E7E-4992-92CF-27CD08F9E237@redhat.com> <3f26db3f-24a9-97e8-b205-37e6c3c13acf@oracle.com> Message-ID: > On Jan 15, 2018, at 6:00 AM, Vladimir Ivanov wrote: > > >> This isn?t much different from the already allowed fine tuning of memory model effects (e.g., observing stale values from a plain write). In those scenarios, as is the case with @Stable, developers have to be aware of and honor the respective contract. It could be argued that bugs arising from contract violations of @Stable are easier to catch, due to their persistent nature. > > The crucial difference is that @Stable doesn?t obey JMM while others do (though you can break that as well using Unsafe or Reflection API), so all the tricks programmers rely on (locks, finals, volatiles, fences) don't work. Right and also VarHandle and AtomicReferenceFieldUpdater, which let you weaken the effect of a write, such that stale reads on the reading side may occur. -Jason From huizhe.wang at oracle.com Tue Jan 16 05:55:36 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Mon, 15 Jan 2018 21:55:36 -0800 Subject: RFR (JDK10/JAXP Doc-only) 8181047: Add comment to technical terms that shall not be translated In-Reply-To: <22095d26-a0fa-17e6-1b96-272054a2da82@oracle.com> References: <56467db4-88b1-c19e-7a7d-6c6b033b3995@oracle.com> <22095d26-a0fa-17e6-1b96-272054a2da82@oracle.com> Message-ID: Added, same for the line that has "href and base", although that seemed to be the norm for the translation team already (I mean they did not translate those terms). But just for the completeness, I added those comments as well. Thanks, Joe On 1/12/2018 5:03 PM, Leo Jiang wrote: > Maybe you should also add a comment for this line: > > ?NoCatalogFound = JAXP09030003: No Catalog is specified. > > Thanks, > Leo > > On 01/13/2018 03:40 AM, huizhe wang wrote: >> Hi, >> >> Please review a doc-only change that adds a few notes to message >> files that serves as a notification to translation teams that some >> technical terms shall not be translated. >> >> JBS: >> https://bugs.openjdk.java.net/browse/JDK-8181047 >> webres: >> http://cr.openjdk.java.net/~joehw/jdk10/8181047/webrev/ >> >> Thanks, >> Joe From goetz.lindenmaier at sap.com Tue Jan 16 07:47:41 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 16 Jan 2018 07:47:41 +0000 Subject: RFR: 8189102: All tools should support -?, -h and --help Message-ID: Hi, The CSR for this change is approved now. https://bugs.openjdk.java.net/browse/JDK-8191477 If there aren't any more objections, I will push this. Webrev rebased and with updated 2018 copyright messages: http://cr.openjdk.java.net/~goetz/wr17/8189102-helpMessage/webrev.08/ Best regards, Goetz. From vipinmv1 at in.ibm.com Tue Jan 16 12:21:38 2018 From: vipinmv1 at in.ibm.com (Vipin Mv1) Date: Tue, 16 Jan 2018 17:51:38 +0530 Subject: Clarification for adding classes shipped with JDK to sun.rmi.registry.registryFilter property in java.security file Message-ID: Hi, After upgrading to oracle 8u121-b12, I get following Exception when running the following testcase. import java.io.*; import java.net.InetAddress; import java.rmi.*; import java.rmi.registry.*; import java.security.Security; /* @test * @run main/othervm RegistryFilterTest */ public class RegistryFilterTest { private static int port = 12345; private static Registry registry; static class RemoteObj implements Serializable, Remote { private static final long serialVersionUID = 01L; final Object obj; RemoteObj(Object obj) { this.obj = obj; } } public static void main(String[] args) throws Exception{ InetAddress in = InetAddress.getLocalHost(); LocateRegistry.createRegistry(port); Registry registry = LocateRegistry.getRegistry("localhost", port); registry.bind("InetAddress", new RemoteObj(in)); registry.unbind("InetAddress"); System.out.println("RMI Registry Test Passed"); } } TestResults ----------------------- Jan 12, 2018 5:32:18 PM java.io.ObjectInputStream filterCheck INFO: ObjectInputFilter REJECTED: class java.net.InetAddress, array length: -1, nRefs: 5, depth: 2, bytes: 216, ex: n/a Caused by: java.io.InvalidClassException: filter status: REJECTED ?? ?at java.io.ObjectInputStream.filterCheck(ObjectInputStream.java:1406) ?? ?at java.io.ObjectInputStream.readNonProxyDesc (ObjectInputStream.java:1997) ?? ?at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1877) ?? ?at java.io.ObjectInputStream.readOrdinaryObject (ObjectInputStream.java:2170) ?? ?at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) ?? ?at java.io.ObjectInputStream.defaultReadFields (ObjectInputStream.java:2415) ?? ?at java.io.ObjectInputStream.readSerialData (ObjectInputStream.java:2339) ?? ?at java.io.ObjectInputStream.readOrdinaryObject (ObjectInputStream.java:2197) ?? ?at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) ?? ?at java.io.ObjectInputStream.readObjectImpl(ObjectInputStream.java:540) ?? ?at java.io.ObjectInputStream.readObject(ObjectInputStream.java:475) ?? ?... 16 more I understand it is due to the new serialization Filtering mechanism introduced to RMI Registry and Distributed Garbage Collection as a part of the following fixes. JDK-8160108 Implement Serialization Filtering JDK-8156804: Better constraint checking(not public) As a result, a new default white list has been introduced for RMIRegistry which allows deserialization only to limited set of classes by default. Which are String.class java.lang.Number.class java.rmi.Remote.class java.lang.reflect.Proxy.class sun.rmi.server.UnicastRef.class java.rmi.server.RMIClientSocketFactory.class java.rmi.server.RMIServerSocketFactory.class java.rmi.activation.ActivationID.class java.rmi.server.UID.class and their subclasses. It also enabled provisions to increase the default white list by 1. updating property "sun.rmi.registry.registryFilter" in java.security file 2. adding -Dsun.rmi.registry.registryFilter in java command line Though the current solution is straight forward, it is difficult from an application point of view to identify such classes without running the application multiple times to come up with comprehensive list to be allowed. This would be a overhead to the developer and should be avoided at least for classes shipped with JDK (for example, java.net.InetAddress as used in testcase). I assume that default list is very limited to reduce the chances of getting exploited by serialization vulnerability. I think, we can spare public packages shipped with JDK by adding them to "sun.rmi.registry.registryFilter" in java.security file out of the box. Please let me know your views. Thanks & Regards, Vipin MV From volker.simonis at gmail.com Tue Jan 16 18:28:07 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Tue, 16 Jan 2018 19:28:07 +0100 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> Message-ID: Hi Goetz, you change looks good. Just one minor nit (no need for a new webrev if you agree with my fix :) In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as file. This is a useless check because there won't be a file "mktemp" in the current exectution directory of the shell script, even if the "mktemp" exists domehwere in the PATH. To check for the existence of a "mktemp" executable you can use the "type" command: type mktempp> /dev/null 2>&1 if ! [ $? -ne 0 ] ; then MKTEMP="/opt/freeware/bin/mktemp" fi Thanks for finally fixing this, Volker On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz wrote: > Hi Matthias, > > I didin't mean to do any changes for other platforms, so > I reverted the part you pointed out: > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ > > But scanning through all the scripts: > Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... > > Best regards, > Goetz. > >> -----Original Message----- >> From: Baesken, Matthias >> Sent: Montag, 15. Januar 2018 09:03 >> To: Lindenmaier, Goetz ; core-libs- >> dev at openjdk.java.net >> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. >> >> Hi Goetz, change looks good except >> >> test/jdk/sun/security/tools/keytool/i18n.sh >> >> where the Cygwin PS setting has been changed from ; to : (probably this >> was not intended) : >> >> PS=":" >> FS="/" >> ;; >> - CYGWIN* ) >> - NULL=/dev/null >> - PS=";" >> >> >> (not a Reviewer however). >> >> Best regards, Matthias >> >> >> >> > -----Original Message----- >> > From: Lindenmaier, Goetz >> > Sent: Mittwoch, 10. Januar 2018 09:23 >> > To: core-libs-dev at openjdk.java.net >> > Cc: Baesken, Matthias >> > Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. >> > >> > Hi, >> > >> > this change adapts a row of jtreg tests to AIX. >> > Shell tests often switch over all OSes, here the case for AIX was >> > missing. PKCSTest needs to know where the libs are on s390. >> > >> > Please review. >> > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ >> > >> > Best regards, >> > Goetz. From brian.burkhalter at oracle.com Tue Jan 16 19:17:31 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 16 Jan 2018 11:17:31 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) Message-ID: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> https://bugs.openjdk.java.net/browse/JDK-8139206 http://cr.openjdk.java.net/~bpb/8139206/webrev.00/ This change would add a new method ?byte[] InputStream.readNBytes(int len)? which would read up to at most ?len? bytes from the stream and return them in an internally allocated array. Thanks, Brian From brent.christian at oracle.com Tue Jan 16 21:21:45 2018 From: brent.christian at oracle.com (Brent Christian) Date: Tue, 16 Jan 2018 13:21:45 -0800 Subject: [11] RFR 8194879 : Runtime.Version parses string which does not conform to spec without throwing IAE Message-ID: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> Hi, Please review this version string parsing fix for Runtime.Version. Issue: https://bugs.openjdk.java.net/browse/JDK-8194879 Webrev: http://cr.openjdk.java.net/~bchristi/8194879/webrev.00/index.html From the bug: -- "As per the specification of Runtime.Version: A version string, $VSTR, is a version number $VNUM, as described above, optionally followed by pre-release and build information, in one of the following formats: 1. $VNUM(-$PRE)?\+$BUILD(-$OPT)? 2. $VNUM-$PRE(-$OPT)? 3. $VNUM(+-$OPT)? So an example of "10--ea" does not confirm to any of the above regex's." -- For "10--ea", the leading '-' gets picked up, and OPT (which allows the '-' character) gets set to "-ea". So the pattern matching of the elements appears to happen as intended. A case can be added in the parse() code to disallow this (case #2 with an OPT present, but no PRE). Thanks, -Brent From iris.clark at oracle.com Wed Jan 17 02:08:15 2018 From: iris.clark at oracle.com (Iris Clark) Date: Tue, 16 Jan 2018 18:08:15 -0800 (PST) Subject: [11] RFR 8194879 : Runtime.Version parses string which does not conform to spec without throwing IAE In-Reply-To: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> References: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> Message-ID: Hi, Brent. > Issue: > https://bugs.openjdk.java.net/browse/JDK-8194879 > Webrev: > http://cr.openjdk.java.net/~bchristi/8194879/webrev.00/index.html Your changes look good to me. Thanks, Iris From goetz.lindenmaier at sap.com Wed Jan 17 11:39:06 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 17 Jan 2018 11:39:06 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> Message-ID: <9668a567570d4baa82a04532ee1db52f@sap.com> Hi, Volker, thanks for your review. Could someone else please have a quick look at this simple fix? Updated webrev with fixed BadPluginConfigurationTest: http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ Best regards, Goetz. > -----Original Message----- > From: Volker Simonis [mailto:volker.simonis at gmail.com] > Sent: Dienstag, 16. Januar 2018 19:28 > To: Lindenmaier, Goetz > Cc: Baesken, Matthias ; core-libs- > dev at openjdk.java.net > Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > > Hi Goetz, > > you change looks good. Just one minor nit (no need for a new webrev if > you agree with my fix :) > > In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as > file. This is a useless check because there won't be a file "mktemp" > in the current exectution directory of the shell script, even if the > "mktemp" exists domehwere in the PATH. To check for the existence of a > "mktemp" executable you can use the "type" command: > > type mktempp> /dev/null 2>&1 > > if ! [ $? -ne 0 ] ; then > MKTEMP="/opt/freeware/bin/mktemp" > fi > > Thanks for finally fixing this, > Volker > > > On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz > wrote: > > Hi Matthias, > > > > I didin't mean to do any changes for other platforms, so > > I reverted the part you pointed out: > > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ > > > > But scanning through all the scripts: > > Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Baesken, Matthias > >> Sent: Montag, 15. Januar 2018 09:03 > >> To: Lindenmaier, Goetz ; core-libs- > >> dev at openjdk.java.net > >> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > >> > >> Hi Goetz, change looks good except > >> > >> test/jdk/sun/security/tools/keytool/i18n.sh > >> > >> where the Cygwin PS setting has been changed from ; to : (probably this > >> was not intended) : > >> > >> PS=":" > >> FS="/" > >> ;; > >> - CYGWIN* ) > >> - NULL=/dev/null > >> - PS=";" > >> > >> > >> (not a Reviewer however). > >> > >> Best regards, Matthias > >> > >> > >> > >> > -----Original Message----- > >> > From: Lindenmaier, Goetz > >> > Sent: Mittwoch, 10. Januar 2018 09:23 > >> > To: core-libs-dev at openjdk.java.net > >> > Cc: Baesken, Matthias > >> > Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > >> > > >> > Hi, > >> > > >> > this change adapts a row of jtreg tests to AIX. > >> > Shell tests often switch over all OSes, here the case for AIX was > >> > missing. PKCSTest needs to know where the libs are on s390. > >> > > >> > Please review. > >> > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ > >> > > >> > Best regards, > >> > Goetz. From david.holmes at oracle.com Wed Jan 17 11:55:01 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 17 Jan 2018 21:55:01 +1000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <9668a567570d4baa82a04532ee1db52f@sap.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> Message-ID: <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> Hi Goetz, test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java I thought this was for AIX recognition not OS/400 ? ;-) Otherwise the AIX changes seem okay. Thought AWT tests should be reviewed on awt-dev. Cheers, David On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: > Hi, > > Volker, thanks for your review. Could someone else please have a > quick look at this simple fix? > > Updated webrev with fixed BadPluginConfigurationTest: > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > > Best regards, > Goetz. > >> -----Original Message----- >> From: Volker Simonis [mailto:volker.simonis at gmail.com] >> Sent: Dienstag, 16. Januar 2018 19:28 >> To: Lindenmaier, Goetz >> Cc: Baesken, Matthias ; core-libs- >> dev at openjdk.java.net >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. >> >> Hi Goetz, >> >> you change looks good. Just one minor nit (no need for a new webrev if >> you agree with my fix :) >> >> In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as >> file. This is a useless check because there won't be a file "mktemp" >> in the current exectution directory of the shell script, even if the >> "mktemp" exists domehwere in the PATH. To check for the existence of a >> "mktemp" executable you can use the "type" command: >> >> type mktempp> /dev/null 2>&1 >> >> if ! [ $? -ne 0 ] ; then >> MKTEMP="/opt/freeware/bin/mktemp" >> fi >> >> Thanks for finally fixing this, >> Volker >> >> >> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz >> wrote: >>> Hi Matthias, >>> >>> I didin't mean to do any changes for other platforms, so >>> I reverted the part you pointed out: >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ >>> >>> But scanning through all the scripts: >>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... >>> >>> Best regards, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Baesken, Matthias >>>> Sent: Montag, 15. Januar 2018 09:03 >>>> To: Lindenmaier, Goetz ; core-libs- >>>> dev at openjdk.java.net >>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >> platforms. >>>> >>>> Hi Goetz, change looks good except >>>> >>>> test/jdk/sun/security/tools/keytool/i18n.sh >>>> >>>> where the Cygwin PS setting has been changed from ; to : (probably this >>>> was not intended) : >>>> >>>> PS=":" >>>> FS="/" >>>> ;; >>>> - CYGWIN* ) >>>> - NULL=/dev/null >>>> - PS=";" >>>> >>>> >>>> (not a Reviewer however). >>>> >>>> Best regards, Matthias >>>> >>>> >>>> >>>>> -----Original Message----- >>>>> From: Lindenmaier, Goetz >>>>> Sent: Mittwoch, 10. Januar 2018 09:23 >>>>> To: core-libs-dev at openjdk.java.net >>>>> Cc: Baesken, Matthias >>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. >>>>> >>>>> Hi, >>>>> >>>>> this change adapts a row of jtreg tests to AIX. >>>>> Shell tests often switch over all OSes, here the case for AIX was >>>>> missing. PKCSTest needs to know where the libs are on s390. >>>>> >>>>> Please review. >>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ >>>>> >>>>> Best regards, >>>>> Goetz. From goetz.lindenmaier at sap.com Wed Jan 17 12:42:08 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 17 Jan 2018 12:42:08 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> Message-ID: <5074508851404c77ae85ee86035866b5@sap.com> Hi David, thanks for looking at the change. We actually run the aix VM also on OS400. And as OS.java says that it's a " Stripped down version of jtreg internal class com.sun.javatest.regtest.config.OS" I not only grabbed the AIX case from there, but also the OS400 one. I don't think this will do any harm. I'll post it on awt-dev, too. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 17. Januar 2018 12:55 > To: Lindenmaier, Goetz ; Volker Simonis > > Cc: core-libs-dev at openjdk.java.net; Baesken, Matthias > > Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > > Hi Goetz, > > test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java > > I thought this was for AIX recognition not OS/400 ? ;-) > > Otherwise the AIX changes seem okay. Thought AWT tests should be > reviewed on awt-dev. > > Cheers, > David > > On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > Volker, thanks for your review. Could someone else please have a > > quick look at this simple fix? > > > > Updated webrev with fixed BadPluginConfigurationTest: > > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Volker Simonis [mailto:volker.simonis at gmail.com] > >> Sent: Dienstag, 16. Januar 2018 19:28 > >> To: Lindenmaier, Goetz > >> Cc: Baesken, Matthias ; core-libs- > >> dev at openjdk.java.net > >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > >> > >> Hi Goetz, > >> > >> you change looks good. Just one minor nit (no need for a new webrev if > >> you agree with my fix :) > >> > >> In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as > >> file. This is a useless check because there won't be a file "mktemp" > >> in the current exectution directory of the shell script, even if the > >> "mktemp" exists domehwere in the PATH. To check for the existence of a > >> "mktemp" executable you can use the "type" command: > >> > >> type mktempp> /dev/null 2>&1 > >> > >> if ! [ $? -ne 0 ] ; then > >> MKTEMP="/opt/freeware/bin/mktemp" > >> fi > >> > >> Thanks for finally fixing this, > >> Volker > >> > >> > >> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz > >> wrote: > >>> Hi Matthias, > >>> > >>> I didin't mean to do any changes for other platforms, so > >>> I reverted the part you pointed out: > >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ > >>> > >>> But scanning through all the scripts: > >>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... > >>> > >>> Best regards, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Baesken, Matthias > >>>> Sent: Montag, 15. Januar 2018 09:03 > >>>> To: Lindenmaier, Goetz ; core-libs- > >>>> dev at openjdk.java.net > >>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>> > >>>> Hi Goetz, change looks good except > >>>> > >>>> test/jdk/sun/security/tools/keytool/i18n.sh > >>>> > >>>> where the Cygwin PS setting has been changed from ; to : (probably > this > >>>> was not intended) : > >>>> > >>>> PS=":" > >>>> FS="/" > >>>> ;; > >>>> - CYGWIN* ) > >>>> - NULL=/dev/null > >>>> - PS=";" > >>>> > >>>> > >>>> (not a Reviewer however). > >>>> > >>>> Best regards, Matthias > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Lindenmaier, Goetz > >>>>> Sent: Mittwoch, 10. Januar 2018 09:23 > >>>>> To: core-libs-dev at openjdk.java.net > >>>>> Cc: Baesken, Matthias > >>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > >>>>> > >>>>> Hi, > >>>>> > >>>>> this change adapts a row of jtreg tests to AIX. > >>>>> Shell tests often switch over all OSes, here the case for AIX was > >>>>> missing. PKCSTest needs to know where the libs are on s390. > >>>>> > >>>>> Please review. > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ > >>>>> > >>>>> Best regards, > >>>>> Goetz. From goetz.lindenmaier at sap.com Wed Jan 17 12:43:36 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Wed, 17 Jan 2018 12:43:36 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> Message-ID: <1290763f577847109b7a49f3f2621d09@sap.com> Posting this to awt-dev, too. This change adapts some tests to run on properly on aix or s390: http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Mittwoch, 17. Januar 2018 12:55 > To: Lindenmaier, Goetz ; Volker Simonis > > Cc: core-libs-dev at openjdk.java.net; Baesken, Matthias > > Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > > Hi Goetz, > > test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java > > I thought this was for AIX recognition not OS/400 ? ;-) > > Otherwise the AIX changes seem okay. Thought AWT tests should be > reviewed on awt-dev. > > Cheers, > David > > On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > Volker, thanks for your review. Could someone else please have a > > quick look at this simple fix? > > > > Updated webrev with fixed BadPluginConfigurationTest: > > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: Volker Simonis [mailto:volker.simonis at gmail.com] > >> Sent: Dienstag, 16. Januar 2018 19:28 > >> To: Lindenmaier, Goetz > >> Cc: Baesken, Matthias ; core-libs- > >> dev at openjdk.java.net > >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > >> > >> Hi Goetz, > >> > >> you change looks good. Just one minor nit (no need for a new webrev if > >> you agree with my fix :) > >> > >> In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as > >> file. This is a useless check because there won't be a file "mktemp" > >> in the current exectution directory of the shell script, even if the > >> "mktemp" exists domehwere in the PATH. To check for the existence of a > >> "mktemp" executable you can use the "type" command: > >> > >> type mktempp> /dev/null 2>&1 > >> > >> if ! [ $? -ne 0 ] ; then > >> MKTEMP="/opt/freeware/bin/mktemp" > >> fi > >> > >> Thanks for finally fixing this, > >> Volker > >> > >> > >> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz > >> wrote: > >>> Hi Matthias, > >>> > >>> I didin't mean to do any changes for other platforms, so > >>> I reverted the part you pointed out: > >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ > >>> > >>> But scanning through all the scripts: > >>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... > >>> > >>> Best regards, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Baesken, Matthias > >>>> Sent: Montag, 15. Januar 2018 09:03 > >>>> To: Lindenmaier, Goetz ; core-libs- > >>>> dev at openjdk.java.net > >>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>> > >>>> Hi Goetz, change looks good except > >>>> > >>>> test/jdk/sun/security/tools/keytool/i18n.sh > >>>> > >>>> where the Cygwin PS setting has been changed from ; to : (probably > this > >>>> was not intended) : > >>>> > >>>> PS=":" > >>>> FS="/" > >>>> ;; > >>>> - CYGWIN* ) > >>>> - NULL=/dev/null > >>>> - PS=";" > >>>> > >>>> > >>>> (not a Reviewer however). > >>>> > >>>> Best regards, Matthias > >>>> > >>>> > >>>> > >>>>> -----Original Message----- > >>>>> From: Lindenmaier, Goetz > >>>>> Sent: Mittwoch, 10. Januar 2018 09:23 > >>>>> To: core-libs-dev at openjdk.java.net > >>>>> Cc: Baesken, Matthias > >>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > >>>>> > >>>>> Hi, > >>>>> > >>>>> this change adapts a row of jtreg tests to AIX. > >>>>> Shell tests often switch over all OSes, here the case for AIX was > >>>>> missing. PKCSTest needs to know where the libs are on s390. > >>>>> > >>>>> Please review. > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ > >>>>> > >>>>> Best regards, > >>>>> Goetz. From Roger.Riggs at Oracle.com Wed Jan 17 15:04:56 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 17 Jan 2018 10:04:56 -0500 Subject: [11] RFR 8194879 : Runtime.Version parses string which does not conform to spec without throwing IAE In-Reply-To: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> References: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> Message-ID: Hi Brent, The bug number should be added to the @bug line in the test. Thanks, Roger On 1/16/2018 4:21 PM, Brent Christian wrote: > Hi, > > Please review this version string parsing fix for Runtime.Version. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8194879 > Webrev: > http://cr.openjdk.java.net/~bchristi/8194879/webrev.00/index.html > > From the bug: > -- > "As per the specification of Runtime.Version: > > A version string, $VSTR, is a version number $VNUM, as described > above, optionally followed by pre-release and build information, in > one of the following formats: > > ?????? 1. $VNUM(-$PRE)?\+$BUILD(-$OPT)? > ?????? 2. $VNUM-$PRE(-$OPT)? > ?????? 3. $VNUM(+-$OPT)? > > So an example of "10--ea" does not confirm to any of the above regex's." > -- > > For "10--ea", the leading '-' gets picked up, and OPT (which allows > the '-' character) gets set to "-ea".? So the pattern matching of the > elements appears to happen as intended. > > A case can be added in the parse() code to disallow this (case #2 with > an OPT present, but no PRE). > > Thanks, > -Brent From Alan.Bateman at oracle.com Wed Jan 17 15:34:11 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 17 Jan 2018 15:34:11 +0000 Subject: [11] RFR 8194879 : Runtime.Version parses string which does not conform to spec without throwing IAE In-Reply-To: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> References: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> Message-ID: <75540e7e-7066-6daf-ffe9-d1206db22ab3@oracle.com> On 16/01/2018 21:21, Brent Christian wrote: > Hi, > > Please review this version string parsing fix for Runtime.Version. > > Issue: > https://bugs.openjdk.java.net/browse/JDK-8194879 > Webrev: > http://cr.openjdk.java.net/~bchristi/8194879/webrev.00/index.html Looks good to me. From brian.burkhalter at oracle.com Wed Jan 17 16:24:19 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 17 Jan 2018 08:24:19 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> Message-ID: The proposed change has been modified to replace the two methods byte[] InputStream.readAllBytes(int) // reads at most ?len? bytes byte[] InputStream.readNBytes(int) // reads exactly ?len? bytes or throws IOException with a single method byte[] InputStream.readNBytes(int) // reads at most ?len? bytes A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ Thanks, Brian On Jan 16, 2018, at 11:17 AM, Brian Burkhalter wrote: > https://bugs.openjdk.java.net/browse/JDK-8139206 > http://cr.openjdk.java.net/~bpb/8139206/webrev.00/ > > This change would add a new method ?byte[] InputStream.readNBytes(int len)? which would read up to at most ?len? bytes from the stream and return them in an internally allocated array. From bob.vandette at oracle.com Wed Jan 17 16:37:32 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 17 Jan 2018 11:37:32 -0500 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers Message-ID: <22077896-E632-4153-B70E-14B6FE10C9DB@oracle.com> Here?s an updated webrev addressing the comments from David Holmes. 1. Moved new cgroup specific support from unix -> Linux and put stubs in all other OS's 2. Reduced the size of the stack arrays in perfMemory_linux.cpp There are still two outstanding issues: [serviceability] : Can we and should we remove the getFile methods in the PerfDataFile class? [core-libs]: Do you have any problems with the module info file changes impacting java.base? Details: Bug: https://bugs.openjdk.java.net/browse/JDK-8193710 Webrev: http://cr.openjdk.java.net/~bobv/8193710/webrev.01/ Summary: This changeset enables the ability to use jcmd and jps running on a Host to list the java processes that are running in docker (cgroup based) containers. I?ve tested this change by examining processes running as root on both host and in docker containers as well as under my userid using ?jps and jcmd -l?. I?ve also tested the getFile functions with a small example program that I wrote. From what I can tell, these two getFile functions are not used in the JDK sources and in fact the PerfDataFile.getFile(int lvmid) method doesn?t appear to have never worked! I?d really like to remove these two methods. 140 candidate = new File(f.getName(), name); < Webrev: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8194554/webrev.00/ This fixes a mismatch between the specification and implementation. If MethodHandles.filterArguments is used to apply two or more non-null filters to a method handle, those filters will be run in right-to-left order in the current implementation whereas the javadoc indicates that these filters are run in normal argument order, left-to-right. Mandy From mandy.chung at oracle.com Wed Jan 17 18:16:35 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 17 Jan 2018 10:16:35 -0800 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers In-Reply-To: <22077896-E632-4153-B70E-14B6FE10C9DB@oracle.com> References: <22077896-E632-4153-B70E-14B6FE10C9DB@oracle.com> Message-ID: <7ea44913-4391-442f-3175-81524b106e7e@oracle.com> Hi Bob, I think the new methods in VMSupport and probablywith VMSupport.getVMTemporaryDirectory should belong to jdk.internal.jvmstat as they are specific for jvmstat and attach API to use. W.r.t VMSupportImpl, the implementation is the same for all platforms except Linux.? One option is to have a shared implementation class and instantiate an alternate implementation class, if present, using Class::forName. Another minor point is that the new getVMTemporaryDirectories method can return a List rather than an array. Mandy On 1/17/18 8:37 AM, Bob Vandette wrote: > Here?s an updated webrev addressing the comments from David Holmes. > > 1. Moved new cgroup specific support from unix -> Linux and put stubs in all other OS's > 2. Reduced the size of the stack arrays in perfMemory_linux.cpp > > > There are still two outstanding issues: > > [serviceability] : Can we and should we remove the getFile methods in the PerfDataFile class? > > [core-libs]: Do you have any problems with the module info file changes impacting java.base? > > Details: > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8193710 > > Webrev: > > http://cr.openjdk.java.net/~bobv/8193710/webrev.01/ > > Summary: > > This changeset enables the ability to use jcmd and jps running on a Host to > list the java processes that are running in docker (cgroup based) containers. > > I?ve tested this change by examining processes running as root on both host and in > docker containers as well as under my userid using ?jps and jcmd -l?. > I?ve also tested the getFile functions with a small example program that I wrote. > From what I can tell, these two getFile functions are not used in the JDK sources > and in fact the PerfDataFile.getFile(int lvmid) method doesn?t appear to have never worked! > I?d really like to remove these two methods. > > 140 candidate = new File(f.getName(), name); < > Here are some implementation details that I?ve added added to one of the Unix > specific source files (src/java.base/unix/classes/jdk/internal/vm/VMSupportImpl.java) > > /* Implementation Details: > * > * Java processes that run in docker containers are typically running > * under cgroups with separate pid namespaces which means that pids > * within the container are different that the pid which is visible > * from the host. The container pids typically start with 1 and > * increase. The java process running in the container will use these > * pids when creating the hsperfdata files. In order to locate java > * processes that are running in containers, we take advantage of > * the Linux proc file system which maps the containers tmp directory > * to the hosts under /proc/{hostpid}/root/tmp. We use the /proc status > * file /proc/{hostpid}/status to determine the containers pid and > * then access the hsperfdata file. The status file contains an > * entry "NSPid:" which shows the mapping from the hostpid to the > * containers pid. > * > * Example: > * > * NSPid: 24345 11 > * > * In this example process 24345 is visible from the host, > * is running under the PID namespace and has a container specific > * pid of 11. > * > * The search for Java processes is done by first looking in the > * traditional /tmp for host process hsperfdata files and then > * the search will container in every /proc/*/root/tmp directory. > * There are of course added complications to this search that > * need to be taken into account. > * > * 1. duplication of tmp directories > * > * /proc/{hostpid}/root/tmp directories exist for many processes > * that are running on a Linux kernel that has cgroups enabled even > * if they are not running in a container. To avoid this duplication, > * we compare the inode of the /proc tmp directories to /tmp and > * skip these duplicated directories. > * > * 2. Containerized processes without PID namespaces being enabled. > * > * If a container is running a Java process without namespaces being > * enabled, an hsperfdata file will only be located at > * /proc/{hostpid}/root/tmp/{hostpid}. This is handled by > * checking the last component in the path for both the hostpid > * and potential namespacepids (if one exists). > */ > > > > Bob. From bob.vandette at oracle.com Wed Jan 17 19:08:49 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Wed, 17 Jan 2018 14:08:49 -0500 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers In-Reply-To: <7ea44913-4391-442f-3175-81524b106e7e@oracle.com> References: <22077896-E632-4153-B70E-14B6FE10C9DB@oracle.com> <7ea44913-4391-442f-3175-81524b106e7e@oracle.com> Message-ID: <28DD8D38-DF9E-448F-AF72-63062D2CF781@oracle.com> I put these new methods in VMSupport since this was the class containg the getVMTemporaryDirectory and the intention of this class was document as: /* * Support class used by JVMTI and VM attach mechanism. */ class VMSupport { I could create a new PerfDataFileImpl or jdk.internal.jvmstat.VMSupport class with a Linux specific alternate implementation that contains these classes: getTemporaryDirectories getTemporaryDirectory getLocalVmId But the getNamespaceVmId method is also needed by the attach mechanism. I removed the duplicated functionality to avoid having to maintain this in 2 places. Shouldn?t the attach functionality, in the future, be decoupled from jvmstat? It looked like jcmd was using these jvmstat classes in order to avoid duplication but I don?t have all the history. Bob. > On Jan 17, 2018, at 1:16 PM, mandy chung wrote: > > Hi Bob, > > I think the new methods in VMSupport and probably with VMSupport.getVMTemporaryDirectory should belong to jdk.internal.jvmstat as they are specific for jvmstat and attach API to use. > > W.r.t VMSupportImpl, the implementation is the same for all platforms except Linux. One option is to have a shared implementation class and instantiate an alternate implementation class, if present, using Class::forName. > Another minor point is that the new getVMTemporaryDirectories method can return a List rather than an array. > > Mandy > > On 1/17/18 8:37 AM, Bob Vandette wrote: >> Here?s an updated webrev addressing the comments from David Holmes. >> >> 1. Moved new cgroup specific support from unix -> Linux and put stubs in all other OS's >> 2. Reduced the size of the stack arrays in perfMemory_linux.cpp >> >> >> There are still two outstanding issues: >> >> [serviceability] : Can we and should we remove the getFile methods in the PerfDataFile class? >> >> [core-libs]: Do you have any problems with the module info file changes impacting java.base? >> >> Details: >> >> Bug: >> >> >> https://bugs.openjdk.java.net/browse/JDK-8193710 >> >> >> Webrev: >> >> >> http://cr.openjdk.java.net/~bobv/8193710/webrev.01/ >> >> >> Summary: >> >> This changeset enables the ability to use jcmd and jps running on a Host to >> list the java processes that are running in docker (cgroup based) containers. >> >> I?ve tested this change by examining processes running as root on both host and in >> docker containers as well as under my userid using ?jps and jcmd -l?. >> I?ve also tested the getFile functions with a small example program that I wrote. >> From what I can tell, these two getFile functions are not used in the JDK sources >> and in fact the PerfDataFile.getFile(int lvmid) method doesn?t appear to have never worked! >> I?d really like to remove these two methods. >> >> 140 candidate = new File(f.getName(), name); <> >> Here are some implementation details that I?ve added added to one of the Unix >> specific source files (src/java.base/unix/classes/jdk/internal/vm/VMSupportImpl.java) >> >> /* Implementation Details: >> * >> * Java processes that run in docker containers are typically running >> * under cgroups with separate pid namespaces which means that pids >> * within the container are different that the pid which is visible >> * from the host. The container pids typically start with 1 and >> * increase. The java process running in the container will use these >> * pids when creating the hsperfdata files. In order to locate java >> * processes that are running in containers, we take advantage of >> * the Linux proc file system which maps the containers tmp directory >> * to the hosts under /proc/{hostpid}/root/tmp. We use the /proc status >> * file /proc/{hostpid}/status to determine the containers pid and >> * then access the hsperfdata file. The status file contains an >> * entry "NSPid:" which shows the mapping from the hostpid to the >> * containers pid. >> * >> * Example: >> * >> * NSPid: 24345 11 >> * >> * In this example process 24345 is visible from the host, >> * is running under the PID namespace and has a container specific >> * pid of 11. >> * >> * The search for Java processes is done by first looking in the >> * traditional /tmp for host process hsperfdata files and then >> * the search will container in every /proc/*/root/tmp directory. >> * There are of course added complications to this search that >> * need to be taken into account. >> * >> * 1. duplication of tmp directories >> * >> * /proc/{hostpid}/root/tmp directories exist for many processes >> * that are running on a Linux kernel that has cgroups enabled even >> * if they are not running in a container. To avoid this duplication, >> * we compare the inode of the /proc tmp directories to /tmp and >> * skip these duplicated directories. >> * >> * 2. Containerized processes without PID namespaces being enabled. >> * >> * If a container is running a Java process without namespaces being >> * enabled, an hsperfdata file will only be located at >> * /proc/{hostpid}/root/tmp/{hostpid}. This is handled by >> * checking the last component in the path for both the hostpid >> * and potential namespacepids (if one exists). >> */ >> >> >> >> Bob. >> > From brent.christian at oracle.com Wed Jan 17 19:18:17 2018 From: brent.christian at oracle.com (Brent Christian) Date: Wed, 17 Jan 2018 11:18:17 -0800 Subject: [11] RFR 8194879 : Runtime.Version parses string which does not conform to spec without throwing IAE In-Reply-To: References: <42c0e98a-a1d9-6e3a-2743-9606dbba6eae@oracle.com> Message-ID: Done[1] - thanks Roger. I also updated the copyright years. -Brent 1. http://cr.openjdk.java.net/~bchristi/8194879/webrev.01/ On 1/17/18 7:04 AM, Roger Riggs wrote: > Hi Brent, > > The bug number should be added to the @bug line in the test. > > Thanks, Roger > > On 1/16/2018 4:21 PM, Brent Christian wrote: >> Hi, >> >> Please review this version string parsing fix for Runtime.Version. >> >> Issue: >> https://bugs.openjdk.java.net/browse/JDK-8194879 >> Webrev: >> http://cr.openjdk.java.net/~bchristi/8194879/webrev.00/index.html >> >> From the bug: >> -- >> "As per the specification of Runtime.Version: >> >> A version string, $VSTR, is a version number $VNUM, as described >> above, optionally followed by pre-release and build information, in >> one of the following formats: >> >> ?????? 1. $VNUM(-$PRE)?\+$BUILD(-$OPT)? >> ?????? 2. $VNUM-$PRE(-$OPT)? >> ?????? 3. $VNUM(+-$OPT)? >> >> So an example of "10--ea" does not confirm to any of the above regex's." >> -- >> >> For "10--ea", the leading '-' gets picked up, and OPT (which allows >> the '-' character) gets set to "-ea".? So the pattern matching of the >> elements appears to happen as intended. >> >> A case can be added in the parse() code to disallow this (case #2 with >> an OPT present, but no PRE). >> >> Thanks, >> -Brent > From paul.sandoz at oracle.com Wed Jan 17 20:16:47 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 17 Jan 2018 12:16:47 -0800 Subject: [11] RFR JDK-8194554: filterArguments runs multiple filters in the wrong order In-Reply-To: <8e91ce78-7440-f881-acba-2c1110c4f57c@oracle.com> References: <8e91ce78-7440-f881-acba-2c1110c4f57c@oracle.com> Message-ID: <12F36661-3CEF-47E9-AB79-B5549517B368@oracle.com> > On 17 Jan 2018, at 08:50, mandy chung wrote: > > Webrev: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8194554/webrev.00/ > > This fixes a mismatch between the specification and implementation. > If MethodHandles.filterArguments is used to apply two or more non-null > filters to a method handle, those filters will be run in right-to-left > order in the current implementation whereas the javadoc indicates that > these filters are run in normal argument order, left-to-right. > Fix looks good, but perhaps the test logic could be made a little simpler? Since the filter methods update shared state what if they just add their identifier to a list so you can observe [?A?, ?B?, C?], or [?B?, ?C?] etc. Then you can test against the the list for the required order of filter calls rather putting logic in the filter methods themselves in what they return (they can just be identity functions with a side-effect). Paul. From mandy.chung at oracle.com Wed Jan 17 20:29:15 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 17 Jan 2018 12:29:15 -0800 Subject: [11] RFR JDK-8194554: filterArguments runs multiple filters in the wrong order In-Reply-To: <12F36661-3CEF-47E9-AB79-B5549517B368@oracle.com> References: <8e91ce78-7440-f881-acba-2c1110c4f57c@oracle.com> <12F36661-3CEF-47E9-AB79-B5549517B368@oracle.com> Message-ID: <25dcce4d-40f3-f40b-6520-1248889c76be@oracle.com> On 1/17/18 12:16 PM, Paul Sandoz wrote: > >> On 17 Jan 2018, at 08:50, mandy chung wrote: >> >> Webrev: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8194554/webrev.00/ >> >> This fixes a mismatch between the specification and implementation. >> If MethodHandles.filterArguments is used to apply two or more non-null >> filters to a method handle, those filters will be run in right-to-left >> order in the current implementation whereas the javadoc indicates that >> these filters are run in normal argument order, left-to-right. >> > Fix looks good, but perhaps the test logic could be made a little simpler? > > Since the filter methods update shared state what if they just add their identifier to a list so you can observe [?A?, ?B?, C?], or [?B?, ?C?] etc. > > Then you can test against the the list for the required order of filter calls rather putting logic in the filter methods themselves in what they return (they can just be identity functions with a side-effect). > That's a good idea (I should have started with a simpler test). Updated: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8194554/webrev.01/ Thanks Mandy From paul.sandoz at oracle.com Wed Jan 17 20:31:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 17 Jan 2018 12:31:17 -0800 Subject: [11] RFR JDK-8194554: filterArguments runs multiple filters in the wrong order In-Reply-To: <25dcce4d-40f3-f40b-6520-1248889c76be@oracle.com> References: <8e91ce78-7440-f881-acba-2c1110c4f57c@oracle.com> <12F36661-3CEF-47E9-AB79-B5549517B368@oracle.com> <25dcce4d-40f3-f40b-6520-1248889c76be@oracle.com> Message-ID: <39DDBEAC-DCD7-4DFB-BDEF-916E28787076@oracle.com> +1 Paul. > On 17 Jan 2018, at 12:29, mandy chung wrote: > > > > On 1/17/18 12:16 PM, Paul Sandoz wrote: >> >>> On 17 Jan 2018, at 08:50, mandy chung wrote: >>> >>> Webrev: >>> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8194554/webrev.00/ >>> >>> This fixes a mismatch between the specification and implementation. >>> If MethodHandles.filterArguments is used to apply two or more non-null >>> filters to a method handle, those filters will be run in right-to-left >>> order in the current implementation whereas the javadoc indicates that >>> these filters are run in normal argument order, left-to-right. >>> >> Fix looks good, but perhaps the test logic could be made a little simpler? >> >> Since the filter methods update shared state what if they just add their identifier to a list so you can observe [?A?, ?B?, C?], or [?B?, ?C?] etc. >> >> Then you can test against the the list for the required order of filter calls rather putting logic in the filter methods themselves in what they return (they can just be identity functions with a side-effect). >> > > That's a good idea (I should have started with a simpler test). > > Updated: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8194554/webrev.01/ > > Thanks > Mandy From john.r.rose at oracle.com Wed Jan 17 20:39:36 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 17 Jan 2018 12:39:36 -0800 Subject: [11] RFR JDK-8194554: filterArguments runs multiple filters in the wrong order In-Reply-To: <39DDBEAC-DCD7-4DFB-BDEF-916E28787076@oracle.com> References: <8e91ce78-7440-f881-acba-2c1110c4f57c@oracle.com> <12F36661-3CEF-47E9-AB79-B5549517B368@oracle.com> <25dcce4d-40f3-f40b-6520-1248889c76be@oracle.com> <39DDBEAC-DCD7-4DFB-BDEF-916E28787076@oracle.com> Message-ID: >> That's a good idea (I should have started with a simpler test). >> >> Updated: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8194554/webrev.01/ Thanks, Mandy and Paul. Reviewed. ? John From david.holmes at oracle.com Wed Jan 17 21:39:56 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Jan 2018 07:39:56 +1000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <5074508851404c77ae85ee86035866b5@sap.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> <5074508851404c77ae85ee86035866b5@sap.com> Message-ID: Hi Goetz, On 17/01/2018 10:42 PM, Lindenmaier, Goetz wrote: > Hi David, > > thanks for looking at the change. > > We actually run the aix VM also on OS400. > And as OS.java says that it's a " Stripped down version of jtreg > internal class com.sun.javatest.regtest.config.OS" > I not only grabbed the AIX case from there, but also the OS400 one. > I don't think this will do any harm. Can't speak to what platforms jtreg runs on but OpenJDK doesn't support OS/400, so it shouldn't be listed. Sorry. Thanks, David > I'll post it on awt-dev, too. > > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Mittwoch, 17. Januar 2018 12:55 >> To: Lindenmaier, Goetz ; Volker Simonis >> >> Cc: core-libs-dev at openjdk.java.net; Baesken, Matthias >> >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. >> >> Hi Goetz, >> >> test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java >> >> I thought this was for AIX recognition not OS/400 ? ;-) >> >> Otherwise the AIX changes seem okay. Thought AWT tests should be >> reviewed on awt-dev. >> >> Cheers, >> David >> >> On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> Volker, thanks for your review. Could someone else please have a >>> quick look at this simple fix? >>> >>> Updated webrev with fixed BadPluginConfigurationTest: >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ >>> >>> Best regards, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com] >>>> Sent: Dienstag, 16. Januar 2018 19:28 >>>> To: Lindenmaier, Goetz >>>> Cc: Baesken, Matthias ; core-libs- >>>> dev at openjdk.java.net >>>> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >> platforms. >>>> >>>> Hi Goetz, >>>> >>>> you change looks good. Just one minor nit (no need for a new webrev if >>>> you agree with my fix :) >>>> >>>> In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as >>>> file. This is a useless check because there won't be a file "mktemp" >>>> in the current exectution directory of the shell script, even if the >>>> "mktemp" exists domehwere in the PATH. To check for the existence of a >>>> "mktemp" executable you can use the "type" command: >>>> >>>> type mktempp> /dev/null 2>&1 >>>> >>>> if ! [ $? -ne 0 ] ; then >>>> MKTEMP="/opt/freeware/bin/mktemp" >>>> fi >>>> >>>> Thanks for finally fixing this, >>>> Volker >>>> >>>> >>>> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz >>>> wrote: >>>>> Hi Matthias, >>>>> >>>>> I didin't mean to do any changes for other platforms, so >>>>> I reverted the part you pointed out: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ >>>>> >>>>> But scanning through all the scripts: >>>>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>>> -----Original Message----- >>>>>> From: Baesken, Matthias >>>>>> Sent: Montag, 15. Januar 2018 09:03 >>>>>> To: Lindenmaier, Goetz ; core-libs- >>>>>> dev at openjdk.java.net >>>>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >>>> platforms. >>>>>> >>>>>> Hi Goetz, change looks good except >>>>>> >>>>>> test/jdk/sun/security/tools/keytool/i18n.sh >>>>>> >>>>>> where the Cygwin PS setting has been changed from ; to : (probably >> this >>>>>> was not intended) : >>>>>> >>>>>> PS=":" >>>>>> FS="/" >>>>>> ;; >>>>>> - CYGWIN* ) >>>>>> - NULL=/dev/null >>>>>> - PS=";" >>>>>> >>>>>> >>>>>> (not a Reviewer however). >>>>>> >>>>>> Best regards, Matthias >>>>>> >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Lindenmaier, Goetz >>>>>>> Sent: Mittwoch, 10. Januar 2018 09:23 >>>>>>> To: core-libs-dev at openjdk.java.net >>>>>>> Cc: Baesken, Matthias >>>>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >> platforms. >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> this change adapts a row of jtreg tests to AIX. >>>>>>> Shell tests often switch over all OSes, here the case for AIX was >>>>>>> missing. PKCSTest needs to know where the libs are on s390. >>>>>>> >>>>>>> Please review. >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz. From volker.simonis at gmail.com Wed Jan 17 23:21:28 2018 From: volker.simonis at gmail.com (Volker Simonis) Date: Wed, 17 Jan 2018 23:21:28 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> <5074508851404c77ae85ee86035866b5@sap.com> Message-ID: David Holmes schrieb am Mi. 17. Jan. 2018 um 22:45: > Hi Goetz, > > On 17/01/2018 10:42 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > thanks for looking at the change. > > > > We actually run the aix VM also on OS400. > > And as OS.java says that it's a " Stripped down version of jtreg > > internal class com.sun.javatest.regtest.config.OS" > > I not only grabbed the AIX case from there, but also the OS400 one. > > I don't think this will do any harm. > > Can't speak to what platforms jtreg runs on but OpenJDK doesn't support > OS/400, so it shouldn't be listed. Sorry. Hi David, you?re right that OpenJDK doesn?t run natively on OS/400, but OS/400 provides an emulation layer called PASE [1] which allows to run native AIX application on OS/400. The change proposed by Goetz (which is already in JTreg since quite some time) is required to test the OpenJDK AIX Port on OS/400 in PASE mode. Regards, Volker [1] https://www.ibm.com/support/knowledgecenter/en/ssw_i5_54/rzalf/rzalfintro.htm > > Thanks, > David > > > I'll post it on awt-dev, too. > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Mittwoch, 17. Januar 2018 12:55 > >> To: Lindenmaier, Goetz ; Volker Simonis > >> > >> Cc: core-libs-dev at openjdk.java.net; Baesken, Matthias > >> > >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > >> > >> Hi Goetz, > >> > >> > test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java > >> > >> I thought this was for AIX recognition not OS/400 ? ;-) > >> > >> Otherwise the AIX changes seem okay. Thought AWT tests should be > >> reviewed on awt-dev. > >> > >> Cheers, > >> David > >> > >> On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> Volker, thanks for your review. Could someone else please have a > >>> quick look at this simple fix? > >>> > >>> Updated webrev with fixed BadPluginConfigurationTest: > >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > >>> > >>> Best regards, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com] > >>>> Sent: Dienstag, 16. Januar 2018 19:28 > >>>> To: Lindenmaier, Goetz > >>>> Cc: Baesken, Matthias ; core-libs- > >>>> dev at openjdk.java.net > >>>> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>> > >>>> Hi Goetz, > >>>> > >>>> you change looks good. Just one minor nit (no need for a new webrev if > >>>> you agree with my fix :) > >>>> > >>>> In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as > >>>> file. This is a useless check because there won't be a file "mktemp" > >>>> in the current exectution directory of the shell script, even if the > >>>> "mktemp" exists domehwere in the PATH. To check for the existence of a > >>>> "mktemp" executable you can use the "type" command: > >>>> > >>>> type mktempp> /dev/null 2>&1 > >>>> > >>>> if ! [ $? -ne 0 ] ; then > >>>> MKTEMP="/opt/freeware/bin/mktemp" > >>>> fi > >>>> > >>>> Thanks for finally fixing this, > >>>> Volker > >>>> > >>>> > >>>> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz > >>>> wrote: > >>>>> Hi Matthias, > >>>>> > >>>>> I didin't mean to do any changes for other platforms, so > >>>>> I reverted the part you pointed out: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ > >>>>> > >>>>> But scanning through all the scripts: > >>>>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: Baesken, Matthias > >>>>>> Sent: Montag, 15. Januar 2018 09:03 > >>>>>> To: Lindenmaier, Goetz ; core-libs- > >>>>>> dev at openjdk.java.net > >>>>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >>>> platforms. > >>>>>> > >>>>>> Hi Goetz, change looks good except > >>>>>> > >>>>>> test/jdk/sun/security/tools/keytool/i18n.sh > >>>>>> > >>>>>> where the Cygwin PS setting has been changed from ; to : (probably > >> this > >>>>>> was not intended) : > >>>>>> > >>>>>> PS=":" > >>>>>> FS="/" > >>>>>> ;; > >>>>>> - CYGWIN* ) > >>>>>> - NULL=/dev/null > >>>>>> - PS=";" > >>>>>> > >>>>>> > >>>>>> (not a Reviewer however). > >>>>>> > >>>>>> Best regards, Matthias > >>>>>> > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Lindenmaier, Goetz > >>>>>>> Sent: Mittwoch, 10. Januar 2018 09:23 > >>>>>>> To: core-libs-dev at openjdk.java.net > >>>>>>> Cc: Baesken, Matthias > >>>>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> this change adapts a row of jtreg tests to AIX. > >>>>>>> Shell tests often switch over all OSes, here the case for AIX was > >>>>>>> missing. PKCSTest needs to know where the libs are on s390. > >>>>>>> > >>>>>>> Please review. > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Goetz. > From mandy.chung at oracle.com Thu Jan 18 00:49:26 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 17 Jan 2018 16:49:26 -0800 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers In-Reply-To: <28DD8D38-DF9E-448F-AF72-63062D2CF781@oracle.com> References: <22077896-E632-4153-B70E-14B6FE10C9DB@oracle.com> <7ea44913-4391-442f-3175-81524b106e7e@oracle.com> <28DD8D38-DF9E-448F-AF72-63062D2CF781@oracle.com> Message-ID: On 1/17/18 11:08 AM, Bob Vandette wrote: > I put these new methods in VMSupport since this was the class containg the getVMTemporaryDirectory and > the intention of this class was document as: > > /* > * Support class used by JVMTI and VM attach mechanism. > */ > class VMSupport { I looked at the history.? Initially VMSupport provides entry points for VM to call out.? VMSupport::serializePropertiesToByteArray and serializeAgentPropertiesToByteArray? are called by VM. VMSupport::getVMTemporaryDirectory was added in JDK 9 and all VMs use the same directory for all hsperfdata files. The API is currently used by jdk.internal.jvmstat and jdk.management.agent.?? But you may be right that when we decouple jdk.attach from jvmstat, jdk.attach may need to use this API (I don't know how it looks like yet). > I could create a new PerfDataFileImpl or jdk.internal.jvmstat.VMSupport class with a Linux specific alternate implementation that contains these classes: > > getTemporaryDirectories > getTemporaryDirectory > getLocalVmId > But the getNamespaceVmId method is also needed by the attach mechanism. > I removed the duplicated functionality to avoid having to maintain this in > 2 places. Ah, it was in jdk.attach. > Shouldn?t the attach functionality, in the future, be decoupled from jvmstat? It looked like jcmd was using these jvmstat classes in order to avoid duplication but I don?t have all the history. Yes and we attempted to eliminate the dependency from jdk.attach to jdk.internal.jvmstat but we haven't found a good way yet.? Since these new methods are implementation for attach and jvmstat, I personally prefer them to live there. ? As for getNamespaceVmId, there is also a native implementation in VM.? I don't have a good suggestion.? Since it's a small utility, duplicating it for now may not be too bad and wait until attach and jvmstat are decoupled. Mandy From david.holmes at oracle.com Thu Jan 18 01:17:41 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Jan 2018 11:17:41 +1000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> <5074508851404c77ae85ee86035866b5@sap.com> Message-ID: <29748bd2-8e37-d336-a00f-5a87750ce6be@oracle.com> Hi Volker, On 18/01/2018 9:21 AM, Volker Simonis wrote: > > David Holmes > > schrieb am Mi. 17. Jan. 2018 um 22:45: > > Hi Goetz, > > On 17/01/2018 10:42 PM, Lindenmaier, Goetz wrote: > > Hi David, > > > > thanks for looking at the change. > > > > We actually run the aix VM also on OS400. > > And as OS.java says that it's a " Stripped down version of jtreg > > internal class com.sun.javatest.regtest.config.OS" > > I not only grabbed the AIX case from there, but also the OS400 one. > > I don't think this will do any harm. > > Can't speak to what platforms jtreg runs on but OpenJDK doesn't support > OS/400, so it shouldn't be listed. Sorry. > > > Hi David, > > you?re right that OpenJDK doesn?t run natively on OS/400, but OS/400 > provides an emulation layer called PASE [1] which allows to run native > AIX application on OS/400. The change proposed by Goetz (which is > already in JTreg since quite some time) is required to test the OpenJDK > AIX Port on OS/400 in PASE mode. Sorry but I don't see how that one occurrence in test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java allows for testing on OS/400 when none of the actual tests that examine the OS have a case for "os400" ?? (Wouldn't it be better to pretend to be AIX in that case?) David ----- > Regards, > Volker > > [1] > https://www.ibm.com/support/knowledgecenter/en/ssw_i5_54/rzalf/rzalfintro.htm > > > > Thanks, > David > > > I'll post it on awt-dev, too. > > > > Best regards, > >? ? Goetz. > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com > ] > >> Sent: Mittwoch, 17. Januar 2018 12:55 > >> To: Lindenmaier, Goetz >; Volker Simonis > >> > > >> Cc: core-libs-dev at openjdk.java.net > ; Baesken, Matthias > >> > > >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests > to platforms. > >> > >> Hi Goetz, > >> > >> > test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java > >> > >> I thought this was for AIX recognition not OS/400 ? ;-) > >> > >> Otherwise the AIX changes seem okay. Thought AWT tests should be > >> reviewed on awt-dev. > >> > >> Cheers, > >> David > >> > >> On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> Volker, thanks for your review. Could someone else please have a > >>> quick look at this simple fix? > >>> > >>> Updated webrev with fixed BadPluginConfigurationTest: > >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > >>> > >>> Best regards, > >>>? ? ?Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com > ] > >>>> Sent: Dienstag, 16. Januar 2018 19:28 > >>>> To: Lindenmaier, Goetz > > >>>> Cc: Baesken, Matthias >; core-libs- > >>>> dev at openjdk.java.net > >>>> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>> > >>>> Hi Goetz, > >>>> > >>>> you change looks good. Just one minor nit (no need for a new > webrev if > >>>> you agree with my fix :) > >>>> > >>>> In BadPluginConfigurationTest.sh you first check if 'mktemp' > exists as > >>>> file. This is a useless check because there won't be a file > "mktemp" > >>>> in the current exectution directory of the shell script, even > if the > >>>> "mktemp" exists domehwere in the PATH. To check for the > existence of a > >>>> "mktemp" executable you can use the "type" command: > >>>> > >>>> type mktempp> /dev/null 2>&1 > >>>> > >>>> if ! [ $? -ne 0 ] ; then > >>>>? ? ?MKTEMP="/opt/freeware/bin/mktemp" > >>>> fi > >>>> > >>>> Thanks for finally fixing this, > >>>> Volker > >>>> > >>>> > >>>> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz > >>>> > > wrote: > >>>>> Hi Matthias, > >>>>> > >>>>> I didin't mean to do any changes for other platforms, so > >>>>> I reverted the part you pointed out: > >>>>> > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ > >>>>> > >>>>> But scanning through all the scripts: > >>>>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. > Strange ... > >>>>> > >>>>> Best regards, > >>>>>? ? ?Goetz. > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: Baesken, Matthias > >>>>>> Sent: Montag, 15. Januar 2018 09:03 > >>>>>> To: Lindenmaier, Goetz >; core-libs- > >>>>>> dev at openjdk.java.net > >>>>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt > tests to > >>>> platforms. > >>>>>> > >>>>>> Hi Goetz, change looks good except > >>>>>> > >>>>>> test/jdk/sun/security/tools/keytool/i18n.sh > >>>>>> > >>>>>> where the Cygwin PS setting has been changed from ; to : > ?(probably > >> this > >>>>>> was not intended) : > >>>>>> > >>>>>>? ? ? ? PS=":" > >>>>>>? ? ? ? FS="/" > >>>>>>? ? ? ? ;; > >>>>>> -? CYGWIN* ) > >>>>>> -? ? NULL=/dev/null > >>>>>> -? ? PS=";" > >>>>>> > >>>>>> > >>>>>> (not a Reviewer however). > >>>>>> > >>>>>> Best regards, Matthias > >>>>>> > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Lindenmaier, Goetz > >>>>>>> Sent: Mittwoch, 10. Januar 2018 09:23 > >>>>>>> To: core-libs-dev at openjdk.java.net > > >>>>>>> Cc: Baesken, Matthias > > >>>>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> this change adapts a row of jtreg tests to AIX. > >>>>>>> Shell tests often switch over all OSes, here the case for > AIX was > >>>>>>> missing. PKCSTest needs to know where the libs are on s390. > >>>>>>> > >>>>>>> Please review. > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ > >>>>>>> > >>>>>>> Best regards, > >>>>>>>? ? ?Goetz. > From Sergey.Bylokhov at oracle.com Thu Jan 18 03:59:15 2018 From: Sergey.Bylokhov at oracle.com (Sergey Bylokhov) Date: Wed, 17 Jan 2018 19:59:15 -0800 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <1290763f577847109b7a49f3f2621d09@sap.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> <1290763f577847109b7a49f3f2621d09@sap.com> Message-ID: <5d8b4472-3328-51c1-6a16-c65f0d3d7afe@oracle.com> The client part looks fine. On 17/01/2018 04:43, Lindenmaier, Goetz wrote: > Posting this to awt-dev, too. > > This change adapts some tests to run on properly on aix or s390: > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Mittwoch, 17. Januar 2018 12:55 >> To: Lindenmaier, Goetz ; Volker Simonis >> >> Cc: core-libs-dev at openjdk.java.net; Baesken, Matthias >> >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. >> >> Hi Goetz, >> >> test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java >> >> I thought this was for AIX recognition not OS/400 ? ;-) >> >> Otherwise the AIX changes seem okay. Thought AWT tests should be >> reviewed on awt-dev. >> >> Cheers, >> David >> >> On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> Volker, thanks for your review. Could someone else please have a >>> quick look at this simple fix? >>> >>> Updated webrev with fixed BadPluginConfigurationTest: >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ >>> >>> Best regards, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com] >>>> Sent: Dienstag, 16. Januar 2018 19:28 >>>> To: Lindenmaier, Goetz >>>> Cc: Baesken, Matthias ; core-libs- >>>> dev at openjdk.java.net >>>> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >> platforms. >>>> >>>> Hi Goetz, >>>> >>>> you change looks good. Just one minor nit (no need for a new webrev if >>>> you agree with my fix :) >>>> >>>> In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as >>>> file. This is a useless check because there won't be a file "mktemp" >>>> in the current exectution directory of the shell script, even if the >>>> "mktemp" exists domehwere in the PATH. To check for the existence of a >>>> "mktemp" executable you can use the "type" command: >>>> >>>> type mktempp> /dev/null 2>&1 >>>> >>>> if ! [ $? -ne 0 ] ; then >>>> MKTEMP="/opt/freeware/bin/mktemp" >>>> fi >>>> >>>> Thanks for finally fixing this, >>>> Volker >>>> >>>> >>>> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz >>>> wrote: >>>>> Hi Matthias, >>>>> >>>>> I didin't mean to do any changes for other platforms, so >>>>> I reverted the part you pointed out: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ >>>>> >>>>> But scanning through all the scripts: >>>>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... >>>>> >>>>> Best regards, >>>>> Goetz. >>>>> >>>>>> -----Original Message----- >>>>>> From: Baesken, Matthias >>>>>> Sent: Montag, 15. Januar 2018 09:03 >>>>>> To: Lindenmaier, Goetz ; core-libs- >>>>>> dev at openjdk.java.net >>>>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >>>> platforms. >>>>>> >>>>>> Hi Goetz, change looks good except >>>>>> >>>>>> test/jdk/sun/security/tools/keytool/i18n.sh >>>>>> >>>>>> where the Cygwin PS setting has been changed from ; to : (probably >> this >>>>>> was not intended) : >>>>>> >>>>>> PS=":" >>>>>> FS="/" >>>>>> ;; >>>>>> - CYGWIN* ) >>>>>> - NULL=/dev/null >>>>>> - PS=";" >>>>>> >>>>>> >>>>>> (not a Reviewer however). >>>>>> >>>>>> Best regards, Matthias >>>>>> >>>>>> >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Lindenmaier, Goetz >>>>>>> Sent: Mittwoch, 10. Januar 2018 09:23 >>>>>>> To: core-libs-dev at openjdk.java.net >>>>>>> Cc: Baesken, Matthias >>>>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >> platforms. >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> this change adapts a row of jtreg tests to AIX. >>>>>>> Shell tests often switch over all OSes, here the case for AIX was >>>>>>> missing. PKCSTest needs to know where the libs are on s390. >>>>>>> >>>>>>> Please review. >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev/ >>>>>>> >>>>>>> Best regards, >>>>>>> Goetz. -- Best regards, Sergey. From huaming.li at oracle.com Thu Jan 18 07:41:34 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Thu, 18 Jan 2018 15:41:34 +0800 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <961a4bb4-9deb-fc5a-7e21-cffdbdd98dc8@oracle.com> References: <961a4bb4-9deb-fc5a-7e21-cffdbdd98dc8@oracle.com> Message-ID: Hi David, Sometime we will run test by command "jtreg -timeoutFactor:xxx ...", xxx may be a large number than usual, e.g. 100. The reason we supply a large number for timeoutFactor is because we're running test on a very slow machine, or we're running test on a docker environment which has very limited resource, without a large timeoutFactor value, the test will finally get timeout exception. At this situation, it's NOT reasonable for user to know the exact timeoutFactor by which the calculated result will be MAX_TIMEOUT_MS, what's helpful is to let user supply a large enough timeoutFactor value and round down the timeout value to MAX_TIMEOUT_MS automatically, rather than throw an exception. Thank you -Hamlin On 18/01/2018 3:15 PM, David Holmes wrote: > Hi Hamlin, > > This should probably be reviewed on core-libs-dev. I don't think > jdk-dev is intended for code reviews. > > On 18/01/2018 4:59 PM, Hamlin Li wrote: >> Would you please review the following patch? >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >> >> webrev as below. > > I don't agree with this. Whatever is passing the incorrect timeout to > the TestLibrary should be fixed. The bug report needs more information > about where the incorrect value is coming from and why. > > Cheers, > David > >> >> Thank you >> >> -Hamlin >> >> ------------------------------------------------------------------------ >> >> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >> 20:07:50 2018 -0800 >> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >> 14:54:50 2018 +0800 >> @@ -188,8 +188,12 @@ >> ????? public static long computeDeadline(long timestamp, long timeout) { >> ????????? final long MAX_TIMEOUT_MS = 3_600_000L; >> >> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >> +??????? if (timeout < 0L) { >> ????????????? throw new IllegalArgumentException("timeout " + timeout >> + "ms out of range"); >> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >> +??????????? System.out.format("timeout value(%d) exceeds >> MAX_TIMEOUT_MS(%d), " >> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >> MAX_TIMEOUT_MS); >> +??????????? timeout = MAX_TIMEOUT_MS; >> ????????? } >> >> ????????? return timestamp + (long)(timeout * getTimeoutFactor()); >> From david.holmes at oracle.com Thu Jan 18 07:48:58 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Jan 2018 17:48:58 +1000 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: References: <961a4bb4-9deb-fc5a-7e21-cffdbdd98dc8@oracle.com> Message-ID: <1d2aa9e9-3b41-04bd-6a54-6b3e419b41dc@oracle.com> On 18/01/2018 5:41 PM, Hamlin Li wrote: > Hi David, > > Sometime we will run test by command "jtreg -timeoutFactor:xxx ...", xxx Yes but that controls how jtreg manages the execution of the test. How is that then being used by the tests that call TestDriver? Further if TestDriver limits the timeout to a smaller value, then the increase in timeoutFactor won't help at all - people will see a timeout, increase the timeoutFactor and still see a timeout! For that matter why is TestDriver even imposing a timeout of its own when jtreg imposes an overall timeout? In general we have been moving away from test initiated timeouts and letting the test harness handle it. Cheers, David > may be a large number than usual, e.g. 100. The reason we supply a large > number for timeoutFactor is because we're running test on a very slow > machine, or we're running test on a docker environment which has very > limited resource, without a large timeoutFactor value, the test will > finally get timeout exception. At this situation, it's NOT reasonable > for user to know the exact timeoutFactor by which the calculated result > will be MAX_TIMEOUT_MS, what's helpful is to let user supply a large > enough timeoutFactor value and round down the timeout value to > MAX_TIMEOUT_MS automatically, rather than throw an exception. > > Thank you > > -Hamlin > > > On 18/01/2018 3:15 PM, David Holmes wrote: >> Hi Hamlin, >> >> This should probably be reviewed on core-libs-dev. I don't think >> jdk-dev is intended for code reviews. >> >> On 18/01/2018 4:59 PM, Hamlin Li wrote: >>> Would you please review the following patch? >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>> >>> webrev as below. >> >> I don't agree with this. Whatever is passing the incorrect timeout to >> the TestLibrary should be fixed. The bug report needs more information >> about where the incorrect value is coming from and why. >> >> Cheers, >> David >> >>> >>> Thank you >>> >>> -Hamlin >>> >>> ------------------------------------------------------------------------ >>> >>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >>> 20:07:50 2018 -0800 >>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >>> 14:54:50 2018 +0800 >>> @@ -188,8 +188,12 @@ >>> ????? public static long computeDeadline(long timestamp, long timeout) { >>> ????????? final long MAX_TIMEOUT_MS = 3_600_000L; >>> >>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>> +??????? if (timeout < 0L) { >>> ????????????? throw new IllegalArgumentException("timeout " + timeout >>> + "ms out of range"); >>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>> +??????????? System.out.format("timeout value(%d) exceeds >>> MAX_TIMEOUT_MS(%d), " >>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>> MAX_TIMEOUT_MS); >>> +??????????? timeout = MAX_TIMEOUT_MS; >>> ????????? } >>> >>> ????????? return timestamp + (long)(timeout * getTimeoutFactor()); >>> > From huaming.li at oracle.com Thu Jan 18 08:05:29 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Thu, 18 Jan 2018 16:05:29 +0800 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <1d2aa9e9-3b41-04bd-6a54-6b3e419b41dc@oracle.com> References: <961a4bb4-9deb-fc5a-7e21-cffdbdd98dc8@oracle.com> <1d2aa9e9-3b41-04bd-6a54-6b3e419b41dc@oracle.com> Message-ID: <4910b950-cc4e-99b9-3b58-100a4076ec1b@oracle.com> On 18/01/2018 3:48 PM, David Holmes wrote: > On 18/01/2018 5:41 PM, Hamlin Li wrote: >> Hi David, >> >> Sometime we will run test by command "jtreg -timeoutFactor:xxx ...", xxx > > Yes but that controls how jtreg manages the execution of the test. right. > How is that then being used by the tests that call TestDriver? I assume you mean TestLibrary rather than TestDriver. If you follow the code in RMID.java, you will find following code, it calculates the timeout value by timeoutFactor value. ??????? long waitTime = (long)(240_000 * TestLibrary.getTimeoutFactor()); ??????? restartTimeout = (long)(waitTime * 0.9); > Further if TestDriver limits the timeout to a smaller value, then the > increase in timeoutFactor won't help at all - people will see a > timeout, increase the timeoutFactor and still see a timeout! No, by pass a large timeoutFactor value, I will get test passed rather than get timeout exception. > > For that matter why is TestDriver even imposing a timeout of its own > when jtreg imposes an overall timeout? It's a history topic, if it's necessary we could file another enhancement or bug to track it. > In general we have been moving away from test initiated timeouts and > letting the test harness handle it. Yes, I agree, that's the reason I refactor it to round down the timeout to MAX_TIMEOUT_MS rather than throw an exception by test itself. Thank you -Hamlin > > Cheers, > David > >> may be a large number than usual, e.g. 100. The reason we supply a >> large number for timeoutFactor is because we're running test on a >> very slow machine, or we're running test on a docker environment >> which has very limited resource, without a large timeoutFactor value, >> the test will finally get timeout exception. At this situation, it's >> NOT reasonable for user to know the exact timeoutFactor by which the >> calculated result will be MAX_TIMEOUT_MS, what's helpful is to let >> user supply a large enough timeoutFactor value and round down the >> timeout value to MAX_TIMEOUT_MS automatically, rather than throw an >> exception. >> >> Thank you >> >> -Hamlin >> >> >> On 18/01/2018 3:15 PM, David Holmes wrote: >>> Hi Hamlin, >>> >>> This should probably be reviewed on core-libs-dev. I don't think >>> jdk-dev is intended for code reviews. >>> >>> On 18/01/2018 4:59 PM, Hamlin Li wrote: >>>> Would you please review the following patch? >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>>> >>>> webrev as below. >>> >>> I don't agree with this. Whatever is passing the incorrect timeout >>> to the TestLibrary should be fixed. The bug report needs more >>> information about where the incorrect value is coming from and why. >>> >>> Cheers, >>> David >>> >>>> >>>> Thank you >>>> >>>> -Hamlin >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >>>> 20:07:50 2018 -0800 >>>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >>>> 14:54:50 2018 +0800 >>>> @@ -188,8 +188,12 @@ >>>> ????? public static long computeDeadline(long timestamp, long >>>> timeout) { >>>> ????????? final long MAX_TIMEOUT_MS = 3_600_000L; >>>> >>>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>>> +??????? if (timeout < 0L) { >>>> ????????????? throw new IllegalArgumentException("timeout " + >>>> timeout + "ms out of range"); >>>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>>> +??????????? System.out.format("timeout value(%d) exceeds >>>> MAX_TIMEOUT_MS(%d), " >>>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>>> MAX_TIMEOUT_MS); >>>> +??????????? timeout = MAX_TIMEOUT_MS; >>>> ????????? } >>>> >>>> ????????? return timestamp + (long)(timeout * getTimeoutFactor()); >>>> >> From goetz.lindenmaier at sap.com Thu Jan 18 08:30:19 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 18 Jan 2018 08:30:19 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <5d8b4472-3328-51c1-6a16-c65f0d3d7afe@oracle.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> <1290763f577847109b7a49f3f2621d09@sap.com> <5d8b4472-3328-51c1-6a16-c65f0d3d7afe@oracle.com> Message-ID: <9da7d508731d4a61ba6e00800f806c4b@sap.com> Thanks Sergey! Best regards, Goetz. > -----Original Message----- > From: Sergey Bylokhov [mailto:Sergey.Bylokhov at oracle.com] > Sent: Donnerstag, 18. Januar 2018 04:59 > To: Lindenmaier, Goetz ; awt- > dev at openjdk.java.net > Cc: core-libs-dev at openjdk.java.net > Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > > The client part looks fine. > > On 17/01/2018 04:43, Lindenmaier, Goetz wrote: > > Posting this to awt-dev, too. > > > > This change adapts some tests to run on properly on aix or s390: > > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > > > > Best regards, > > Goetz. > > > >> -----Original Message----- > >> From: David Holmes [mailto:david.holmes at oracle.com] > >> Sent: Mittwoch, 17. Januar 2018 12:55 > >> To: Lindenmaier, Goetz ; Volker Simonis > >> > >> Cc: core-libs-dev at openjdk.java.net; Baesken, Matthias > >> > >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > platforms. > >> > >> Hi Goetz, > >> > >> > test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java > >> > >> I thought this was for AIX recognition not OS/400 ? ;-) > >> > >> Otherwise the AIX changes seem okay. Thought AWT tests should be > >> reviewed on awt-dev. > >> > >> Cheers, > >> David > >> > >> On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: > >>> Hi, > >>> > >>> Volker, thanks for your review. Could someone else please have a > >>> quick look at this simple fix? > >>> > >>> Updated webrev with fixed BadPluginConfigurationTest: > >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.03/ > >>> > >>> Best regards, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com] > >>>> Sent: Dienstag, 16. Januar 2018 19:28 > >>>> To: Lindenmaier, Goetz > >>>> Cc: Baesken, Matthias ; core-libs- > >>>> dev at openjdk.java.net > >>>> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>> > >>>> Hi Goetz, > >>>> > >>>> you change looks good. Just one minor nit (no need for a new webrev > if > >>>> you agree with my fix :) > >>>> > >>>> In BadPluginConfigurationTest.sh you first check if 'mktemp' exists as > >>>> file. This is a useless check because there won't be a file "mktemp" > >>>> in the current exectution directory of the shell script, even if the > >>>> "mktemp" exists domehwere in the PATH. To check for the existence > of a > >>>> "mktemp" executable you can use the "type" command: > >>>> > >>>> type mktempp> /dev/null 2>&1 > >>>> > >>>> if ! [ $? -ne 0 ] ; then > >>>> MKTEMP="/opt/freeware/bin/mktemp" > >>>> fi > >>>> > >>>> Thanks for finally fixing this, > >>>> Volker > >>>> > >>>> > >>>> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz > >>>> wrote: > >>>>> Hi Matthias, > >>>>> > >>>>> I didin't mean to do any changes for other platforms, so > >>>>> I reverted the part you pointed out: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869- > jtregAix/webrev.02/ > >>>>> > >>>>> But scanning through all the scripts: > >>>>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. Strange ... > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >>>>> > >>>>>> -----Original Message----- > >>>>>> From: Baesken, Matthias > >>>>>> Sent: Montag, 15. Januar 2018 09:03 > >>>>>> To: Lindenmaier, Goetz ; core-libs- > >>>>>> dev at openjdk.java.net > >>>>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >>>> platforms. > >>>>>> > >>>>>> Hi Goetz, change looks good except > >>>>>> > >>>>>> test/jdk/sun/security/tools/keytool/i18n.sh > >>>>>> > >>>>>> where the Cygwin PS setting has been changed from ; to : (probably > >> this > >>>>>> was not intended) : > >>>>>> > >>>>>> PS=":" > >>>>>> FS="/" > >>>>>> ;; > >>>>>> - CYGWIN* ) > >>>>>> - NULL=/dev/null > >>>>>> - PS=";" > >>>>>> > >>>>>> > >>>>>> (not a Reviewer however). > >>>>>> > >>>>>> Best regards, Matthias > >>>>>> > >>>>>> > >>>>>> > >>>>>>> -----Original Message----- > >>>>>>> From: Lindenmaier, Goetz > >>>>>>> Sent: Mittwoch, 10. Januar 2018 09:23 > >>>>>>> To: core-libs-dev at openjdk.java.net > >>>>>>> Cc: Baesken, Matthias > >>>>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > >> platforms. > >>>>>>> > >>>>>>> Hi, > >>>>>>> > >>>>>>> this change adapts a row of jtreg tests to AIX. > >>>>>>> Shell tests often switch over all OSes, here the case for AIX was > >>>>>>> missing. PKCSTest needs to know where the libs are on s390. > >>>>>>> > >>>>>>> Please review. > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869- > jtregAix/webrev/ > >>>>>>> > >>>>>>> Best regards, > >>>>>>> Goetz. > > > -- > Best regards, Sergey. From goetz.lindenmaier at sap.com Thu Jan 18 09:00:43 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 18 Jan 2018 09:00:43 +0000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <29748bd2-8e37-d336-a00f-5a87750ce6be@oracle.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> <5074508851404c77ae85ee86035866b5@sap.com> <29748bd2-8e37-d336-a00f-5a87750ce6be@oracle.com> Message-ID: <3de55587221d4c0ba62b01068794023f@sap.com> Hi , I don't think the check would do any harm here, but I removed it anyways. No big deal. Thanks for reviewing! Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Donnerstag, 18. Januar 2018 02:18 > To: Volker Simonis > Cc: Baesken, Matthias ; Lindenmaier, Goetz > ; core-libs-dev at openjdk.java.net > Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. > > Hi Volker, > > On 18/01/2018 9:21 AM, Volker Simonis wrote: > > > > David Holmes > > > schrieb am Mi. 17. Jan. 2018 um 22:45: > > > > Hi Goetz, > > > > On 17/01/2018 10:42 PM, Lindenmaier, Goetz wrote: > > > Hi David, > > > > > > thanks for looking at the change. > > > > > > We actually run the aix VM also on OS400. > > > And as OS.java says that it's a " Stripped down version of jtreg > > > internal class com.sun.javatest.regtest.config.OS" > > > I not only grabbed the AIX case from there, but also the OS400 one. > > > I don't think this will do any harm. > > > > Can't speak to what platforms jtreg runs on but OpenJDK doesn't support > > OS/400, so it shouldn't be listed. Sorry. > > > > > > Hi David, > > > > you?re right that OpenJDK doesn?t run natively on OS/400, but OS/400 > > provides an emulation layer called PASE [1] which allows to run native > > AIX application on OS/400. The change proposed by Goetz (which is > > already in JTreg since quite some time) is required to test the OpenJDK > > AIX Port on OS/400 in PASE mode. > > Sorry but I don't see how that one occurrence in > > test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java > > allows for testing on OS/400 when none of the actual tests that examine > the OS have a case for "os400" ?? (Wouldn't it be better to pretend to > be AIX in that case?) > > David > ----- > > > Regards, > > Volker > > > > [1] > > > https://www.ibm.com/support/knowledgecenter/en/ssw_i5_54/rzalf/rzalfi > ntro.htm > > > > > > > > Thanks, > > David > > > > > I'll post it on awt-dev, too. > > > > > > Best regards, > > >? ? Goetz. > > > > > >> -----Original Message----- > > >> From: David Holmes [mailto:david.holmes at oracle.com > > ] > > >> Sent: Mittwoch, 17. Januar 2018 12:55 > > >> To: Lindenmaier, Goetz > >; Volker Simonis > > >> > > > >> Cc: core-libs-dev at openjdk.java.net > > ; Baesken, Matthias > > >> > > > >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests > > to platforms. > > >> > > >> Hi Goetz, > > >> > > >> > > > test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java > > >> > > >> I thought this was for AIX recognition not OS/400 ? ;-) > > >> > > >> Otherwise the AIX changes seem okay. Thought AWT tests should be > > >> reviewed on awt-dev. > > >> > > >> Cheers, > > >> David > > >> > > >> On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: > > >>> Hi, > > >>> > > >>> Volker, thanks for your review. Could someone else please have a > > >>> quick look at this simple fix? > > >>> > > >>> Updated webrev with fixed BadPluginConfigurationTest: > > >>> http://cr.openjdk.java.net/~goetz/wr18/8194869- > jtregAix/webrev.03/ > > >>> > > >>> Best regards, > > >>>? ? ?Goetz. > > >>> > > >>>> -----Original Message----- > > >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com > > ] > > >>>> Sent: Dienstag, 16. Januar 2018 19:28 > > >>>> To: Lindenmaier, Goetz > > > > >>>> Cc: Baesken, Matthias > >; core-libs- > > >>>> dev at openjdk.java.net > > >>>> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > > >> platforms. > > >>>> > > >>>> Hi Goetz, > > >>>> > > >>>> you change looks good. Just one minor nit (no need for a new > > webrev if > > >>>> you agree with my fix :) > > >>>> > > >>>> In BadPluginConfigurationTest.sh you first check if 'mktemp' > > exists as > > >>>> file. This is a useless check because there won't be a file > > "mktemp" > > >>>> in the current exectution directory of the shell script, even > > if the > > >>>> "mktemp" exists domehwere in the PATH. To check for the > > existence of a > > >>>> "mktemp" executable you can use the "type" command: > > >>>> > > >>>> type mktempp> /dev/null 2>&1 > > >>>> > > >>>> if ! [ $? -ne 0 ] ; then > > >>>>? ? ?MKTEMP="/opt/freeware/bin/mktemp" > > >>>> fi > > >>>> > > >>>> Thanks for finally fixing this, > > >>>> Volker > > >>>> > > >>>> > > >>>> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz > > >>>> > > > wrote: > > >>>>> Hi Matthias, > > >>>>> > > >>>>> I didin't mean to do any changes for other platforms, so > > >>>>> I reverted the part you pointed out: > > >>>>> > > http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ > > >>>>> > > >>>>> But scanning through all the scripts: > > >>>>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. > > Strange ... > > >>>>> > > >>>>> Best regards, > > >>>>>? ? ?Goetz. > > >>>>> > > >>>>>> -----Original Message----- > > >>>>>> From: Baesken, Matthias > > >>>>>> Sent: Montag, 15. Januar 2018 09:03 > > >>>>>> To: Lindenmaier, Goetz > >; core-libs- > > >>>>>> dev at openjdk.java.net > > >>>>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt > > tests to > > >>>> platforms. > > >>>>>> > > >>>>>> Hi Goetz, change looks good except > > >>>>>> > > >>>>>> test/jdk/sun/security/tools/keytool/i18n.sh > > >>>>>> > > >>>>>> where the Cygwin PS setting has been changed from ; to : > > ?(probably > > >> this > > >>>>>> was not intended) : > > >>>>>> > > >>>>>>? ? ? ? PS=":" > > >>>>>>? ? ? ? FS="/" > > >>>>>>? ? ? ? ;; > > >>>>>> -? CYGWIN* ) > > >>>>>> -? ? NULL=/dev/null > > >>>>>> -? ? PS=";" > > >>>>>> > > >>>>>> > > >>>>>> (not a Reviewer however). > > >>>>>> > > >>>>>> Best regards, Matthias > > >>>>>> > > >>>>>> > > >>>>>> > > >>>>>>> -----Original Message----- > > >>>>>>> From: Lindenmaier, Goetz > > >>>>>>> Sent: Mittwoch, 10. Januar 2018 09:23 > > >>>>>>> To: core-libs-dev at openjdk.java.net > > > > >>>>>>> Cc: Baesken, Matthias > > > > >>>>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to > > >> platforms. > > >>>>>>> > > >>>>>>> Hi, > > >>>>>>> > > >>>>>>> this change adapts a row of jtreg tests to AIX. > > >>>>>>> Shell tests often switch over all OSes, here the case for > > AIX was > > >>>>>>> missing. PKCSTest needs to know where the libs are on s390. > > >>>>>>> > > >>>>>>> Please review. > > >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869- > jtregAix/webrev/ > > >>>>>>> > > >>>>>>> Best regards, > > >>>>>>>? ? ?Goetz. > > From david.holmes at oracle.com Thu Jan 18 09:07:10 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Jan 2018 19:07:10 +1000 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <4910b950-cc4e-99b9-3b58-100a4076ec1b@oracle.com> References: <961a4bb4-9deb-fc5a-7e21-cffdbdd98dc8@oracle.com> <1d2aa9e9-3b41-04bd-6a54-6b3e419b41dc@oracle.com> <4910b950-cc4e-99b9-3b58-100a4076ec1b@oracle.com> Message-ID: <373965ad-17f3-7e46-024b-22970982f8e2@oracle.com> On 18/01/2018 6:05 PM, Hamlin Li wrote: > On 18/01/2018 3:48 PM, David Holmes wrote: >> On 18/01/2018 5:41 PM, Hamlin Li wrote: >>> Hi David, >>> >>> Sometime we will run test by command "jtreg -timeoutFactor:xxx ...", xxx >> >> Yes but that controls how jtreg manages the execution of the test. > right. >> How is that then being used by the tests that call TestDriver? > I assume you mean TestLibrary rather than TestDriver. Yes - sorry. > If you follow the code in RMID.java, you will find following code, it > calculates the timeout value by timeoutFactor value. > > ??????? long waitTime = (long)(240_000 * TestLibrary.getTimeoutFactor()); > ??????? restartTimeout = (long)(waitTime * 0.9); >> Further if TestDriver limits the timeout to a smaller value, then the >> increase in timeoutFactor won't help at all - people will see a >> timeout, increase the timeoutFactor and still see a timeout! > No, by pass a large timeoutFactor value, I will get test passed rather > than get timeout exception. Sorry you are right - the timeout is limited before scaling by the timeout-factor, so it will increase. Though it's possible for the resulting scaled timeout to still be smaller than the original requested one. But the exception is telling you there is a mismatch between the timing expectations of the caller and the actual behaviour in TestLibrary. Simply placing a ceiling on the timeout value doesn't fix that mismatch. I don't see any point in computeDeadline imposing an arbitrary maximum timeout of 1 hour. (Though it concerns me that tests are trying to set timeouts even longer than an hour!) David ----- >> >> For that matter why is TestDriver even imposing a timeout of its own >> when jtreg imposes an overall timeout? > It's a history topic, if it's necessary we could file another > enhancement or bug to track it. >> In general we have been moving away from test initiated timeouts and >> letting the test harness handle it. > Yes, I agree, that's the reason I refactor it to round down the timeout > to MAX_TIMEOUT_MS rather than throw an exception by test itself. > > Thank you > -Hamlin >> >> Cheers, >> David >> >>> may be a large number than usual, e.g. 100. The reason we supply a >>> large number for timeoutFactor is because we're running test on a >>> very slow machine, or we're running test on a docker environment >>> which has very limited resource, without a large timeoutFactor value, >>> the test will finally get timeout exception. At this situation, it's >>> NOT reasonable for user to know the exact timeoutFactor by which the >>> calculated result will be MAX_TIMEOUT_MS, what's helpful is to let >>> user supply a large enough timeoutFactor value and round down the >>> timeout value to MAX_TIMEOUT_MS automatically, rather than throw an >>> exception. >>> >>> Thank you >>> >>> -Hamlin >>> >>> >>> On 18/01/2018 3:15 PM, David Holmes wrote: >>>> Hi Hamlin, >>>> >>>> This should probably be reviewed on core-libs-dev. I don't think >>>> jdk-dev is intended for code reviews. >>>> >>>> On 18/01/2018 4:59 PM, Hamlin Li wrote: >>>>> Would you please review the following patch? >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>>>> >>>>> webrev as below. >>>> >>>> I don't agree with this. Whatever is passing the incorrect timeout >>>> to the TestLibrary should be fixed. The bug report needs more >>>> information about where the incorrect value is coming from and why. >>>> >>>> Cheers, >>>> David >>>> >>>>> >>>>> Thank you >>>>> >>>>> -Hamlin >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>>>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >>>>> 20:07:50 2018 -0800 >>>>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >>>>> 14:54:50 2018 +0800 >>>>> @@ -188,8 +188,12 @@ >>>>> ????? public static long computeDeadline(long timestamp, long >>>>> timeout) { >>>>> ????????? final long MAX_TIMEOUT_MS = 3_600_000L; >>>>> >>>>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>>>> +??????? if (timeout < 0L) { >>>>> ????????????? throw new IllegalArgumentException("timeout " + >>>>> timeout + "ms out of range"); >>>>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>>>> +??????????? System.out.format("timeout value(%d) exceeds >>>>> MAX_TIMEOUT_MS(%d), " >>>>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>>>> MAX_TIMEOUT_MS); >>>>> +??????????? timeout = MAX_TIMEOUT_MS; >>>>> ????????? } >>>>> >>>>> ????????? return timestamp + (long)(timeout * getTimeoutFactor()); >>>>> >>> > From david.holmes at oracle.com Thu Jan 18 09:09:45 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Jan 2018 19:09:45 +1000 Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. In-Reply-To: <3de55587221d4c0ba62b01068794023f@sap.com> References: <16aa381c9347437e8e270fbf6be0a6ce@sap.com> <9668a567570d4baa82a04532ee1db52f@sap.com> <36031bad-d0ee-5bdf-ef10-c39bf2c7698e@oracle.com> <5074508851404c77ae85ee86035866b5@sap.com> <29748bd2-8e37-d336-a00f-5a87750ce6be@oracle.com> <3de55587221d4c0ba62b01068794023f@sap.com> Message-ID: <6e443e89-283d-aa39-76d8-0a8eecc88bf7@oracle.com> On 18/01/2018 7:00 PM, Lindenmaier, Goetz wrote: > Hi , > > I don't think the check would do any harm here, > but I removed it anyways. No big deal. Thanks. But I'm still curious - if you really want to test on OS/400+PASE wouldn't you want to pretend it is actually AIX? :) Cheers, David > Thanks for reviewing! > > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Donnerstag, 18. Januar 2018 02:18 >> To: Volker Simonis >> Cc: Baesken, Matthias ; Lindenmaier, Goetz >> ; core-libs-dev at openjdk.java.net >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to platforms. >> >> Hi Volker, >> >> On 18/01/2018 9:21 AM, Volker Simonis wrote: >>> >>> David Holmes > > >>> schrieb am Mi. 17. Jan. 2018 um 22:45: >>> >>> Hi Goetz, >>> >>> On 17/01/2018 10:42 PM, Lindenmaier, Goetz wrote: >>> > Hi David, >>> > >>> > thanks for looking at the change. >>> > >>> > We actually run the aix VM also on OS400. >>> > And as OS.java says that it's a " Stripped down version of jtreg >>> > internal class com.sun.javatest.regtest.config.OS" >>> > I not only grabbed the AIX case from there, but also the OS400 one. >>> > I don't think this will do any harm. >>> >>> Can't speak to what platforms jtreg runs on but OpenJDK doesn't support >>> OS/400, so it shouldn't be listed. Sorry. >>> >>> >>> Hi David, >>> >>> you?re right that OpenJDK doesn?t run natively on OS/400, but OS/400 >>> provides an emulation layer called PASE [1] which allows to run native >>> AIX application on OS/400. The change proposed by Goetz (which is >>> already in JTreg since quite some time) is required to test the OpenJDK >>> AIX Port on OS/400 in PASE mode. >> >> Sorry but I don't see how that one occurrence in >> >> test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java >> >> allows for testing on OS/400 when none of the actual tests that examine >> the OS have a case for "os400" ?? (Wouldn't it be better to pretend to >> be AIX in that case?) >> >> David >> ----- >> >>> Regards, >>> Volker >>> >>> [1] >>> >> https://www.ibm.com/support/knowledgecenter/en/ssw_i5_54/rzalf/rzalfi >> ntro.htm >>> >>> >>> >>> Thanks, >>> David >>> >>> > I'll post it on awt-dev, too. >>> > >>> > Best regards, >>> >? ? Goetz. >>> > >>> >> -----Original Message----- >>> >> From: David Holmes [mailto:david.holmes at oracle.com >>> ] >>> >> Sent: Mittwoch, 17. Januar 2018 12:55 >>> >> To: Lindenmaier, Goetz >> >; Volker Simonis >>> >> > >>> >> Cc: core-libs-dev at openjdk.java.net >>> ; Baesken, Matthias >>> >> > >>> >> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests >>> to platforms. >>> >> >>> >> Hi Goetz, >>> >> >>> >> >>> >> test/failure_handler/src/share/classes/jdk/test/failurehandler/jtreg/OS.java >>> >> >>> >> I thought this was for AIX recognition not OS/400 ? ;-) >>> >> >>> >> Otherwise the AIX changes seem okay. Thought AWT tests should be >>> >> reviewed on awt-dev. >>> >> >>> >> Cheers, >>> >> David >>> >> >>> >> On 17/01/2018 9:39 PM, Lindenmaier, Goetz wrote: >>> >>> Hi, >>> >>> >>> >>> Volker, thanks for your review. Could someone else please have a >>> >>> quick look at this simple fix? >>> >>> >>> >>> Updated webrev with fixed BadPluginConfigurationTest: >>> >>> http://cr.openjdk.java.net/~goetz/wr18/8194869- >> jtregAix/webrev.03/ >>> >>> >>> >>> Best regards, >>> >>>? ? ?Goetz. >>> >>> >>> >>>> -----Original Message----- >>> >>>> From: Volker Simonis [mailto:volker.simonis at gmail.com >>> ] >>> >>>> Sent: Dienstag, 16. Januar 2018 19:28 >>> >>>> To: Lindenmaier, Goetz >> > >>> >>>> Cc: Baesken, Matthias >> >; core-libs- >>> >>>> dev at openjdk.java.net >>> >>>> Subject: Re: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >>> >> platforms. >>> >>>> >>> >>>> Hi Goetz, >>> >>>> >>> >>>> you change looks good. Just one minor nit (no need for a new >>> webrev if >>> >>>> you agree with my fix :) >>> >>>> >>> >>>> In BadPluginConfigurationTest.sh you first check if 'mktemp' >>> exists as >>> >>>> file. This is a useless check because there won't be a file >>> "mktemp" >>> >>>> in the current exectution directory of the shell script, even >>> if the >>> >>>> "mktemp" exists domehwere in the PATH. To check for the >>> existence of a >>> >>>> "mktemp" executable you can use the "type" command: >>> >>>> >>> >>>> type mktempp> /dev/null 2>&1 >>> >>>> >>> >>>> if ! [ $? -ne 0 ] ; then >>> >>>>? ? ?MKTEMP="/opt/freeware/bin/mktemp" >>> >>>> fi >>> >>>> >>> >>>> Thanks for finally fixing this, >>> >>>> Volker >>> >>>> >>> >>>> >>> >>>> On Mon, Jan 15, 2018 at 12:12 PM, Lindenmaier, Goetz >>> >>>> > > >>> wrote: >>> >>>>> Hi Matthias, >>> >>>>> >>> >>>>> I didin't mean to do any changes for other platforms, so >>> >>>>> I reverted the part you pointed out: >>> >>>>> >>> http://cr.openjdk.java.net/~goetz/wr18/8194869-jtregAix/webrev.02/ >>> >>>>> >>> >>>>> But scanning through all the scripts: >>> >>>>> Sometimes PS on Cygwin is ';' and sometimes it is ':'. >>> Strange ... >>> >>>>> >>> >>>>> Best regards, >>> >>>>>? ? ?Goetz. >>> >>>>> >>> >>>>>> -----Original Message----- >>> >>>>>> From: Baesken, Matthias >>> >>>>>> Sent: Montag, 15. Januar 2018 09:03 >>> >>>>>> To: Lindenmaier, Goetz >> >; core-libs- >>> >>>>>> dev at openjdk.java.net >>> >>>>>> Subject: RE: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt >>> tests to >>> >>>> platforms. >>> >>>>>> >>> >>>>>> Hi Goetz, change looks good except >>> >>>>>> >>> >>>>>> test/jdk/sun/security/tools/keytool/i18n.sh >>> >>>>>> >>> >>>>>> where the Cygwin PS setting has been changed from ; to : >>> ?(probably >>> >> this >>> >>>>>> was not intended) : >>> >>>>>> >>> >>>>>>? ? ? ? PS=":" >>> >>>>>>? ? ? ? FS="/" >>> >>>>>>? ? ? ? ;; >>> >>>>>> -? CYGWIN* ) >>> >>>>>> -? ? NULL=/dev/null >>> >>>>>> -? ? PS=";" >>> >>>>>> >>> >>>>>> >>> >>>>>> (not a Reviewer however). >>> >>>>>> >>> >>>>>> Best regards, Matthias >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>>> -----Original Message----- >>> >>>>>>> From: Lindenmaier, Goetz >>> >>>>>>> Sent: Mittwoch, 10. Januar 2018 09:23 >>> >>>>>>> To: core-libs-dev at openjdk.java.net >>> >>> >>>>>>> Cc: Baesken, Matthias >> > >>> >>>>>>> Subject: RFR(S): 8194869: [TESTBUG][aix, s390] Adapt tests to >>> >> platforms. >>> >>>>>>> >>> >>>>>>> Hi, >>> >>>>>>> >>> >>>>>>> this change adapts a row of jtreg tests to AIX. >>> >>>>>>> Shell tests often switch over all OSes, here the case for >>> AIX was >>> >>>>>>> missing. PKCSTest needs to know where the libs are on s390. >>> >>>>>>> >>> >>>>>>> Please review. >>> >>>>>>> http://cr.openjdk.java.net/~goetz/wr18/8194869- >> jtregAix/webrev/ >>> >>>>>>> >>> >>>>>>> Best regards, >>> >>>>>>>? ? ?Goetz. >>> From david.holmes at oracle.com Thu Jan 18 11:20:32 2018 From: david.holmes at oracle.com (David Holmes) Date: Thu, 18 Jan 2018 21:20:32 +1000 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <7f5babcd-6b37-9af3-581b-f061718350e8@oracle.com> References: <961a4bb4-9deb-fc5a-7e21-cffdbdd98dc8@oracle.com> <1d2aa9e9-3b41-04bd-6a54-6b3e419b41dc@oracle.com> <4910b950-cc4e-99b9-3b58-100a4076ec1b@oracle.com> <373965ad-17f3-7e46-024b-22970982f8e2@oracle.com> <7f5babcd-6b37-9af3-581b-f061718350e8@oracle.com> Message-ID: <0c1ecf1f-6c67-ece7-e519-9a5305cde1bf@oracle.com> On 18/01/2018 7:21 PM, Hamlin Li wrote: > > > On 18/01/2018 5:07 PM, David Holmes wrote: >> On 18/01/2018 6:05 PM, Hamlin Li wrote: >>> On 18/01/2018 3:48 PM, David Holmes wrote: >>>> On 18/01/2018 5:41 PM, Hamlin Li wrote: >>>>> Hi David, >>>>> >>>>> Sometime we will run test by command "jtreg -timeoutFactor:xxx >>>>> ...", xxx >>>> >>>> Yes but that controls how jtreg manages the execution of the test. >>> right. >>>> How is that then being used by the tests that call TestDriver? >>> I assume you mean TestLibrary rather than TestDriver. >> >> Yes - sorry. >> >>> If you follow the code in RMID.java, you will find following code, it >>> calculates the timeout value by timeoutFactor value. >>> >>> ???????? long waitTime = (long)(240_000 * >>> TestLibrary.getTimeoutFactor()); >>> ???????? restartTimeout = (long)(waitTime * 0.9); >>>> Further if TestDriver limits the timeout to a smaller value, then >>>> the increase in timeoutFactor won't help at all - people will see a >>>> timeout, increase the timeoutFactor and still see a timeout! >>> No, by pass a large timeoutFactor value, I will get test passed >>> rather than get timeout exception. >> >> Sorry you are right - the timeout is limited before scaling by the >> timeout-factor, so it will increase. Though it's possible for the >> resulting scaled timeout to still be smaller than the original >> requested one. >> >> But the exception is telling you there is a mismatch between the >> timing expectations of the caller and the actual behaviour in >> TestLibrary. Simply placing a ceiling on the timeout value doesn't fix >> that mismatch. > The patch is not to fix any mismatch, it's just let user to supply a > large enough value for timeoutFactor (if it's accepted by jtreg) and let > test pass, it's meaningless to throw an exception. By dropping the exception you're trying to address the mismatch. >> I don't see any point in computeDeadline imposing an arbitrary maximum >> timeout of 1 hour. (Though it concerns me that tests are trying to set >> timeouts even longer than an hour!) > I neither, but it's history code, currently not necessary to touch it, > how do you think about it? I think the max timeout is pointless. I think other code trying to set timeouts bigger than an hour is a real problem! Let's what if anyone else has an opinion. Thanks, David > Thank you > -Hamlin >> >> David >> ----- >> >>>> >>>> For that matter why is TestDriver even imposing a timeout of its own >>>> when jtreg imposes an overall timeout? >>> It's a history topic, if it's necessary we could file another >>> enhancement or bug to track it. >>>> In general we have been moving away from test initiated timeouts and >>>> letting the test harness handle it. >>> Yes, I agree, that's the reason I refactor it to round down the >>> timeout to MAX_TIMEOUT_MS rather than throw an exception by test itself. >>> >>> Thank you >>> -Hamlin >>>> >>>> Cheers, >>>> David >>>> >>>>> may be a large number than usual, e.g. 100. The reason we supply a >>>>> large number for timeoutFactor is because we're running test on a >>>>> very slow machine, or we're running test on a docker environment >>>>> which has very limited resource, without a large timeoutFactor >>>>> value, the test will finally get timeout exception. At this >>>>> situation, it's NOT reasonable for user to know the exact >>>>> timeoutFactor by which the calculated result will be >>>>> MAX_TIMEOUT_MS, what's helpful is to let user supply a large enough >>>>> timeoutFactor value and round down the timeout value to >>>>> MAX_TIMEOUT_MS automatically, rather than throw an exception. >>>>> >>>>> Thank you >>>>> >>>>> -Hamlin >>>>> >>>>> >>>>> On 18/01/2018 3:15 PM, David Holmes wrote: >>>>>> Hi Hamlin, >>>>>> >>>>>> This should probably be reviewed on core-libs-dev. I don't think >>>>>> jdk-dev is intended for code reviews. >>>>>> >>>>>> On 18/01/2018 4:59 PM, Hamlin Li wrote: >>>>>>> Would you please review the following patch? >>>>>>> >>>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>>>>>> >>>>>>> webrev as below. >>>>>> >>>>>> I don't agree with this. Whatever is passing the incorrect timeout >>>>>> to the TestLibrary should be fixed. The bug report needs more >>>>>> information about where the incorrect value is coming from and why. >>>>>> >>>>>> Cheers, >>>>>> David >>>>>> >>>>>>> >>>>>>> Thank you >>>>>>> >>>>>>> -Hamlin >>>>>>> >>>>>>> ------------------------------------------------------------------------ >>>>>>> >>>>>>> >>>>>>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>>>>>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java Wed Jan 17 >>>>>>> 20:07:50 2018 -0800 >>>>>>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java Thu Jan 18 >>>>>>> 14:54:50 2018 +0800 >>>>>>> @@ -188,8 +188,12 @@ >>>>>>> ????? public static long computeDeadline(long timestamp, long >>>>>>> timeout) { >>>>>>> ????????? final long MAX_TIMEOUT_MS = 3_600_000L; >>>>>>> >>>>>>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>>>>>> +??????? if (timeout < 0L) { >>>>>>> ????????????? throw new IllegalArgumentException("timeout " + >>>>>>> timeout + "ms out of range"); >>>>>>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>>>>>> +??????????? System.out.format("timeout value(%d) exceeds >>>>>>> MAX_TIMEOUT_MS(%d), " >>>>>>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>>>>>> MAX_TIMEOUT_MS); >>>>>>> +??????????? timeout = MAX_TIMEOUT_MS; >>>>>>> ????????? } >>>>>>> >>>>>>> ????????? return timestamp + (long)(timeout * getTimeoutFactor()); >>>>>>> >>>>> >>> > From huaming.li at oracle.com Thu Jan 18 09:21:04 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Thu, 18 Jan 2018 17:21:04 +0800 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <373965ad-17f3-7e46-024b-22970982f8e2@oracle.com> References: <961a4bb4-9deb-fc5a-7e21-cffdbdd98dc8@oracle.com> <1d2aa9e9-3b41-04bd-6a54-6b3e419b41dc@oracle.com> <4910b950-cc4e-99b9-3b58-100a4076ec1b@oracle.com> <373965ad-17f3-7e46-024b-22970982f8e2@oracle.com> Message-ID: <7f5babcd-6b37-9af3-581b-f061718350e8@oracle.com> On 18/01/2018 5:07 PM, David Holmes wrote: > On 18/01/2018 6:05 PM, Hamlin Li wrote: >> On 18/01/2018 3:48 PM, David Holmes wrote: >>> On 18/01/2018 5:41 PM, Hamlin Li wrote: >>>> Hi David, >>>> >>>> Sometime we will run test by command "jtreg -timeoutFactor:xxx >>>> ...", xxx >>> >>> Yes but that controls how jtreg manages the execution of the test. >> right. >>> How is that then being used by the tests that call TestDriver? >> I assume you mean TestLibrary rather than TestDriver. > > Yes - sorry. > >> If you follow the code in RMID.java, you will find following code, it >> calculates the timeout value by timeoutFactor value. >> >> ???????? long waitTime = (long)(240_000 * >> TestLibrary.getTimeoutFactor()); >> ???????? restartTimeout = (long)(waitTime * 0.9); >>> Further if TestDriver limits the timeout to a smaller value, then >>> the increase in timeoutFactor won't help at all - people will see a >>> timeout, increase the timeoutFactor and still see a timeout! >> No, by pass a large timeoutFactor value, I will get test passed >> rather than get timeout exception. > > Sorry you are right - the timeout is limited before scaling by the > timeout-factor, so it will increase. Though it's possible for the > resulting scaled timeout to still be smaller than the original > requested one. > > But the exception is telling you there is a mismatch between the > timing expectations of the caller and the actual behaviour in > TestLibrary. Simply placing a ceiling on the timeout value doesn't fix > that mismatch. The patch is not to fix any mismatch, it's just let user to supply a large enough value for timeoutFactor (if it's accepted by jtreg) and let test pass, it's meaningless to throw an exception. > I don't see any point in computeDeadline imposing an arbitrary maximum > timeout of 1 hour. (Though it concerns me that tests are trying to set > timeouts even longer than an hour!) I neither, but it's history code, currently not necessary to touch it, how do you think about it? Thank you -Hamlin > > David > ----- > >>> >>> For that matter why is TestDriver even imposing a timeout of its own >>> when jtreg imposes an overall timeout? >> It's a history topic, if it's necessary we could file another >> enhancement or bug to track it. >>> In general we have been moving away from test initiated timeouts and >>> letting the test harness handle it. >> Yes, I agree, that's the reason I refactor it to round down the >> timeout to MAX_TIMEOUT_MS rather than throw an exception by test itself. >> >> Thank you >> -Hamlin >>> >>> Cheers, >>> David >>> >>>> may be a large number than usual, e.g. 100. The reason we supply a >>>> large number for timeoutFactor is because we're running test on a >>>> very slow machine, or we're running test on a docker environment >>>> which has very limited resource, without a large timeoutFactor >>>> value, the test will finally get timeout exception. At this >>>> situation, it's NOT reasonable for user to know the exact >>>> timeoutFactor by which the calculated result will be >>>> MAX_TIMEOUT_MS, what's helpful is to let user supply a large enough >>>> timeoutFactor value and round down the timeout value to >>>> MAX_TIMEOUT_MS automatically, rather than throw an exception. >>>> >>>> Thank you >>>> >>>> -Hamlin >>>> >>>> >>>> On 18/01/2018 3:15 PM, David Holmes wrote: >>>>> Hi Hamlin, >>>>> >>>>> This should probably be reviewed on core-libs-dev. I don't think >>>>> jdk-dev is intended for code reviews. >>>>> >>>>> On 18/01/2018 4:59 PM, Hamlin Li wrote: >>>>>> Would you please review the following patch? >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>>>>> >>>>>> webrev as below. >>>>> >>>>> I don't agree with this. Whatever is passing the incorrect timeout >>>>> to the TestLibrary should be fixed. The bug report needs more >>>>> information about where the incorrect value is coming from and why. >>>>> >>>>> Cheers, >>>>> David >>>>> >>>>>> >>>>>> Thank you >>>>>> >>>>>> -Hamlin >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> >>>>>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>>>>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java Wed Jan 17 >>>>>> 20:07:50 2018 -0800 >>>>>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java Thu Jan 18 >>>>>> 14:54:50 2018 +0800 >>>>>> @@ -188,8 +188,12 @@ >>>>>> ????? public static long computeDeadline(long timestamp, long >>>>>> timeout) { >>>>>> ????????? final long MAX_TIMEOUT_MS = 3_600_000L; >>>>>> >>>>>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>>>>> +??????? if (timeout < 0L) { >>>>>> ????????????? throw new IllegalArgumentException("timeout " + >>>>>> timeout + "ms out of range"); >>>>>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>>>>> +??????????? System.out.format("timeout value(%d) exceeds >>>>>> MAX_TIMEOUT_MS(%d), " >>>>>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>>>>> MAX_TIMEOUT_MS); >>>>>> +??????????? timeout = MAX_TIMEOUT_MS; >>>>>> ????????? } >>>>>> >>>>>> ????????? return timestamp + (long)(timeout * getTimeoutFactor()); >>>>>> >>>> >> From daniel.fuchs at oracle.com Thu Jan 18 15:12:27 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Thu, 18 Jan 2018 15:12:27 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat Message-ID: Hi, Please find below a proposed fix for: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat https://bugs.openjdk.java.net/browse/JDK-8195096 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.00/ The issue appeared after the fix for https://bugs.openjdk.java.net/browse/JDK-8191033 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works Tomcat is apparently reusing the ".handlers" property, to specify class names in a format that the LogManager cannot understand (hence the exceptions logged on the console). The proposed fix is to add a protected hook that subclass of LogManager could extend to turn off parsing of the ".handler" property. best regards, -- daniel From goetz.lindenmaier at sap.com Thu Jan 18 16:24:45 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Thu, 18 Jan 2018 16:24:45 +0000 Subject: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError: HelpFlagsTest failed: In-Reply-To: <45e66d48b4224ef8a5342d8b64a2b24c@sap.com> References: <19A6164F-1766-4C30-8A34-05B1185D9525@oracle.com> <45e66d48b4224ef8a5342d8b64a2b24c@sap.com> Message-ID: Posting this to core-libs-dev. Please review http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ which excludes the Oracle proprietary tools from the test. I can not verify this. Best regards, Goetz. > -----Original Message----- > From: jdk-dev [mailto:jdk-dev-bounces at openjdk.java.net] On Behalf Of > Lindenmaier, Goetz > Sent: Donnerstag, 18. Januar 2018 17:02 > To: Weijun Wang > Cc: jdk-dev at openjdk.java.net > Subject: RE: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails > with java.lang.AssertionError: HelpFlagsTest failed: > > Hi, > > Added ssvagent: > http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ > > Best regards, > Goetz. > > > -----Original Message----- > > From: Weijun Wang [mailto:weijun.wang at oracle.com] > > Sent: Donnerstag, 18. Januar 2018 16:44 > > To: Lindenmaier, Goetz > > Cc: Kumar Srinivasan ; jdk- > > dev at openjdk.java.net > > Subject: Re: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails > > with java.lang.AssertionError: HelpFlagsTest failed: > > > > On Windows, there is also a ssvagent.exe. > > > > > On Jan 18, 2018, at 11:37 PM, Lindenmaier, Goetz > > wrote: > > > > > > Hi, > > > > > > the most simple fix is to just exclude the four tools: > > > http://cr.openjdk.java.net/~goetz/wr18/8195663- > fixHelpTest/webrev.01/ > > > Unfortunately I can't test this, as my build does not contain these > > > tools. > > > > > > If somebody tells me how to build these tools, I will adapt > > > the test to check them, and eventually adapt the tools to > > > CSR 8191477 https://bugs.openjdk.java.net/browse/JDK-8191477. > > > > > > What list should this be reviewed on? I just posted to jdk-dev for now. > > > > > > Best regards, > > > Goetz. From claes.redestad at oracle.com Thu Jan 18 17:51:51 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Thu, 18 Jan 2018 18:51:51 +0100 Subject: [11] RFR: 8193128: Reduce number of implementation classes returned by List/Set/Map.of() In-Reply-To: <1bcb7c02-5ce0-be4f-e28c-842f229004bc@gmail.com> References: <7d367ca5-2915-bd80-8247-c748260f93c8@oracle.com> <461185061.1582235.1512601042847.JavaMail.zimbra@u-pem.fr> <8746a7ab-4c1c-98fc-a0ba-05594bce794c@oracle.com> <71b8baf0-0f4b-2e64-36d9-610e2286b59e@oracle.com> <4ec012ce-cd0c-003b-c20a-4c77e317e40f@gmail.com> <14910571-3cc8-75a5-9f80-8d633b23ac4b@oracle.com> <94ea8e2e-418c-4408-2dc2-7bea1ca099d4@gmail.com> <2845d01f-742a-bcfe-330c-5fbd586eaf88@oracle.com> <1bcb7c02-5ce0-be4f-e28c-842f229004bc@gmail.com> Message-ID: <98bb59c5-c091-a90d-723e-1a0ce1d6df93@oracle.com> Hi, noticed I had replied off-list to Peter by accident. Sorry for the unintended delay.. I think we can settle for the simplified equals (as previously outlined) along with an abstract hashCode to remind ourselves it needs to be overridden: http://cr.openjdk.java.net/~redestad/8193128/open.06/ As for the race that's can make equals() return true when the other set is cleared after calling size() on it but before iterating over the collection, then I think that's behavior you could provoke with most of the non-j.u.c Set implementations in the JDK (which typically defer to AbstractSet::equals) - including the existing Set.of() implementation - so I consider it out of scope for this improvement. Thanks! /Claes On 2018-01-09 13:27, Peter Levart wrote: > Hi Claes, > > On 01/09/2018 02:17 AM, Claes Redestad wrote: >> If we took it a step further we could save us the trouble of throwing >> and swallowing NPEs in >> AbstractImmutableSet::equals when the other set contains null, which >> might be an optimization >> in that case: >> >> ??????? @Override >> ??????? public boolean equals(Object o) { >> ??????????? if (o == this) >> ??????????????? return true; >> >> ??????????? if (!(o instanceof Set)) >> ??????????????? return false; >> >> ??????????? Collection c = (Collection) o; >> ??????????? if (c.size() != size()) >> ??????????????? return false; >> >> ??????????? for (Object e : c) >> ??????????????? if (e == null || !contains(e)) >> ??????????????????? return false; >> ??????????? return true; >> ??????? } >> >> It might be rare enough that it's not worth bothering with, though. > > That's nicer in my opinion. What to do with ClassCastException from > contains(e)? Perhaps just document that any future implementation that > might throw it from contains() should also modify the equals() method > and catch it there. > > And if you already do explicit iteration over elements, you could as > well count them and do the size check with counted size at the end. > Imagine the following scenario: > > ConcurrentHashMap m = new ChocurrentHashMap(); > m.put(1, 1); > m.put(2, 2); > m.put(3, 3); > > Set s = Set.of("a", "b", "c"); > > Thread 1: > > s.equals(m.keySet()) > > Thread 2: > > m.clear(); > > It is surprising when Thread 1 encounters 'true' as the result of the > equals() above which can happen when code checks for sizes separately. > Iterating CHM.keySet() does not provide a snapshot of the content > either, but counting iterated and compared elements nevertheless > exhibits less surprising behavior. I don't know if it matters though. > > Regards, Peter > From jason_mehrens at hotmail.com Thu Jan 18 21:19:37 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Thu, 18 Jan 2018 21:19:37 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: Message-ID: Daniel, As long as the org.apache.juli.ClassLoaderLogManager overrides getProperty it shouldn't really matter what the value format is in the file as long as it is translated on return. Is there a code path in j.u.l.LogManager that doesn't call getProperty? If so I would think that is the core issue. Jason ________________________________________ From: core-libs-dev on behalf of Daniel Fuchs Sent: Thursday, January 18, 2018 9:12 AM To: core-libs-dev Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat Hi, Please find below a proposed fix for: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat https://bugs.openjdk.java.net/browse/JDK-8195096 webrev: http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.00/ The issue appeared after the fix for https://bugs.openjdk.java.net/browse/JDK-8191033 8191033: Regression in logging.properties: specifying .handlers= for root logger (instead of handlers=) no longer works Tomcat is apparently reusing the ".handlers" property, to specify class names in a format that the LogManager cannot understand (hence the exceptions logged on the console). The proposed fix is to add a protected hook that subclass of LogManager could extend to turn off parsing of the ".handler" property. best regards, -- daniel From jeremymanson at google.com Fri Jan 19 01:28:47 2018 From: jeremymanson at google.com (Jeremy Manson) Date: Thu, 18 Jan 2018 17:28:47 -0800 Subject: Unsafe and JPMS Message-ID: Hey folks, I know this has come up before, but I wanted to revive the question. As we play more and more with JPMS and Java 9, we're finding more and more places where Unsafe is now the only feasible way to achieve what we want to achieve. The big use case is, of course, access to the internals of modularized libraries in ways that would have used standard reflection in Java 8 and earlier, but for which we now fail with an exception. The answer of adding command line flags doesn't really work for general purpose libraries. If I want to do this in (for example) Guava, asking every Guava user to add add-opens on the command line is a complete non-starter. Many people have no idea what libraries they are using. The answer of creating an agent is also a non-starter for similar reasons. The answer of using privateLookupIn assumes that you have access to or control over the target class, which is often not the case, especially if you are introspecting into JDK internals. As people within Google try Java 9, this is coming up in quite a bit of our infrastructure, especially in diagnostic code. The latest place I've found it was a tool that printed out / counted the number of objects reachable from a root object; it died when it got into a JDK library. I've noticed quite a few people using Unsafe as a workaround in OSS libraries, too: Netty , Kryo , cglib . So... OpenJDK has this commitment not to replace Unsafe without providing supported replacements for its functionality and a transition period. Does that include the functionality that Unsafe can break module encapsulation? Thanks! Jeremy From david.holmes at oracle.com Fri Jan 19 01:57:50 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Jan 2018 11:57:50 +1000 Subject: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError: HelpFlagsTest failed: In-Reply-To: References: <19A6164F-1766-4C30-8A34-05B1185D9525@oracle.com> <45e66d48b4224ef8a5342d8b64a2b24c@sap.com> Message-ID: <1b30b8eb-1472-2d9f-a51b-fcd5b01f944d@oracle.com> Hi Goetz, On 19/01/2018 2:24 AM, Lindenmaier, Goetz wrote: > Posting this to core-libs-dev. > > Please review > http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ > which excludes the Oracle proprietary tools from the test. > I can not verify this. Wouldn't a whitelist work better here than a blacklist? Other distributions might add other tools (in theory). David > Best regards, > Goetz. > > >> -----Original Message----- >> From: jdk-dev [mailto:jdk-dev-bounces at openjdk.java.net] On Behalf Of >> Lindenmaier, Goetz >> Sent: Donnerstag, 18. Januar 2018 17:02 >> To: Weijun Wang >> Cc: jdk-dev at openjdk.java.net >> Subject: RE: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails >> with java.lang.AssertionError: HelpFlagsTest failed: >> >> Hi, >> >> Added ssvagent: >> http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ >> >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: Weijun Wang [mailto:weijun.wang at oracle.com] >>> Sent: Donnerstag, 18. Januar 2018 16:44 >>> To: Lindenmaier, Goetz >>> Cc: Kumar Srinivasan ; jdk- >>> dev at openjdk.java.net >>> Subject: Re: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails >>> with java.lang.AssertionError: HelpFlagsTest failed: >>> >>> On Windows, there is also a ssvagent.exe. >>> >>>> On Jan 18, 2018, at 11:37 PM, Lindenmaier, Goetz >>> wrote: >>>> >>>> Hi, >>>> >>>> the most simple fix is to just exclude the four tools: >>>> http://cr.openjdk.java.net/~goetz/wr18/8195663- >> fixHelpTest/webrev.01/ >>>> Unfortunately I can't test this, as my build does not contain these >>>> tools. >>>> >>>> If somebody tells me how to build these tools, I will adapt >>>> the test to check them, and eventually adapt the tools to >>>> CSR 8191477 https://bugs.openjdk.java.net/browse/JDK-8191477. >>>> >>>> What list should this be reviewed on? I just posted to jdk-dev for now. >>>> >>>> Best regards, >>>> Goetz. > From huaming.li at oracle.com Fri Jan 19 05:04:09 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Fri, 19 Jan 2018 13:04:09 +0800 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: References: Message-ID: <02311325-7c1d-c76d-a004-c944dab420c4@oracle.com> Hi Roger, David Please check the updated webrev at: http://cr.openjdk.java.net/~mli/8194284/webrev.00/ Thank you -Hamlin On 18/01/2018 10:33 PM, Roger Riggs wrote: > Hi Hamlin, > > The base bug is that the timeoutFactor is being applied twice, once in > the RMID constructor > and again in computeDeadline.? It is better to cleanup the > implementation of the test library > than to fudge the values.? Either respect the timeouts passed in > (remove the scaling from computeDeadline) > or consistently leave it to the lower level routines.? Pick 1. > > Thanks, Roger > > > On 1/18/2018 1:59 AM, Hamlin Li wrote: >> Would you please review the following patch? >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >> >> webrev as below. >> >> >> Thank you >> >> -Hamlin >> >> ------------------------------------------------------------------------ >> >> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >> 20:07:50 2018 -0800 >> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >> 14:54:50 2018 +0800 >> @@ -188,8 +188,12 @@ >> ???? public static long computeDeadline(long timestamp, long timeout) { >> ???????? final long MAX_TIMEOUT_MS = 3_600_000L; >> >> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >> +??????? if (timeout < 0L) { >> ???????????? throw new IllegalArgumentException("timeout " + timeout >> + "ms out of range"); >> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >> +??????????? System.out.format("timeout value(%d) exceeds >> MAX_TIMEOUT_MS(%d), " >> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >> MAX_TIMEOUT_MS); >> +??????????? timeout = MAX_TIMEOUT_MS; >> ???????? } >> >> ???????? return timestamp + (long)(timeout * getTimeoutFactor()); >> > From david.holmes at oracle.com Fri Jan 19 06:11:07 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Jan 2018 16:11:07 +1000 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <02311325-7c1d-c76d-a004-c944dab420c4@oracle.com> References: <02311325-7c1d-c76d-a004-c944dab420c4@oracle.com> Message-ID: <5a49a3e7-cb4c-f52a-1d7e-8c1178addfad@oracle.com> Hi Hamlin, On 19/01/2018 3:04 PM, Hamlin Li wrote: > Hi Roger, David > > Please check the updated webrev at: > http://cr.openjdk.java.net/~mli/8194284/webrev.00/ RMID.java: This comment no longer makes sense: // when restart rmid, it may take more time than usual because of // "port in use" by a possible interloper (check JDK-8168975), // so need to set a longer timeout for restart. ! private static final long RESTART_TIMEOUT = (long)(TIMEOUT_BASE * 0.9); Actually I'm not sure it originally made sense - longer than what? But as it stands RESTART_TIMEOUT is smaller than TIMEOUT_BASE so the comment really seems odd. Perhaps 8168975 will shed some light on the intent. ?? The TestLibrary change looks good. Thanks, David > Thank you > > -Hamlin > > > On 18/01/2018 10:33 PM, Roger Riggs wrote: >> Hi Hamlin, >> >> The base bug is that the timeoutFactor is being applied twice, once in >> the RMID constructor >> and again in computeDeadline.? It is better to cleanup the >> implementation of the test library >> than to fudge the values.? Either respect the timeouts passed in >> (remove the scaling from computeDeadline) >> or consistently leave it to the lower level routines.? Pick 1. >> >> Thanks, Roger >> >> >> On 1/18/2018 1:59 AM, Hamlin Li wrote: >>> Would you please review the following patch? >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>> >>> webrev as below. >>> >>> >>> Thank you >>> >>> -Hamlin >>> >>> ------------------------------------------------------------------------ >>> >>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >>> 20:07:50 2018 -0800 >>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >>> 14:54:50 2018 +0800 >>> @@ -188,8 +188,12 @@ >>> ???? public static long computeDeadline(long timestamp, long timeout) { >>> ???????? final long MAX_TIMEOUT_MS = 3_600_000L; >>> >>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>> +??????? if (timeout < 0L) { >>> ???????????? throw new IllegalArgumentException("timeout " + timeout >>> + "ms out of range"); >>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>> +??????????? System.out.format("timeout value(%d) exceeds >>> MAX_TIMEOUT_MS(%d), " >>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>> MAX_TIMEOUT_MS); >>> +??????????? timeout = MAX_TIMEOUT_MS; >>> ???????? } >>> >>> ???????? return timestamp + (long)(timeout * getTimeoutFactor()); >>> >> > From huaming.li at oracle.com Fri Jan 19 06:28:24 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Fri, 19 Jan 2018 14:28:24 +0800 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <5a49a3e7-cb4c-f52a-1d7e-8c1178addfad@oracle.com> References: <02311325-7c1d-c76d-a004-c944dab420c4@oracle.com> <5a49a3e7-cb4c-f52a-1d7e-8c1178addfad@oracle.com> Message-ID: <08d8d770-5961-c246-fb5f-ef2a4d4dbcc0@oracle.com> On 19/01/2018 2:11 PM, David Holmes wrote: > Hi Hamlin, > > On 19/01/2018 3:04 PM, Hamlin Li wrote: >> Hi Roger, David >> >> Please check the updated webrev at: >> http://cr.openjdk.java.net/~mli/8194284/webrev.00/ > > RMID.java: > > This comment no longer makes sense: > > ????? // when restart rmid, it may take more time than usual because of > ????? // "port in use" by a possible interloper (check JDK-8168975), > ????? // so need to set a longer timeout for restart. > !???? private static final long RESTART_TIMEOUT = (long)(TIMEOUT_BASE > * 0.9); > > Actually I'm not sure it originally made sense - longer than what? But > as it stands RESTART_TIMEOUT is smaller than TIMEOUT_BASE so the > comment really seems odd. Perhaps 8168975 will shed some light on the > intent. ?? Hi David, The comment still makes sense. Before 8168975, restart() calls start() which is indeed start(STARTTIME_MS), where STARTTIME_MS is 15_000L, but we found out in some situation restart() needs more time than start(); So in 8168975, restart() calls start(restartTimeout), where restartTimeout is RESTART_TIMEOUT in 8194284. > > The TestLibrary change looks good. Do you also think the comment makes sense with my explanation? Thank you -Hamlin > > Thanks, > David > >> Thank you >> >> -Hamlin >> >> >> On 18/01/2018 10:33 PM, Roger Riggs wrote: >>> Hi Hamlin, >>> >>> The base bug is that the timeoutFactor is being applied twice, once >>> in the RMID constructor >>> and again in computeDeadline.? It is better to cleanup the >>> implementation of the test library >>> than to fudge the values.? Either respect the timeouts passed in >>> (remove the scaling from computeDeadline) >>> or consistently leave it to the lower level routines.? Pick 1. >>> >>> Thanks, Roger >>> >>> >>> On 1/18/2018 1:59 AM, Hamlin Li wrote: >>>> Would you please review the following patch? >>>> >>>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>>> >>>> webrev as below. >>>> >>>> >>>> Thank you >>>> >>>> -Hamlin >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >>>> 20:07:50 2018 -0800 >>>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >>>> 14:54:50 2018 +0800 >>>> @@ -188,8 +188,12 @@ >>>> ???? public static long computeDeadline(long timestamp, long >>>> timeout) { >>>> ???????? final long MAX_TIMEOUT_MS = 3_600_000L; >>>> >>>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>>> +??????? if (timeout < 0L) { >>>> ???????????? throw new IllegalArgumentException("timeout " + >>>> timeout + "ms out of range"); >>>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>>> +??????????? System.out.format("timeout value(%d) exceeds >>>> MAX_TIMEOUT_MS(%d), " >>>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>>> MAX_TIMEOUT_MS); >>>> +??????????? timeout = MAX_TIMEOUT_MS; >>>> ???????? } >>>> >>>> ???????? return timestamp + (long)(timeout * getTimeoutFactor()); >>>> >>> >> From david.holmes at oracle.com Fri Jan 19 07:09:09 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 19 Jan 2018 17:09:09 +1000 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <08d8d770-5961-c246-fb5f-ef2a4d4dbcc0@oracle.com> References: <02311325-7c1d-c76d-a004-c944dab420c4@oracle.com> <5a49a3e7-cb4c-f52a-1d7e-8c1178addfad@oracle.com> <08d8d770-5961-c246-fb5f-ef2a4d4dbcc0@oracle.com> Message-ID: <7c023796-4519-2d0b-d32c-84782cb60a18@oracle.com> Hi Hamlin, On 19/01/2018 4:28 PM, Hamlin Li wrote: > > On 19/01/2018 2:11 PM, David Holmes wrote: >> Hi Hamlin, >> >> On 19/01/2018 3:04 PM, Hamlin Li wrote: >>> Hi Roger, David >>> >>> Please check the updated webrev at: >>> http://cr.openjdk.java.net/~mli/8194284/webrev.00/ >> >> RMID.java: >> >> This comment no longer makes sense: >> >> ????? // when restart rmid, it may take more time than usual because of >> ????? // "port in use" by a possible interloper (check JDK-8168975), >> ????? // so need to set a longer timeout for restart. >> !???? private static final long RESTART_TIMEOUT = (long)(TIMEOUT_BASE >> * 0.9); >> >> Actually I'm not sure it originally made sense - longer than what? But >> as it stands RESTART_TIMEOUT is smaller than TIMEOUT_BASE so the >> comment really seems odd. Perhaps 8168975 will shed some light on the >> intent. ?? > Hi David, > > The comment still makes sense. > Before 8168975, restart() calls start() which is indeed > start(STARTTIME_MS), where STARTTIME_MS is 15_000L, but we found out in > some situation restart() needs more time than start(); > So in 8168975, restart() calls start(restartTimeout), where > restartTimeout is RESTART_TIMEOUT in 8194284. Okay, then I suggest changing: // so need to set a longer timeout for restart. to // so need to set a longer timeout than STARTTIME_MS for restart. Thanks, David >> >> The TestLibrary change looks good. > Do you also think the comment makes sense with my explanation? > > Thank you > -Hamlin >> >> Thanks, >> David >> >>> Thank you >>> >>> -Hamlin >>> >>> >>> On 18/01/2018 10:33 PM, Roger Riggs wrote: >>>> Hi Hamlin, >>>> >>>> The base bug is that the timeoutFactor is being applied twice, once >>>> in the RMID constructor >>>> and again in computeDeadline.? It is better to cleanup the >>>> implementation of the test library >>>> than to fudge the values.? Either respect the timeouts passed in >>>> (remove the scaling from computeDeadline) >>>> or consistently leave it to the lower level routines.? Pick 1. >>>> >>>> Thanks, Roger >>>> >>>> >>>> On 1/18/2018 1:59 AM, Hamlin Li wrote: >>>>> Would you please review the following patch? >>>>> >>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>>>> >>>>> webrev as below. >>>>> >>>>> >>>>> Thank you >>>>> >>>>> -Hamlin >>>>> >>>>> ------------------------------------------------------------------------ >>>>> >>>>> >>>>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>>>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Wed Jan 17 >>>>> 20:07:50 2018 -0800 >>>>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java??? Thu Jan 18 >>>>> 14:54:50 2018 +0800 >>>>> @@ -188,8 +188,12 @@ >>>>> ???? public static long computeDeadline(long timestamp, long >>>>> timeout) { >>>>> ???????? final long MAX_TIMEOUT_MS = 3_600_000L; >>>>> >>>>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>>>> +??????? if (timeout < 0L) { >>>>> ???????????? throw new IllegalArgumentException("timeout " + >>>>> timeout + "ms out of range"); >>>>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>>>> +??????????? System.out.format("timeout value(%d) exceeds >>>>> MAX_TIMEOUT_MS(%d), " >>>>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>>>> MAX_TIMEOUT_MS); >>>>> +??????????? timeout = MAX_TIMEOUT_MS; >>>>> ???????? } >>>>> >>>>> ???????? return timestamp + (long)(timeout * getTimeoutFactor()); >>>>> >>>> >>> > From huaming.li at oracle.com Fri Jan 19 07:18:07 2018 From: huaming.li at oracle.com (Hamlin Li) Date: Fri, 19 Jan 2018 15:18:07 +0800 Subject: RFR of 8194284: CheckRegisterInLog.java fails with java.lang.RuntimeException: CheckRegisterInLog got exception timeout 6480000ms out of range In-Reply-To: <7c023796-4519-2d0b-d32c-84782cb60a18@oracle.com> References: <02311325-7c1d-c76d-a004-c944dab420c4@oracle.com> <5a49a3e7-cb4c-f52a-1d7e-8c1178addfad@oracle.com> <08d8d770-5961-c246-fb5f-ef2a4d4dbcc0@oracle.com> <7c023796-4519-2d0b-d32c-84782cb60a18@oracle.com> Message-ID: On 19/01/2018 3:09 PM, David Holmes wrote: > Hi Hamlin, > > On 19/01/2018 4:28 PM, Hamlin Li wrote: >> >> On 19/01/2018 2:11 PM, David Holmes wrote: >>> Hi Hamlin, >>> >>> On 19/01/2018 3:04 PM, Hamlin Li wrote: >>>> Hi Roger, David >>>> >>>> Please check the updated webrev at: >>>> http://cr.openjdk.java.net/~mli/8194284/webrev.00/ >>> >>> RMID.java: >>> >>> This comment no longer makes sense: >>> >>> ????? // when restart rmid, it may take more time than usual because of >>> ????? // "port in use" by a possible interloper (check JDK-8168975), >>> ????? // so need to set a longer timeout for restart. >>> !???? private static final long RESTART_TIMEOUT = >>> (long)(TIMEOUT_BASE * 0.9); >>> >>> Actually I'm not sure it originally made sense - longer than what? >>> But as it stands RESTART_TIMEOUT is smaller than TIMEOUT_BASE so the >>> comment really seems odd. Perhaps 8168975 will shed some light on >>> the intent. ?? >> Hi David, >> >> The comment still makes sense. >> Before 8168975, restart() calls start() which is indeed >> start(STARTTIME_MS), where STARTTIME_MS is 15_000L, but we found out >> in some situation restart() needs more time than start(); >> So in 8168975, restart() calls start(restartTimeout), where >> restartTimeout is RESTART_TIMEOUT in 8194284. > > Okay, then I suggest changing: > > ?// so need to set a longer timeout for restart. > > to > > ?// so need to set a longer timeout than STARTTIME_MS for restart. Hi David, Yes, it's more clear, will push the code with your suggestion. Thank you -Hamlin > > Thanks, > David > >>> >>> The TestLibrary change looks good. >> Do you also think the comment makes sense with my explanation? >> >> Thank you >> -Hamlin >>> >>> Thanks, >>> David >>> >>>> Thank you >>>> >>>> -Hamlin >>>> >>>> >>>> On 18/01/2018 10:33 PM, Roger Riggs wrote: >>>>> Hi Hamlin, >>>>> >>>>> The base bug is that the timeoutFactor is being applied twice, >>>>> once in the RMID constructor >>>>> and again in computeDeadline.? It is better to cleanup the >>>>> implementation of the test library >>>>> than to fudge the values.? Either respect the timeouts passed in >>>>> (remove the scaling from computeDeadline) >>>>> or consistently leave it to the lower level routines. Pick 1. >>>>> >>>>> Thanks, Roger >>>>> >>>>> >>>>> On 1/18/2018 1:59 AM, Hamlin Li wrote: >>>>>> Would you please review the following patch? >>>>>> >>>>>> bug: https://bugs.openjdk.java.net/browse/JDK-8194284 >>>>>> >>>>>> webrev as below. >>>>>> >>>>>> >>>>>> Thank you >>>>>> >>>>>> -Hamlin >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> >>>>>> diff -r 0dec8c41170c test/jdk/java/rmi/testlibrary/TestLibrary.java >>>>>> --- a/test/jdk/java/rmi/testlibrary/TestLibrary.java Wed Jan 17 >>>>>> 20:07:50 2018 -0800 >>>>>> +++ b/test/jdk/java/rmi/testlibrary/TestLibrary.java Thu Jan 18 >>>>>> 14:54:50 2018 +0800 >>>>>> @@ -188,8 +188,12 @@ >>>>>> ???? public static long computeDeadline(long timestamp, long >>>>>> timeout) { >>>>>> ???????? final long MAX_TIMEOUT_MS = 3_600_000L; >>>>>> >>>>>> -??????? if (timeout < 0L || timeout > MAX_TIMEOUT_MS) { >>>>>> +??????? if (timeout < 0L) { >>>>>> ???????????? throw new IllegalArgumentException("timeout " + >>>>>> timeout + "ms out of range"); >>>>>> +??????? } else if (timeout > MAX_TIMEOUT_MS) { >>>>>> +??????????? System.out.format("timeout value(%d) exceeds >>>>>> MAX_TIMEOUT_MS(%d), " >>>>>> +??????????????????? + "use MAX_TIMEOUT_MS instead!%n", timeout, >>>>>> MAX_TIMEOUT_MS); >>>>>> +??????????? timeout = MAX_TIMEOUT_MS; >>>>>> ???????? } >>>>>> >>>>>> ???????? return timestamp + (long)(timeout * getTimeoutFactor()); >>>>>> >>>>> >>>> >> From daniel.fuchs at oracle.com Fri Jan 19 10:13:05 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 19 Jan 2018 10:13:05 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: Message-ID: Hi Jason, On 18/01/2018 21:19, Jason Mehrens wrote: > Daniel, > > As long as the org.apache.juli.ClassLoaderLogManager overrides getProperty it shouldn't really matter what the value format is in the file as long as it is translated on return. > Is there a code path in j.u.l.LogManager that doesn't call getProperty? If so I would think that is the core issue. That's a very good question. Why didn't I think of it? AFAICS there's only one place where we do that, in the 'new' updateConfiguration method, and that's only for levels when a level value is changed by the mapper callback (the new value is used directly without invoking getProperty). Maybe I should log a bug to fix that, but then that's probably not a big issue as it's returned by the mapper. However in the case at hand the private createLoggerHandler method eventually does calls LogManager.getProperty(".handlers"). In the base LogManager that should be the only place where LogManager.getProperty(".handlers") is called - I think. Hmmm... I assume that might also depend on whether other classes call org.apache.juli.ClassLoaderLogManager::getProperty(".handlers") from outside, but then maybe that can be worked around in Tomcat as well. Let me try if I can get feedback on this suggestion. best regards, -- daniel > > Jason From goetz.lindenmaier at sap.com Fri Jan 19 11:48:15 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 19 Jan 2018 11:48:15 +0000 Subject: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError: HelpFlagsTest failed: In-Reply-To: References: <19A6164F-1766-4C30-8A34-05B1185D9525@oracle.com> <45e66d48b4224ef8a5342d8b64a2b24c@sap.com> Message-ID: Hi, this webrev adds a real test case for javapackager. I also removed unused variable dontExecuteWithWrongFlags. http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.03/ Feel free to use this change (webrev03) or http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ with the pure fix for this issue. Best regards, Goetz. > -----Original Message----- > From: Lindenmaier, Goetz > Sent: Donnerstag, 18. Januar 2018 17:25 > To: 'Lindenmaier, Goetz' ; Weijun Wang > > Cc: jdk-dev at openjdk.java.net; core-libs-dev at openjdk.java.net > Subject: RE: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails > with java.lang.AssertionError: HelpFlagsTest failed: > > Posting this to core-libs-dev. > > Please review > http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ > which excludes the Oracle proprietary tools from the test. > I can not verify this. > > Best regards, > Goetz. > > > > -----Original Message----- > > From: jdk-dev [mailto:jdk-dev-bounces at openjdk.java.net] On Behalf Of > > Lindenmaier, Goetz > > Sent: Donnerstag, 18. Januar 2018 17:02 > > To: Weijun Wang > > Cc: jdk-dev at openjdk.java.net > > Subject: RE: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails > > with java.lang.AssertionError: HelpFlagsTest failed: > > > > Hi, > > > > Added ssvagent: > > http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ > > > > Best regards, > > Goetz. > > > > > -----Original Message----- > > > From: Weijun Wang [mailto:weijun.wang at oracle.com] > > > Sent: Donnerstag, 18. Januar 2018 16:44 > > > To: Lindenmaier, Goetz > > > Cc: Kumar Srinivasan ; jdk- > > > dev at openjdk.java.net > > > Subject: Re: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails > > > with java.lang.AssertionError: HelpFlagsTest failed: > > > > > > On Windows, there is also a ssvagent.exe. > > > > > > > On Jan 18, 2018, at 11:37 PM, Lindenmaier, Goetz > > > wrote: > > > > > > > > Hi, > > > > > > > > the most simple fix is to just exclude the four tools: > > > > http://cr.openjdk.java.net/~goetz/wr18/8195663- > > fixHelpTest/webrev.01/ > > > > Unfortunately I can't test this, as my build does not contain these > > > > tools. > > > > > > > > If somebody tells me how to build these tools, I will adapt > > > > the test to check them, and eventually adapt the tools to > > > > CSR 8191477 https://bugs.openjdk.java.net/browse/JDK-8191477. > > > > > > > > What list should this be reviewed on? I just posted to jdk-dev for now. > > > > > > > > Best regards, > > > > Goetz. From kumar.x.srinivasan at oracle.com Fri Jan 19 13:31:41 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Fri, 19 Jan 2018 05:31:41 -0800 Subject: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError: HelpFlagsTest failed: In-Reply-To: <1b30b8eb-1472-2d9f-a51b-fcd5b01f944d@oracle.com> References: <19A6164F-1766-4C30-8A34-05B1185D9525@oracle.com> <45e66d48b4224ef8a5342d8b64a2b24c@sap.com> <1b30b8eb-1472-2d9f-a51b-fcd5b01f944d@oracle.com> Message-ID: <5A61F33D.8080101@oracle.com> Hello, >> Posting this to core-libs-dev. >> >> Please review >> http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ >> which excludes the Oracle proprietary tools from the test. >> I can not verify this. > > Wouldn't a whitelist work better here than a blacklist? Other > distributions might add other tools (in theory). Yes, that was my suggestion as well in the JBS entry for this bug. Kumar > > David > >> Best regards, >> Goetz. >> >> >>> -----Original Message----- >>> From: jdk-dev [mailto:jdk-dev-bounces at openjdk.java.net] On Behalf Of >>> Lindenmaier, Goetz >>> Sent: Donnerstag, 18. Januar 2018 17:02 >>> To: Weijun Wang >>> Cc: jdk-dev at openjdk.java.net >>> Subject: RE: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java >>> fails >>> with java.lang.AssertionError: HelpFlagsTest failed: >>> >>> Hi, >>> >>> Added ssvagent: >>> http://cr.openjdk.java.net/~goetz/wr18/8195663-fixHelpTest/webrev.02/ >>> >>> Best regards, >>> Goetz. >>> >>>> -----Original Message----- >>>> From: Weijun Wang [mailto:weijun.wang at oracle.com] >>>> Sent: Donnerstag, 18. Januar 2018 16:44 >>>> To: Lindenmaier, Goetz >>>> Cc: Kumar Srinivasan ; jdk- >>>> dev at openjdk.java.net >>>> Subject: Re: RFR(XS): 8195663: >>>> jdk/tools/launcher/HelpFlagsTest.java fails >>>> with java.lang.AssertionError: HelpFlagsTest failed: >>>> >>>> On Windows, there is also a ssvagent.exe. >>>> >>>>> On Jan 18, 2018, at 11:37 PM, Lindenmaier, Goetz >>>> wrote: >>>>> >>>>> Hi, >>>>> >>>>> the most simple fix is to just exclude the four tools: >>>>> http://cr.openjdk.java.net/~goetz/wr18/8195663- >>> fixHelpTest/webrev.01/ >>>>> Unfortunately I can't test this, as my build does not contain these >>>>> tools. >>>>> >>>>> If somebody tells me how to build these tools, I will adapt >>>>> the test to check them, and eventually adapt the tools to >>>>> CSR 8191477 https://bugs.openjdk.java.net/browse/JDK-8191477. >>>>> >>>>> What list should this be reviewed on? I just posted to jdk-dev for >>>>> now. >>>>> >>>>> Best regards, >>>>> Goetz. >> From vlv.spb.ru at mail.ru Fri Jan 19 13:38:03 2018 From: vlv.spb.ru at mail.ru (Vladimir Yaroslavskiy) Date: Fri, 19 Jan 2018 16:38:03 +0300 Subject: The new optimized version of Dual-Pivot Quicksort Message-ID: <6defd04f-28e0-64fb-a18c-9bb320203f35@mail.ru> Hi team, In Sept 2009 Josh Bloch, Jon Bentley and I introduced new sorting algorithm, Dual-Pivot Quicksort, for primitives in JDK 7 and later I suggested several improvements of Dual-Pivot Quicksort, which were integrated into JDK 8. Now I have more optimized and faster version of Dual-Pivot Quicksort. I have been working on it for the last 5 years. Please, find the summary of changes below and sources / diff at webrev [1]. All tests and benchmarking were run on the most recent build of JDK 10, jdk-10-ea+39. The new version shows the better performance on different inputs and guarantees n*log(n) on any data. The new implementation of Dual-Pivot Quicksort is 'all-in-one' version: it contains one code for both parallel and sequential sorting algorithms. Suggested version is 10-20% faster on random data, 1.5-4 times faster on nearly structured arrays, 1.5-2 times faster on period inputs. Parallel Dual-Pivot Quicksort is 1.5-3 times faster than current algorithm based on merge sort. Benchmarking on the test suite, suggested by Jon Bentley, shows the boost of performance in 1.4 times. This test suite contains several types of inputs, such as random data, nearly structured arrays, data with period and so on. Also there are several modifications of inputs and parameters which are used in data building. We run sorting on all combinations to compare two algorithms. Please let me know if you have any questions / comments. Summary of changes: DualPivotQuicksort class ------------------------ * Pivots are chosen with another step, the 1-st and 5-th candidates are taken as pivots instead of 2-nd and 4-th. * Splitting into parts is related to the golden ratio * Pivot candidates are sorted by combination of 5-element network sorting + insertion sort * New backwards partitioning is simpler and more efficient * Quicksort tuning parameters were updated * Merging sort is invoked on each iteration from Quicksort * Additional steps for float/double were fully rewritten * Heap sort is invoked on the leftmost part * Heap sort is used as a guard against quadratic time * Parallel sorting is based on Dual-Pivot Quicksort instead of merge sort SortingAlgorithms class ----------------------- * Merging sort and pair insertion sort were moved from DualPivotQuicksort class * Pair insertion sort was simplified and optimized * New nano insertion sort was introduced for tiny arrays * Merging sort was fully rewritten * Optimized merging partitioning is used * Merging parameters were updated * Merging of runs was fully rewritten * Fast version of heap sort was introduced * Parallel merging sort was also provided Sorting / ParallelSorting classes --------------------------------- * New test cases were added * Sources of these classes were unified Arrays class ------------ * Calls of Dual-Pivot Quicksort were updated * Parallel sorting of primitives was switched to parallel Dual-Pivot Quicksort * Javadoc was modified ArraysParallelSortHelpers class ------------------------------- * Old implementation of parallel sorting for primitives was removed * Javadoc was updated Thank you, Vladimir -------------------------------------------------------------------- [1] http://cr.openjdk.java.net/~alanb/DualPivotSortUpdate/webrev.01/ -------------------------------------------------------------------- From goetz.lindenmaier at sap.com Fri Jan 19 14:00:49 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Fri, 19 Jan 2018 14:00:49 +0000 Subject: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError: HelpFlagsTest failed: In-Reply-To: <5A61F33D.8080101@oracle.com> References: <19A6164F-1766-4C30-8A34-05B1185D9525@oracle.com> <45e66d48b4224ef8a5342d8b64a2b24c@sap.com> <1b30b8eb-1472-2d9f-a51b-fcd5b01f944d@oracle.com> <5A61F33D.8080101@oracle.com> Message-ID: <8855b429551e4da0a44489597e404a1f@sap.com> Hi, I understand the effects of a whitelist. But the test is just intended to dig up tools that don't adhere to the CSR. It's easy enough to fix if you add a tool of yourself, as you can see in this change. Thanks for testing webrev 03. Feel free to push it if it solves your problems in case it's night in Europe :) Best regards, Goetz. > -----Original Message----- > From: Kumar Srinivasan [mailto:kumar.x.srinivasan at oracle.com] > Sent: Freitag, 19. Januar 2018 14:32 > To: David Holmes ; Lindenmaier, Goetz > > Cc: Weijun Wang ; core-libs- > dev at openjdk.java.net > Subject: Re: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java fails > with java.lang.AssertionError: HelpFlagsTest failed: > > Hello, > > >> Posting this to core-libs-dev. > >> > >> Please review > >> http://cr.openjdk.java.net/~goetz/wr18/8195663- > fixHelpTest/webrev.02/ > >> which excludes the Oracle proprietary tools from the test. > >> I can not verify this. > > > > Wouldn't a whitelist work better here than a blacklist? Other > > distributions might add other tools (in theory). > > Yes, that was my suggestion as well in the JBS entry for this bug. > > Kumar > > > > > David > > > >> Best regards, > >> Goetz. > >> > >> > >>> -----Original Message----- > >>> From: jdk-dev [mailto:jdk-dev-bounces at openjdk.java.net] On Behalf > Of > >>> Lindenmaier, Goetz > >>> Sent: Donnerstag, 18. Januar 2018 17:02 > >>> To: Weijun Wang > >>> Cc: jdk-dev at openjdk.java.net > >>> Subject: RE: RFR(XS): 8195663: jdk/tools/launcher/HelpFlagsTest.java > >>> fails > >>> with java.lang.AssertionError: HelpFlagsTest failed: > >>> > >>> Hi, > >>> > >>> Added ssvagent: > >>> http://cr.openjdk.java.net/~goetz/wr18/8195663- > fixHelpTest/webrev.02/ > >>> > >>> Best regards, > >>> Goetz. > >>> > >>>> -----Original Message----- > >>>> From: Weijun Wang [mailto:weijun.wang at oracle.com] > >>>> Sent: Donnerstag, 18. Januar 2018 16:44 > >>>> To: Lindenmaier, Goetz > >>>> Cc: Kumar Srinivasan ; jdk- > >>>> dev at openjdk.java.net > >>>> Subject: Re: RFR(XS): 8195663: > >>>> jdk/tools/launcher/HelpFlagsTest.java fails > >>>> with java.lang.AssertionError: HelpFlagsTest failed: > >>>> > >>>> On Windows, there is also a ssvagent.exe. > >>>> > >>>>> On Jan 18, 2018, at 11:37 PM, Lindenmaier, Goetz > >>>> wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> the most simple fix is to just exclude the four tools: > >>>>> http://cr.openjdk.java.net/~goetz/wr18/8195663- > >>> fixHelpTest/webrev.01/ > >>>>> Unfortunately I can't test this, as my build does not contain these > >>>>> tools. > >>>>> > >>>>> If somebody tells me how to build these tools, I will adapt > >>>>> the test to check them, and eventually adapt the tools to > >>>>> CSR 8191477 https://bugs.openjdk.java.net/browse/JDK-8191477. > >>>>> > >>>>> What list should this be reviewed on? I just posted to jdk-dev for > >>>>> now. > >>>>> > >>>>> Best regards, > >>>>> Goetz. > >> From daniel.fuchs at oracle.com Fri Jan 19 14:22:48 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 19 Jan 2018 14:22:48 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: Message-ID: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> Hi, For the record, I updated the JBS issue [1] with Jason's suggestion and asked to get feedback from the submitter. best regards, -- daniel [1] https://bugs.openjdk.java.net/browse/JDK-8195096 On 19/01/2018 10:13, Daniel Fuchs wrote: > Hi Jason, > > On 18/01/2018 21:19, Jason Mehrens wrote: >> Daniel, >> >> As long as the org.apache.juli.ClassLoaderLogManager overrides >> getProperty it shouldn't really matter what the value format is in the >> file as long as it is translated on return. >> Is there a code path in j.u.l.LogManager that doesn't call >> getProperty?? If so I would think that is the core issue. > > That's a very good question. Why didn't I think of it? > > AFAICS there's only one place where we do that, in the > 'new' updateConfiguration method, and that's only for > levels when a level value is changed by the mapper > callback (the new value is used directly without invoking > getProperty). Maybe I should log a bug to fix that, but > then that's probably not a big issue as it's returned > by the mapper. > > However in the case at hand the private createLoggerHandler > method eventually does calls LogManager.getProperty(".handlers"). > In the base LogManager that should be the only place where > LogManager.getProperty(".handlers") is called - I think. > > Hmmm... I assume that might also depend on whether other classes > call org.apache.juli.ClassLoaderLogManager::getProperty(".handlers") > from outside, but then maybe that can be worked around in Tomcat > as well. > > Let me try if I can get feedback on this suggestion. > > best regards, > > -- daniel > >> >> Jason > From Roger.Riggs at Oracle.com Fri Jan 19 15:14:25 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 19 Jan 2018 10:14:25 -0500 Subject: Clarification for adding classes shipped with JDK to sun.rmi.registry.registryFilter property in java.security file In-Reply-To: References: Message-ID: Hi Vipin, A couple of suggestions to make working with serialization filters easier. JEP 290 enabled logging of filtering actions by setting java.io.serialization.level =? FINEST. Add that to the logging.properties and supply it on the command line with "-Djava.util.logging.config.file=logging.properties". If set to FINEST, it will log every call to the filter with the arguments of class, array size, etc. If set to FINE, it will log only the cases where the filter rejects based on its arguments. The pattern based filters can allow or disallow entire packages. For example, "java.net.**" will allow any class in any package with a "java.net." prefix. For development purposes, serialized classes for the registry could be identified using a filter of "*" with logging enabled as FINEST and examining the log. Expanding the white list for the registry is a possibility with a very conservative evaluation of specific cases. Regards, Roger [1] http://openjdk.java.net/jeps/290 On 1/16/2018 7:21 AM, Vipin Mv1 wrote: > > Hi, > > After upgrading to oracle 8u121-b12, I get following Exception when running > the following testcase. > > import java.io.*; > import java.net.InetAddress; > import java.rmi.*; > import java.rmi.registry.*; > import java.security.Security; > > /* @test > * @run main/othervm RegistryFilterTest > */ > > public class RegistryFilterTest { > > private static int port = 12345; > private static Registry registry; > > static class RemoteObj implements Serializable, Remote { > private static final long serialVersionUID = 01L; > > final Object obj; > > RemoteObj(Object obj) { > this.obj = obj; > } > > > } > > > public static void main(String[] args) throws Exception{ > > InetAddress in = InetAddress.getLocalHost(); > LocateRegistry.createRegistry(port); > Registry registry = LocateRegistry.getRegistry("localhost", > port); > registry.bind("InetAddress", new RemoteObj(in)); > registry.unbind("InetAddress"); > System.out.println("RMI Registry Test Passed"); > > } > > } > > TestResults > ----------------------- > Jan 12, 2018 5:32:18 PM java.io.ObjectInputStream filterCheck > INFO: ObjectInputFilter REJECTED: class java.net.InetAddress, array length: > -1, nRefs: 5, depth: 2, bytes: 216, ex: n/a > Caused by: java.io.InvalidClassException: filter status: REJECTED > ?? ?at java.io.ObjectInputStream.filterCheck(ObjectInputStream.java:1406) > ?? ?at java.io.ObjectInputStream.readNonProxyDesc > (ObjectInputStream.java:1997) > ?? ?at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1877) > ?? ?at java.io.ObjectInputStream.readOrdinaryObject > (ObjectInputStream.java:2170) > ?? ?at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) > ?? ?at java.io.ObjectInputStream.defaultReadFields > (ObjectInputStream.java:2415) > ?? ?at java.io.ObjectInputStream.readSerialData > (ObjectInputStream.java:2339) > ?? ?at java.io.ObjectInputStream.readOrdinaryObject > (ObjectInputStream.java:2197) > ?? ?at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) > ?? ?at java.io.ObjectInputStream.readObjectImpl(ObjectInputStream.java:540) > ?? ?at java.io.ObjectInputStream.readObject(ObjectInputStream.java:475) > ?? ?... 16 more > > I understand it is due to the new serialization Filtering mechanism > introduced to RMI Registry and Distributed Garbage Collection as a part of > the following fixes. > > JDK-8160108 Implement Serialization Filtering > JDK-8156804: Better constraint checking(not public) > > As a result, a new default white list has been introduced for RMIRegistry > which allows deserialization only to limited set of classes by default. > Which are > > String.class > java.lang.Number.class > java.rmi.Remote.class > java.lang.reflect.Proxy.class > sun.rmi.server.UnicastRef.class > java.rmi.server.RMIClientSocketFactory.class > java.rmi.server.RMIServerSocketFactory.class > java.rmi.activation.ActivationID.class > java.rmi.server.UID.class > and their subclasses. > > It also enabled provisions to increase the default white list by > > 1. updating property "sun.rmi.registry.registryFilter" in java.security > file > 2. adding -Dsun.rmi.registry.registryFilter in java command line > > Though the current solution is straight forward, it is difficult from an > application point of view to identify such classes without running the > application multiple times to come up with comprehensive list to be > allowed. This would be a overhead to the developer and should be avoided at > least for classes shipped with JDK (for example, java.net.InetAddress as > used in testcase). > > I assume that default list is very limited to reduce the chances of getting > exploited by serialization vulnerability. I think, we can spare public > packages shipped with JDK by adding them to > "sun.rmi.registry.registryFilter" in java.security file out of the box. > > Please let me know your views. > > Thanks & Regards, > Vipin MV From jason_mehrens at hotmail.com Fri Jan 19 16:00:46 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Fri, 19 Jan 2018 16:00:46 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> References: , <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> Message-ID: Daniel, Looking at: https://github.com/apache/tomcat/blob/trunk/java/org/apache/juli/ClassLoaderLogManager.java it looks like code in "readConfiguration(InputStream, ClassLoader)" does the digit translation. The tomcat developers should be able to move some of that code to CLLM.getProperty to do the translation. If they don't want JUL to create the root handlers then another option would be to alter CLLM.getProperty to determine if the j.ul.LogManager is trying to initialize ".handers" before CLLM.readConfiguration is called and return null to effectively disable creating handlers at that time. My guess is that this change is probably more compatible with what was already happening in Tomcat. Jason ________________________________________ From: core-libs-dev on behalf of Daniel Fuchs Sent: Friday, January 19, 2018 8:22 AM To: core-libs-dev Subject: Re: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat Hi, For the record, I updated the JBS issue [1] with Jason's suggestion and asked to get feedback from the submitter. best regards, -- daniel [1] https://bugs.openjdk.java.net/browse/JDK-8195096 On 19/01/2018 10:13, Daniel Fuchs wrote: > Hi Jason, > > On 18/01/2018 21:19, Jason Mehrens wrote: >> Daniel, >> >> As long as the org.apache.juli.ClassLoaderLogManager overrides >> getProperty it shouldn't really matter what the value format is in the >> file as long as it is translated on return. >> Is there a code path in j.u.l.LogManager that doesn't call >> getProperty? If so I would think that is the core issue. > > That's a very good question. Why didn't I think of it? > > AFAICS there's only one place where we do that, in the > 'new' updateConfiguration method, and that's only for > levels when a level value is changed by the mapper > callback (the new value is used directly without invoking > getProperty). Maybe I should log a bug to fix that, but > then that's probably not a big issue as it's returned > by the mapper. > > However in the case at hand the private createLoggerHandler > method eventually does calls LogManager.getProperty(".handlers"). > In the base LogManager that should be the only place where > LogManager.getProperty(".handlers") is called - I think. > > Hmmm... I assume that might also depend on whether other classes > call org.apache.juli.ClassLoaderLogManager::getProperty(".handlers") > from outside, but then maybe that can be worked around in Tomcat > as well. > > Let me try if I can get feedback on this suggestion. > > best regards, > > -- daniel > >> >> Jason > From jason_mehrens at hotmail.com Fri Jan 19 16:51:57 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Fri, 19 Jan 2018 16:51:57 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: , Message-ID: Daniel, Double check me on this but, I think I've found the source of why the error occurs on JDK9 and not JDK8. In https://bugs.openjdk.java.net/browse/JDK-8191033 the new code should have been added to initializeGlobalHandlers. That would make the read of ".handers" lazy and keep the read of "handlers" eager. Jason ________________________________________ From: Daniel Fuchs Sent: Friday, January 19, 2018 4:13 AM To: Jason Mehrens; core-libs-dev Subject: Re: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat Hi Jason, On 18/01/2018 21:19, Jason Mehrens wrote: > Daniel, > > As long as the org.apache.juli.ClassLoaderLogManager overrides getProperty it shouldn't really matter what the value format is in the file as long as it is translated on return. > Is there a code path in j.u.l.LogManager that doesn't call getProperty? If so I would think that is the core issue. That's a very good question. Why didn't I think of it? AFAICS there's only one place where we do that, in the 'new' updateConfiguration method, and that's only for levels when a level value is changed by the mapper callback (the new value is used directly without invoking getProperty). Maybe I should log a bug to fix that, but then that's probably not a big issue as it's returned by the mapper. However in the case at hand the private createLoggerHandler method eventually does calls LogManager.getProperty(".handlers"). In the base LogManager that should be the only place where LogManager.getProperty(".handlers") is called - I think. Hmmm... I assume that might also depend on whether other classes call org.apache.juli.ClassLoaderLogManager::getProperty(".handlers") from outside, but then maybe that can be worked around in Tomcat as well. Let me try if I can get feedback on this suggestion. best regards, -- daniel > > Jason From markt at apache.org Fri Jan 19 17:34:32 2018 From: markt at apache.org (Mark Thomas) Date: Fri, 19 Jan 2018 17:34:32 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> Message-ID: On 19/01/18 14:22, Daniel Fuchs wrote: > Hi, > > For the record, I updated the JBS issue [1] with Jason's suggestion > and asked to get feedback from the submitter. Providing feedback, as requested. Thanks for looking at this so quickly. The speedy response is appreciated. I did look to try and handle this in getProperty() but I ran into the following problem. ClassLoaderLogManager expects getProperty() to return the custom value for .handlers that includes the modified class name. If this value is returned, LogManager generates the unwanted stack trace. If the value is 'corrected' so LogManager doesn't generate the stack trace, ClassLoaderLogManager sees the wrong value and isn't configured correctly. If I've missed a way to handle this, a pointer to that solution is appreciated. I spent some time looking at a way to work around it before raising a bug and I couldn't find a practical solution. I really didn't want to start down the road of looking at the call stack to determine how to handle values in getProperty(). The protected addInitialRootLoggerHandlers() solution looks to be exactly what is required. Kind regards, Mark > > best regards, > > -- daniel > [1] https://bugs.openjdk.java.net/browse/JDK-8195096 > > > On 19/01/2018 10:13, Daniel Fuchs wrote: >> Hi Jason, >> >> On 18/01/2018 21:19, Jason Mehrens wrote: >>> Daniel, >>> >>> As long as the overrides >>> getProperty it shouldn't really matter what the value format is in >>> the file as long as it is translated on return. >>> Is there a code path in j.u.l.LogManager that doesn't call >>> getProperty? If so I would think that is the core issue. >> >> That's a very good question. Why didn't I think of it? >> >> AFAICS there's only one place where we do that, in the >> 'new' updateConfiguration method, and that's only for >> levels when a level value is changed by the mapper >> callback (the new value is used directly without invoking >> getProperty). Maybe I should log a bug to fix that, but >> then that's probably not a big issue as it's returned >> by the mapper. >> >> However in the case at hand the private createLoggerHandler >> method eventually does calls LogManager.getProperty(".handlers"). >> In the base LogManager that should be the only place where >> LogManager.getProperty(".handlers") is called - I think. >> >> Hmmm... I assume that might also depend on whether other classes >> call org.apache.juli.ClassLoaderLogManager::getProperty(".handlers") >> from outside, but then maybe that can be worked around in Tomcat >> as well. >> >> Let me try if I can get feedback on this suggestion. >> >> best regards, >> >> -- daniel >> >>> >>> Jason >> > From daniel.fuchs at oracle.com Fri Jan 19 19:20:55 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 19 Jan 2018 19:20:55 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: Message-ID: <1d37ba6a-52de-0fda-35b7-f5f0d89c7b63@oracle.com> Hi Jason, I'm afraid the issue is more subtle than that. ".handler" has never been handled lazily - it was also eagerly called in 8. What changes is that in JDK 9, in ensureLogManagerInitialized(), I moved the creation of the RootLogger just before calling the call to readPrimordialConfiguration(), as that made sure the RootLogger would be created and added to the LoggerContext before the "global" logger. This in turn has had the subtle side effect to cause the root logger to be added to the LoggerContext earlier than before, during the reset() call that readConfiguration() makes. Which means that in JDK 9 when we returned from readPrimordialConfiguration() then the RootLogger was already present in the LoggerContext, and when we later call addLogger(root), then loadLoggerHandlers("", ".handler") was no longer called. The sad truth is that we had absolutely no test with a configuration setting the ".handlers" property. All our tests were using "handlers", and so the subtle behavior change between 8 and 9 went completely unnoticed. I wasn't even aware that setting ".handler" for the root logger was (kind of) supported. In JDK 10 I fixed that regression by forcing a call to loadLoggerHandlers("", ".handler") just after reading the primordial configuration and calling addLogger(), which is restoring the behavior we had in 8 (or AFAICT very close to it). best regards, -- daniel On 19/01/2018 16:51, Jason Mehrens wrote: > Daniel, > > Double check me on this but, I think I've found the source of why the error occurs on JDK9 and not JDK8. > Inhttps://bugs.openjdk.java.net/browse/JDK-8191033 the new code should have been added to initializeGlobalHandlers. That would make the read of ".handers" lazy and keep the read of "handlers" eager. > > Jason From daniel.fuchs at oracle.com Fri Jan 19 19:25:07 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Fri, 19 Jan 2018 19:25:07 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> Message-ID: Hi Mark, Would it be possible for you to call super.getProperty() when you want to see the 'raw' value, and this.getProperty() when you want to see the 'corrected' value? best regards, -- daniel On 19/01/2018 17:34, Mark Thomas wrote: > On 19/01/18 14:22, Daniel Fuchs wrote: >> Hi, >> >> For the record, I updated the JBS issue [1] with Jason's suggestion >> and asked to get feedback from the submitter. > > Providing feedback, as requested. > > Thanks for looking at this so quickly. The speedy response is appreciated. > > I did look to try and handle this in getProperty() but I ran into the > following problem. > > ClassLoaderLogManager expects getProperty() to return the custom value > for .handlers that includes the modified class name. If this value is > returned, LogManager generates the unwanted stack trace. If the value is > 'corrected' so LogManager doesn't generate the stack trace, > ClassLoaderLogManager sees the wrong value and isn't configured correctly. > > If I've missed a way to handle this, a pointer to that solution is > appreciated. I spent some time looking at a way to work around it before > raising a bug and I couldn't find a practical solution. I really didn't > want to start down the road of looking at the call stack to determine > how to handle values in getProperty(). > > The protected addInitialRootLoggerHandlers() solution looks to be > exactly what is required. > > Kind regards, > > Mark From Roger.Riggs at Oracle.com Fri Jan 19 19:49:28 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Fri, 19 Jan 2018 14:49:28 -0500 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> Message-ID: <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> Hi Brian, Looks good, A pre-existing typo: ??? line 67 "{@code skip()}" should be "{@code skip(*long*)}". Since the public readNBytes suffices for readAllBytes, I would rename the private readAtMostNBytes to readNBytes and avoid the duplication of javadoc. Keeping the existing readAllBytes before readNBytes in the source file will make the diff easier to follow and the methods be in alphabetical order. Thanks, Roger On 1/17/2018 11:24 AM, Brian Burkhalter wrote: > The proposed change has been modified to replace the two methods > > byte[] InputStream.readAllBytes(int) // reads at most ?len? bytes > byte[] InputStream.readNBytes(int) // reads exactly ?len? bytes or throws IOException > > with a single method > > byte[] InputStream.readNBytes(int) // reads at most ?len? bytes > > A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. > > http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ > > Thanks, > > Brian > > On Jan 16, 2018, at 11:17 AM, Brian Burkhalter wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8139206 >> http://cr.openjdk.java.net/~bpb/8139206/webrev.00/ >> >> This change would add a new method ?byte[] InputStream.readNBytes(int len)? which would read up to at most ?len? bytes from the stream and return them in an internally allocated array. From markt at apache.org Fri Jan 19 20:14:57 2018 From: markt at apache.org (Mark Thomas) Date: Fri, 19 Jan 2018 20:14:57 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> Message-ID: On 19/01/18 19:25, Daniel Fuchs wrote: > Hi Mark, > > Would it be possible for you to call super.getProperty() > when you want to see the 'raw' value, and > this.getProperty() when you want to see the 'corrected' > value? Hi Daniel, I see where you are coming from but I don't think so. Jason made a suggestion off-list that got me thinking in slightly different direction. I'm going to look at whether I can use a ThreadLocal to indicate where the call originates so requests for ".handlers" are responded to differently for CLLM and LM. I'm not sure what is possible at this point. I need to do some more research. Thanks to both of you for the hints that have got me thinking in new directions for a workaround. Mark > > best regards, > > -- daniel > > On 19/01/2018 17:34, Mark Thomas wrote: >> On 19/01/18 14:22, Daniel Fuchs wrote: >>> Hi, >>> >>> For the record, I updated the JBS issue [1] with Jason's suggestion >>> and asked to get feedback from the submitter. >> >> Providing feedback, as requested. >> >> Thanks for looking at this so quickly. The speedy response is >> appreciated. >> >> I did look to try and handle this in getProperty() but I ran into the >> following problem. >> >> ClassLoaderLogManager expects getProperty() to return the custom value >> for .handlers that includes the modified class name. If this value is >> returned, LogManager generates the unwanted stack trace. If the value is >> 'corrected' so LogManager doesn't generate the stack trace, >> ClassLoaderLogManager sees the wrong value and isn't configured >> correctly. >> >> If I've missed a way to handle this, a pointer to that solution is >> appreciated. I spent some time looking at a way to work around it before >> raising a bug and I couldn't find a practical solution. I really didn't >> want to start down the road of looking at the call stack to determine >> how to handle values in getProperty(). >> >> The protected addInitialRootLoggerHandlers() solution looks to be >> exactly what is required. >> >> Kind regards, >> >> Mark From markt at apache.org Fri Jan 19 20:56:03 2018 From: markt at apache.org (Mark Thomas) Date: Fri, 19 Jan 2018 20:56:03 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> Message-ID: <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> On 19/01/18 20:14, Mark Thomas wrote: > Thanks to both of you for the hints that have got me thinking in new > directions for a workaround. Thanks again for the hints. I have a work-around that seems to work: https://svn.apache.org/viewvc?view=revision&revision=1821708 It feels a bit hacky. I'd still prefer to be able to override addInitialRootLoggerHandlers() as that strikes me as a cleaner solution. Cheers, Mark From jason_mehrens at hotmail.com Fri Jan 19 22:33:49 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Fri, 19 Jan 2018 22:33:49 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> , <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> Message-ID: Not my call but adding addInitialRootLoggerHandlers() seems too much like a private detail that would be forever published as part of the public API. I would rather have have a protected reportError added to LogManager and redirect all internal errors from directly calling System.err to that method. Then sub-classes could override that method and ignore internal errors generated by the super class. Jason ________________________________________ From: core-libs-dev on behalf of Mark Thomas Sent: Friday, January 19, 2018 2:56 PM To: core-libs-dev at openjdk.java.net Subject: Re: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat On 19/01/18 20:14, Mark Thomas wrote: > Thanks to both of you for the hints that have got me thinking in new > directions for a workaround. Thanks again for the hints. I have a work-around that seems to work: https://svn.apache.org/viewvc?view=revision&revision=1821708 It feels a bit hacky. I'd still prefer to be able to override addInitialRootLoggerHandlers() as that strikes me as a cleaner solution. Cheers, Mark From daniel.fuchs at oracle.com Sat Jan 20 13:17:35 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Sat, 20 Jan 2018 13:17:35 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> Message-ID: <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> Hi Mark, Thanks for these links and for trying out a solution. > On 19/01/18 20:14, Mark Thomas wrote: > Thanks again for the hints. I have a work-around that seems to work: > https://svn.apache.org/viewvc?view=revision&revision=1821708 I see... so what I think is happening is that in JDK 8, the handlers from ".handlers" where added to the root logger from within addLogger(rootLogger) and because ClassLoaderLogManager overrides LogManager::addLogger, then that part was skipped. In JDK 9 - that behavior changed - the handlers from ".handlers" were no longer added because the root logger was already present in the context when addLogger got called (that was a regression), but since ClassLoaderLogManager overrides LogManager::addLogger anyway - it made no difference for ClassLoaderLogManager. In JDK 10, to fix the issue, I forced a call that would add the handlers from ".handlers" to the root logger, but because this now happens outside of addLogger(rootLogger) (just after) then ClassLoaderLogManager feels the difference. hmmm... let me think on this a bit more. best regards, -- daniel On 19/01/18 20:56, Mark Thomas wrote: > On 19/01/18 20:14, Mark Thomas wrote: > > > >> Thanks to both of you for the hints that have got me thinking in new >> directions for a workaround. > > Thanks again for the hints. I have a work-around that seems to work: > https://svn.apache.org/viewvc?view=revision&revision=1821708 > > It feels a bit hacky. I'd still prefer to be able to override > addInitialRootLoggerHandlers() as that strikes me as a cleaner solution. > > Cheers, > > Mark > From kak at google.com Mon Jan 22 02:58:55 2018 From: kak at google.com (Kurt Alfred Kluever) Date: Sun, 21 Jan 2018 21:58:55 -0500 Subject: Microsecond support in java.time.Duration/Instant? Message-ID: Hi core-libs-dev, At Google, we often use microsecond precision for instants/durations (probably because of historical reasons). Therefore, we have a handful of static utility functions for creating/converting java.time types and microseconds: public static Duration ofMicros(long micros) public static long toMicros(Duration duration) public static Instant ofEpochMicros(long micros) public static long toEpochMicros(Instant instant) Duration includes APIs for creating/converting to days, hours, minutes, seconds, millis, and nanos. Instant includes APIs for creating/converting to seconds and millis. Micros are noticeably absent. I'm curious how these sets of units were chosen or decided upon? I understand that the line must be drawn somewhere (or else someone may come along asking for centisecond support), but I'm curious as to the rational. Thanks! -- kak From weijun.wang at oracle.com Mon Jan 22 05:12:45 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 22 Jan 2018 13:12:45 +0800 Subject: Cannot add attribute into main attributes of a jar if there is no version Message-ID: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> src/java.base/share/classes/java/util/jar/Attributes.java: 329 @SuppressWarnings("deprecation") 330 void writeMain(DataOutputStream out) throws IOException 331 { 332 // write out the *-Version header first, if it exists 333 String vername = Name.MANIFEST_VERSION.toString(); 334 String version = getValue(vername); 335 if (version == null) { 336 vername = Name.SIGNATURE_VERSION.toString(); 337 version = getValue(vername); 338 } 339 340 if (version != null) { 341 out.writeBytes(vername+": "+version+"\r\n"); 342 } 343 344 // write out all attributes except for the version 345 // we wrote out earlier 346 for (Entry e : entrySet()) { 347 String name = ((Name) e.getKey()).toString(); 348 if ((version != null) && !(name.equalsIgnoreCase(vername))) { So, if there is no existing MANIFEST_VERSION or SIGNATURE_VERSION, then version is null and the check above will be false for ever and any other attribute cannot be written out. Is this intended? If so, we can exit with an else block after line 342. Thanks Max p.s. I am writing a test and notice this. 349 350 StringBuffer buffer = new StringBuffer(name); 351 buffer.append(": "); 352 353 String value = (String) e.getValue(); 354 if (value != null) { 355 byte[] vb = value.getBytes("UTF8"); 356 value = new String(vb, 0, 0, vb.length); 357 } 358 buffer.append(value); 359 360 buffer.append("\r\n"); 361 Manifest.make72Safe(buffer); 362 out.writeBytes(buffer.toString()); 363 } 364 } 365 out.writeBytes("\r\n"); 366 } From peter.levart at gmail.com Mon Jan 22 08:44:21 2018 From: peter.levart at gmail.com (Peter Levart) Date: Mon, 22 Jan 2018 09:44:21 +0100 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> Message-ID: Hi, On 01/19/2018 08:49 PM, Roger Riggs wrote: > Hi Brian, > > Looks good, > > A pre-existing typo: > ??? line 67 "{@code skip()}" should be "{@code skip(*long*)}". > > Since the public readNBytes suffices for readAllBytes, I would rename > the private readAtMostNBytes > to readNBytes and avoid the duplication of javadoc. > > Keeping the existing readAllBytes before readNBytes in the source file > will make the diff easier to follow > and the methods be in alphabetical order. > > Thanks, Roger The delegation to public method (readAllBytes -> readNBytes) should then be documented so that subclasses know that overriding readNBytes, if needed, is sufficient. (likewise read(byte[]) which delegates to read(byte[], int, int) which then delegates to read() - these delegations are historically known and are part of the de-facto specification on which many subclasses depend). Perhaps, in a different issue, the delegation among all methods in InputStream (and OutputStream) could be documented as @implSpec? Regards, Peter > > > > On 1/17/2018 11:24 AM, Brian Burkhalter wrote: >> The proposed change has been modified to replace the two methods >> >> byte[] InputStream.readAllBytes(int) // reads at most ?len? bytes >> byte[] InputStream.readNBytes(int) // reads exactly ?len? bytes or >> throws IOException >> >> with a single method >> >> byte[] InputStream.readNBytes(int) // reads at most ?len? bytes >> >> A negative value of ?len? will now cause an IllegalArgumentException >> instead of an IndexOutOfBoundsException. Also some verbiage has been >> improved. >> >> http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ >> >> Thanks, >> >> Brian >> >> On Jan 16, 2018, at 11:17 AM, Brian Burkhalter >> wrote: >> >>> https://bugs.openjdk.java.net/browse/JDK-8139206 >>> http://cr.openjdk.java.net/~bpb/8139206/webrev.00/ >>> >>> This change would add a new method ?byte[] >>> InputStream.readNBytes(int len)? which would read up to at most >>> ?len? bytes from? the stream and return them in an internally >>> allocated array. > From Alan.Bateman at oracle.com Mon Jan 22 08:52:55 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Jan 2018 08:52:55 +0000 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> Message-ID: <26706258-adaa-e7c1-240d-cf8a6e55ca09@oracle.com> On 17/01/2018 16:24, Brian Burkhalter wrote: > : > > A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. > > http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ > The updated version looks good. I just wonde about the "For example ..." in the @throws OOME description. The API can't be called with a len > Integer.MAX_VALUE so this example could be confusing - I think just drop that sentence. Minor formatting in passing. At L128 and L339 it would be easier to read if the "throws IOException" were on the previous line. Also L355 might be a bit clear if the Math.min was indented (have to look twice to see that it's not in the while body). -Alan From weijun.wang at oracle.com Mon Jan 22 09:01:37 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 22 Jan 2018 17:01:37 +0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <26706258-adaa-e7c1-240d-cf8a6e55ca09@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <26706258-adaa-e7c1-240d-cf8a6e55ca09@oracle.com> Message-ID: I wonder when readNBytes(n) will be useful if the return value has less than n bytes. > On Jan 22, 2018, at 4:52 PM, Alan Bateman wrote: > > > > On 17/01/2018 16:24, Brian Burkhalter wrote: >> : >> >> A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. >> >> http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ >> > The updated version looks good. I just wonde about the "For example ..." in the @throws OOME description. The API can't be called with a len > Integer.MAX_VALUE so this example could be confusing - I think just drop that sentence. > > Minor formatting in passing. At L128 and L339 it would be easier to read if the "throws IOException" were on the previous line. Also L355 might be a bit clear if the Math.min was indented (have to look twice to see that it's not in the while body). > > -Alan From Alan.Bateman at oracle.com Mon Jan 22 09:14:59 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Jan 2018 09:14:59 +0000 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> Message-ID: <9264ed50-8d2d-4ab3-c731-16226d165bb0@oracle.com> On 22/01/2018 08:44, Peter Levart wrote: > > The delegation to public method (readAllBytes -> readNBytes) should > then be documented so that subclasses know that overriding readNBytes, > if needed, is sufficient. It doesn't delegate in the latest version of the patch so the documentation is right. However, I think you bring up a good point as specifying it that way means that sub-classes need only override readNBytes. -Alan From weijun.wang at oracle.com Mon Jan 22 10:42:00 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 22 Jan 2018 18:42:00 +0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <26706258-adaa-e7c1-240d-cf8a6e55ca09@oracle.com> Message-ID: > On Jan 22, 2018, at 5:01 PM, Weijun Wang wrote: > > I wonder when readNBytes(n) will be useful if the return value has less than n bytes. I mean I have seen protocols saying reading rest of the content, or reading N bytes, but never reading at most N bytes. I would rather the method throwing an EOFException if there are less than N bytes. This is also what DataInput::readFully is doing. --Max > >> On Jan 22, 2018, at 4:52 PM, Alan Bateman wrote: >> >> >> >> On 17/01/2018 16:24, Brian Burkhalter wrote: >>> : >>> >>> A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. >>> >>> http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ >>> >> The updated version looks good. I just wonde about the "For example ..." in the @throws OOME description. The API can't be called with a len > Integer.MAX_VALUE so this example could be confusing - I think just drop that sentence. >> >> Minor formatting in passing. At L128 and L339 it would be easier to read if the "throws IOException" were on the previous line. Also L355 might be a bit clear if the Math.min was indented (have to look twice to see that it's not in the while body). >> >> -Alan > From Roger.Riggs at Oracle.com Mon Jan 22 14:16:50 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 22 Jan 2018 09:16:50 -0500 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <26706258-adaa-e7c1-240d-cf8a6e55ca09@oracle.com> Message-ID: Hi Max, Both methods that throw and not throw have been proposed. But adding two methods seems like to too much clutter in the API and the methods appear too similar. The (at most) readNBytes method provides the needed functionality of allocating/reallocating the array and performing well. For the cases where reading less than N is significant, the caller should be checking the return length to validate it got the input it expected. $.02, Roger On 1/22/2018 5:42 AM, Weijun Wang wrote: >> On Jan 22, 2018, at 5:01 PM, Weijun Wang wrote: >> >> I wonder when readNBytes(n) will be useful if the return value has less than n bytes. > I mean I have seen protocols saying reading rest of the content, or reading N bytes, but never reading at most N bytes. I would rather the method throwing an EOFException if there are less than N bytes. This is also what DataInput::readFully is doing. > > --Max > >>> On Jan 22, 2018, at 4:52 PM, Alan Bateman wrote: >>> >>> >>> >>> On 17/01/2018 16:24, Brian Burkhalter wrote: >>>> : >>>> >>>> A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. >>>> >>>> http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ >>>> >>> The updated version looks good. I just wonde about the "For example ..." in the @throws OOME description. The API can't be called with a len > Integer.MAX_VALUE so this example could be confusing - I think just drop that sentence. >>> >>> Minor formatting in passing. At L128 and L339 it would be easier to read if the "throws IOException" were on the previous line. Also L355 might be a bit clear if the Math.min was indented (have to look twice to see that it's not in the while body). >>> >>> -Alan From Roger.Riggs at Oracle.com Mon Jan 22 14:25:00 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 22 Jan 2018 09:25:00 -0500 Subject: Microsecond support in java.time.Duration/Instant? In-Reply-To: References: Message-ID: Hi Kurt, In JSR 310, the thinking was that higher precision was desirable and that adding micros methods just bulked up the API without sufficient payback. At the time there were insufficient use cases to motivate the addition.? If the use is more common than previously thought it is a straightforward addition. Regards, Roger On 1/21/2018 9:58 PM, Kurt Alfred Kluever wrote: > Hi core-libs-dev, > > At Google, we often use microsecond precision for instants/durations > (probably because of historical reasons). Therefore, we have a handful of > static utility functions for creating/converting java.time types and > microseconds: > > public static Duration ofMicros(long micros) > public static long toMicros(Duration duration) > > public static Instant ofEpochMicros(long micros) > public static long toEpochMicros(Instant instant) > > Duration includes APIs for creating/converting to days, hours, minutes, > seconds, millis, and nanos. Instant includes APIs for creating/converting > to seconds and millis. Micros are noticeably absent. > > I'm curious how these sets of units were chosen or decided upon? I > understand that the line must be drawn somewhere (or else someone may come > along asking for centisecond support), but I'm curious as to the rational. > > Thanks! > From scolebourne at joda.org Mon Jan 22 15:00:11 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 22 Jan 2018 15:00:11 +0000 Subject: Microsecond support in java.time.Duration/Instant? In-Reply-To: References: Message-ID: On 22 January 2018 at 02:58, Kurt Alfred Kluever wrote: > I'm curious how these sets of units were chosen or decided upon? I > understand that the line must be drawn somewhere (or else someone may come > along asking for centisecond support), but I'm curious as to the rational. Nanos have to be supported as they are the smallest available. Millis are supported as they are the historic form. Micros is only one of the other possible ones - .NET ticks might be another. A line has to be drawn somewhere... Stephen From weijun.wang at oracle.com Mon Jan 22 15:05:11 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Mon, 22 Jan 2018 23:05:11 +0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <26706258-adaa-e7c1-240d-cf8a6e55ca09@oracle.com> Message-ID: <83B63ACF-EDF9-498D-8117-F1BF98203814@oracle.com> > On Jan 22, 2018, at 10:16 PM, Roger Riggs wrote: > > Hi Max, > > Both methods that throw and not throw have been proposed. But adding two methods > seems like to too much clutter in the API and the methods appear too similar. Sorry I wasn't aware of earlier discussions on this. My feeling is that readNBytesFully() is more useful than readNBytesAtMost(). > > The (at most) readNBytes method provides the needed functionality of allocating/reallocating > the array and performing well. Sure, this is a good enhancement. I understand it's easy to add a check to readNBytesAtMost() to get the function of readNBytesFully(), and the reverse is not possible. > > For the cases where reading less than N is significant, the caller should be checking the > return length to validate it got the input it expected. I've checked some of the codes I wrote and each time readNBytes() can be used I only want the return value to be of exactly N bytes. Thanks Max > > $.02, Roger > > On 1/22/2018 5:42 AM, Weijun Wang wrote: >>> On Jan 22, 2018, at 5:01 PM, Weijun Wang wrote: >>> >>> I wonder when readNBytes(n) will be useful if the return value has less than n bytes. >> I mean I have seen protocols saying reading rest of the content, or reading N bytes, but never reading at most N bytes. I would rather the method throwing an EOFException if there are less than N bytes. This is also what DataInput::readFully is doing. >> >> --Max >> >>>> On Jan 22, 2018, at 4:52 PM, Alan Bateman wrote: >>>> >>>> >>>> >>>> On 17/01/2018 16:24, Brian Burkhalter wrote: >>>>> : >>>>> >>>>> A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. >>>>> >>>>> http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ >>>>> >>>> The updated version looks good. I just wonde about the "For example ..." in the @throws OOME description. The API can't be called with a len > Integer.MAX_VALUE so this example could be confusing - I think just drop that sentence. >>>> >>>> Minor formatting in passing. At L128 and L339 it would be easier to read if the "throws IOException" were on the previous line. Also L355 might be a bit clear if the Math.min was indented (have to look twice to see that it's not in the while body). >>>> >>>> -Alan > From scolebourne at joda.org Mon Jan 22 15:11:03 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Mon, 22 Jan 2018 15:11:03 +0000 Subject: Bug in javac release flag? Message-ID: I think I have a problem with the java release flag, reported by a user here: https://github.com/ThreeTen/threeten-extra/issues/91 The particular case that is a problem are the new methods on java.lang.Math. Java 8 had these methods: floorDiv(long,long) floorDiv(int,int) Java 9 has these methods: floorDiv(long,long) floorDiv(int,int) floorDiv(long,int) (there are similar new overloads for other method names) Clearly, if you build on Java 9, code can link to the new(long,int) overload, but that code will get a NoSuchMethodError on Java 8. But, I am using Java 9 with the javac release flag set to be 8. If I understand the release flag correctly, the (long,int) overload should not be visible, and thus a different overload should be selected. So, is this a bug? Perhaps the tool that captures new method signatures doesn't handle overloads? (ThreeTen-Extra v1.3 was built using Maven on Java 9 using Maven's implementation of the release flag, so it could be Maven that is buggy, but I don't think so as it produces the correct classfile version number). thanks Stephen From jonathan.gibbons at oracle.com Mon Jan 22 15:34:54 2018 From: jonathan.gibbons at oracle.com (Jonathan Gibbons) Date: Mon, 22 Jan 2018 07:34:54 -0800 Subject: Bug in javac release flag? In-Reply-To: References: Message-ID: Forwarding to compiler-dev, which is a better place to discuss this issue. -- Jon On 1/22/18 7:11 AM, Stephen Colebourne wrote: > I think I have a problem with the java release flag, reported by a user here: > https://github.com/ThreeTen/threeten-extra/issues/91 > > The particular case that is a problem are the new methods on java.lang.Math. > > Java 8 had these methods: > floorDiv(long,long) > floorDiv(int,int) > > Java 9 has these methods: > floorDiv(long,long) > floorDiv(int,int) > floorDiv(long,int) > > (there are similar new overloads for other method names) > > Clearly, if you build on Java 9, code can link to the new(long,int) > overload, but that code will get a NoSuchMethodError on Java 8. > > But, I am using Java 9 with the javac release flag set to be 8. If I > understand the release flag correctly, the (long,int) overload should > not be visible, and thus a different overload should be selected. > > So, is this a bug? Perhaps the tool that captures new method > signatures doesn't handle overloads? > > (ThreeTen-Extra v1.3 was built using Maven on Java 9 using Maven's > implementation of the release flag, so it could be Maven that is > buggy, but I don't think so as it produces the correct classfile > version number). > > thanks > Stephen From brian.burkhalter at oracle.com Mon Jan 22 15:51:50 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 22 Jan 2018 07:51:50 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <83B63ACF-EDF9-498D-8117-F1BF98203814@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <26706258-adaa-e7c1-240d-cf8a6e55ca09@oracle.com> <83B63ACF-EDF9-498D-8117-F1BF98203814@oracle.com> Message-ID: On Jan 22, 2018, at 7:05 AM, Weijun Wang wrote: >> Both methods that throw and not throw have been proposed. But adding two methods >> seems like to too much clutter in the API and the methods appear too similar. > > Sorry I wasn't aware of earlier discussions on this. My feeling is that readNBytesFully() is more useful than readNBytesAtMost(). There was also a proposal to add a single method byte[] readNBytes(int len, boolean exact) which would read at most ?len? bytes but would throw an IOE if the length of the returned array were less than ?len? and ?exact? were true. This was rejected as being too confusing. Brian From adam.petcher at oracle.com Mon Jan 22 15:56:12 2018 From: adam.petcher at oracle.com (Adam Petcher) Date: Mon, 22 Jan 2018 10:56:12 -0500 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> Message-ID: The spec of the new method doesn't give me enough information to determine whether it is safe to call it when the value of the length argument is much larger than the number of bytes I expect to actually read. This use case comes up frequently in security libraries, because we have to handle length values that were chosen by an attacker. Would it be possible to add a sentence or two to the spec to clarify this situation? Possible wording, if this method can be called with large length values: "The total amount of memory allocated by this method is proportional to the number of bytes read from the stream. Therefore, the method may be safely called with very large values of {@code len}. Possible wording, otherwise: "The total amount of memory allocated by this method may be proportional to the value of {@code len}. Therefore, calling this method with very large values of {@code len} is not recommended." On 1/17/2018 11:24 AM, Brian Burkhalter wrote: > The proposed change has been modified to replace the two methods > > byte[] InputStream.readAllBytes(int) // reads at most ?len? bytes > byte[] InputStream.readNBytes(int) // reads exactly ?len? bytes or throws IOException > > with a single method > > byte[] InputStream.readNBytes(int) // reads at most ?len? bytes > > A negative value of ?len? will now cause an IllegalArgumentException instead of an IndexOutOfBoundsException. Also some verbiage has been improved. > > http://cr.openjdk.java.net/~bpb/8139206/webrev.01/ > > Thanks, > > Brian > > On Jan 16, 2018, at 11:17 AM, Brian Burkhalter wrote: > >> https://bugs.openjdk.java.net/browse/JDK-8139206 >> http://cr.openjdk.java.net/~bpb/8139206/webrev.00/ >> >> This change would add a new method ?byte[] InputStream.readNBytes(int len)? which would read up to at most ?len? bytes from the stream and return them in an internally allocated array. From brian.burkhalter at oracle.com Mon Jan 22 16:46:31 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 22 Jan 2018 08:46:31 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> Message-ID: <743D5953-B123-4563-8916-8B8536F5BA74@oracle.com> Certainly some verbiage like that could be added. Going back to last month in the discussion about improving the performance of readAllBytes() I calculated the exact number of bytes allocated [1]. For the initial implementation in that change this was B + L for L <= B N = B + 2*L for L > B where B is the internal buffer size and L is len. I would need to double check the accuracy of the above but I think your suggestion of ?proportional? is sufficient. Thanks, Brian [1] http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050686.html On Jan 22, 2018, at 7:56 AM, Adam Petcher wrote: > The spec of the new method doesn't give me enough information to determine whether it is safe to call it when the value of the length argument is much larger than the number of bytes I expect to actually read. This use case comes up frequently in security libraries, because we have to handle length values that were chosen by an attacker. Would it be possible to add a sentence or two to the spec to clarify this situation? > > Possible wording, if this method can be called with large length values: > > "The total amount of memory allocated by this method is proportional to the number of bytes read from the stream. Therefore, the method may be safely called with very large values of {@code len}. > > Possible wording, otherwise: > > "The total amount of memory allocated by this method may be proportional to the value of {@code len}. Therefore, calling this method with very large values of {@code len} is not recommended." From daniel.fuchs at oracle.com Mon Jan 22 16:47:05 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 22 Jan 2018 16:47:05 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> Message-ID: <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Hi guys, Here is my second take at solving the issue. There's no change in the public API (no CSR, Yay!), and it shouldn't require any changes on the Tomcat side. However I haven't tested my changes with Tomcat yet. Mark - would you be able to help me with that? http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.01/ best regards, -- daniel On 20/01/2018 13:17, Daniel Fuchs wrote: > Hi Mark, > > Thanks for these links and for trying out a solution. > > > On 19/01/18 20:14, Mark Thomas wrote: > > Thanks again for the hints. I have a work-around that seems to work: > > https://svn.apache.org/viewvc?view=revision&revision=1821708 > > I see... so what I think is happening is that in JDK 8, the > handlers from ".handlers" where added to the root logger from > within addLogger(rootLogger) and because ClassLoaderLogManager > overrides LogManager::addLogger, then that part was skipped. > > In JDK 9 - that behavior changed - the handlers from ".handlers" > were no longer added because the root logger was already present > in the context when addLogger got called (that was a regression), > but since? ClassLoaderLogManager overrides LogManager::addLogger > anyway - it made no difference for ClassLoaderLogManager. > > In JDK 10, to fix the issue, I forced a call that would > add the handlers from ".handlers" to the root logger, but > because this now happens outside of addLogger(rootLogger) (just after) > then ClassLoaderLogManager feels the difference. > > hmmm... let me think on this a bit more. > > best regards, > > -- daniel From bob.vandette at oracle.com Mon Jan 22 17:20:17 2018 From: bob.vandette at oracle.com (Bob Vandette) Date: Mon, 22 Jan 2018 12:20:17 -0500 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers Message-ID: Please review this change that resolves the detection of Java processes that are running in cgroup based containers. This latest (and hopefully final) update of this fix addresses comments from David Holmes and Mandy Chung. Bug: https://bugs.openjdk.java.net/browse/JDK-8193710 Webrev: http://cr.openjdk.java.net/~bobv/8193710/webrev.02/ Summary: This changeset enables the ability to use jcmd and jps running on a Host to list the java processes that are running in docker (cgroup based) containers. I?ve tested this change by examining processes running as root on both host and in docker containers as well as under my userid using ?jps and jcmd -l?. I?ve also tested updates to the getFile functions with a small example program that I wrote. Here are some implementation details that I?ve added to the Linux specific implementation class: src/jdk.internal.jvmstat/linux/classes/sun/jvmstat/PlatformSupportImpl.java /* Implementation Details: * * Java processes that run in docker containers are typically running * under cgroups with separate pid namespaces which means that pids * within the container are different that the pid which is visible * from the host. The container pids typically start with 1 and * increase. The java process running in the container will use these * pids when creating the hsperfdata files. In order to locate java * processes that are running in containers, we take advantage of * the Linux proc file system which maps the containers tmp directory * to the hosts under /proc/{hostpid}/root/tmp. We use the /proc status * file /proc/{hostpid}/status to determine the containers pid and * then access the hsperfdata file. The status file contains an * entry "NSPid:" which shows the mapping from the hostpid to the * containers pid. * * Example: * * NSPid: 24345 11 * * In this example process 24345 is visible from the host, * is running under the PID namespace and has a container specific * pid of 11. * * The search for Java processes is done by first looking in the * traditional /tmp for host process hsperfdata files and then * the search will container in every /proc/*/root/tmp directory. * There are of course added complications to this search that * need to be taken into account. * * 1. duplication of tmp directories * * /proc/{hostpid}/root/tmp directories exist for many processes * that are running on a Linux kernel that has cgroups enabled even * if they are not running in a container. To avoid this duplication, * we compare the inode of the /proc tmp directories to /tmp and * skip these duplicated directories. * * 2. Containerized processes without PID namespaces being enabled. * * If a container is running a Java process without namespaces being * enabled, an hsperfdata file will only be located at * /proc/{hostpid}/root/tmp/{hostpid}. This is handled by * checking the last component in the path for both the hostpid * and potential namespacepids (if one exists). */ Bob. From markt at apache.org Mon Jan 22 17:33:48 2018 From: markt at apache.org (Mark Thomas) Date: Mon, 22 Jan 2018 17:33:48 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: On 22/01/18 16:47, Daniel Fuchs wrote: > Hi guys, > > Here is my second take at solving the issue. > There's no change in the public API (no CSR, Yay!), > and it shouldn't require any changes on the Tomcat > side. However I haven't tested my changes with Tomcat > yet. > > Mark - would you be able to help me with that? Sure. I suspect it will be easier for you to download a Tomcat release and run it that it will be for me to try and build the JRE. That said, if you can point me to a snapshot JRE build (Linux, Windows or OSX) I can test it. The steps to test this should be: 1. Download Tomcat 9.0.2 (or 9.0.4 which should appear in the next 24 hours) from https://tomcat.apache.org/download-90.cgi (zip or tar.gz binary distribution. 2. Unpack it to $UNPACK_DIR 3. Set he JAVA_HOME environment variable to point to the JRE you want to test. 4. Open a command prompt. 5. Navigate to $UNPACK_DIR/apache-tomcat-9.0.2/bin 6. Call start.[sh|bat] as appropriate for your platform. 7. Tomcat should start without stack-trace appearing on the console. Notes: - You should see a stack trace if you use Java10 ea38 - You wont see a stack trace if you build Tomcat from source (the workaround is in all current versions) Mark > > http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.01/ > > best regards, > > -- daniel > > On 20/01/2018 13:17, Daniel Fuchs wrote: >> Hi Mark, >> >> Thanks for these links and for trying out a solution. >> >> ?> On 19/01/18 20:14, Mark Thomas wrote: >> ?> Thanks again for the hints. I have a work-around that seems to work: >> ?> https://svn.apache.org/viewvc?view=revision&revision=1821708 >> >> I see... so what I think is happening is that in JDK 8, the >> handlers from ".handlers" where added to the root logger from >> within addLogger(rootLogger) and because ClassLoaderLogManager >> overrides LogManager::addLogger, then that part was skipped. >> >> In JDK 9 - that behavior changed - the handlers from ".handlers" >> were no longer added because the root logger was already present >> in the context when addLogger got called (that was a regression), >> but since? ClassLoaderLogManager overrides LogManager::addLogger >> anyway - it made no difference for ClassLoaderLogManager. >> >> In JDK 10, to fix the issue, I forced a call that would >> add the handlers from ".handlers" to the root logger, but >> because this now happens outside of addLogger(rootLogger) (just after) >> then ClassLoaderLogManager feels the difference. >> >> hmmm... let me think on this a bit more. >> >> best regards, >> >> -- daniel > From daniel.fuchs at oracle.com Mon Jan 22 18:16:02 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Mon, 22 Jan 2018 18:16:02 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: Hi Mark, On 22/01/2018 17:33, Mark Thomas wrote: > Notes: > - You should see a stack trace if you use Java10 ea38 I'm happy to report that I am observing the stack trace in catalina.out without my fix, and that it is no longer there with my fix applied. > - You wont see a stack trace if you build Tomcat from source (the > workaround is in all current versions) I haven't tried to build Tomcat from the sources. Thanks for reporting the issue and for all the help in trying to pinpoint it! best regards, -- daniel On 22/01/18 16:47, Daniel Fuchs wrote: > > http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.01/ > > best regards, > > -- daniel From mandy.chung at oracle.com Mon Jan 22 18:32:32 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 22 Jan 2018 10:32:32 -0800 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers In-Reply-To: References: Message-ID: On 1/22/18 9:20 AM, Bob Vandette wrote: > : > > Webrev: > > http://cr.openjdk.java.net/~bobv/8193710/webrev.02/ Thanks for the update.? Looks good to me. It would be good to add a comment in PlatformSupport::getInstance to explain that PlatformSupport provides the default implementation that can be overridden by a platform-specific version PlatformSupportImpl, if present. We discussed that PlatformSupport::getLocalVmId could throw IAE rather than NumberFormatException if the given file is invalid. This carries from existing jvmstat code and so agree to leave it as a future clean up.? PlatformSupportImpl::getTemporaryDirectories can be simplified with java.nio.file.Path but what you have is fine with me. Mandy From markt at apache.org Mon Jan 22 19:22:43 2018 From: markt at apache.org (Mark Thomas) Date: Mon, 22 Jan 2018 19:22:43 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: <3af2e998-b00f-0f89-bb8b-559dd31cd10e@apache.org> On 22/01/18 18:16, Daniel Fuchs wrote: > Hi Mark, > > On 22/01/2018 17:33, Mark Thomas wrote: >> Notes: >> - You should see a stack trace if you use Java10 ea38 > > I'm happy to report that I am observing the stack trace > in catalina.out without my fix, and that it is no longer > there with my fix applied. > >> - You wont see a stack trace if you build Tomcat from source (the >> ?? workaround is in all current versions) > > I haven't tried to build Tomcat from the sources. > > Thanks for reporting the issue and for all the help > in trying to pinpoint it! Excellent news. Thanks for all your efforts on this. Mark > > best regards, > > -- daniel > > On 22/01/18 16:47, Daniel Fuchs wrote: >> >> http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.01/ >> >> best regards, >> >> -- daniel From Roger.Riggs at Oracle.com Mon Jan 22 20:03:33 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 22 Jan 2018 15:03:33 -0500 Subject: JDK-6372077: JarFile.getManifest() should handle manifest attribute names up to 70 bytes In-Reply-To: References: <9a4edee0-4edd-caaa-9970-61df083d39df@paratix.ch> <02abcb68-2294-c733-5048-b1f81ee99435@oracle.com> <88b89995-cfe0-d439-d63d-699c628e2315@paratix.ch> <07e62d8d-ae3a-9ffa-d426-81ab548ad252@Oracle.com> Message-ID: <9a08b2c0-3659-1833-f113-23aaceca84bc@Oracle.com> Hi Philipp, I'm tending to agree with the suggestion about line length interpretation. To meet OpenJDK IP requirements, please attach the .patch file or include it in the body of the message. Thanks, Roger On 12/18/2017 11:17 PM, Philipp Kunz wrote: > Hi Roger, > > My suggested and also preferred approach is to read the manifest > specification [1] in a way such that the line breaks are not included > when counting the maximum line length. The specification does not > state explicitly whether or not to include line breaks within the > maximum line length limit but the following sentence from the > specifications gives a hint: > > ??? Because header names cannot be continued, the maximum length of a > ??? header name is 70 bytes (there must be a colon and a SPACE after the > ??? name). > > Above statement can be true only if line breaks are not counted for > the maximum line length limit. Assuming so would in my opinion allow > to understand the complete manifest specification without a conflict > and effectively result in wider manifest files (maximum each line), > wider by two bytes of a line break. > > In the meantime since the mail you replied to, I created a patch [3] > mentioned in [2] which might be useful provided the manifest > specification discussion is resolved. > > Regards, > Philipp > > > [1] > https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Notes_on_Manifest_and_Signature_Files > / > https://docs.oracle.com/javase/9/docs/specs/jar/jar.html#Notes_on_Manifest_and_Signature_Files > [2] > http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050500.html > [3] http://files.paratix.ch/jdk/6372077/webrev.01/ > > > > On 18.12.2017 16:46, Roger Riggs wrote: >> Hi Phillip, >> >> Sorry for the silence... >> >> I/we haven't had time to full understand the ramifications of the >> change you propose. >> It seems there is a difficult/unresolvable conflict in the >> specifications between the line length >> requirements and the header specs. >> >> Regards, Roger >> >> On 11/21/2017 1:18 AM, Philipp Kunz wrote: >>> Hi everyone, >>> >>> I haven't got any reply now for around three weeks and now i start >>> to wonder if I just missed it or if I should refine my approach to >>> find a sponsor or if it helped if I presented a ready patch or if >>> noone considers this important enough or anything else whatever. >>> This is only my second contribution hence I don't know the procedure >>> well. >>> >>> One point maybe worth to point out again is that I don't want to >>> support manifest headers longer than 70 character, just up to 70, >>> which has always been the intention but has only worked up to 68. >>> This might have been written confusingly in my last email. >>> >>> As far as I understood, I should first get a sponsor. In any case, >>> is there any suggestion for how to proceed? >>> >>> Regards, >>> Philipp >>> >>> >>> >>> On 03.11.2017 00:04, Philipp Kunz wrote: >>>> Hi Sean and Max and all others, >>>> >>>> Thank you Sean for the hint about the right mailing list. And >>>> thanks also for his hint to Max to make smaller portions of changes. >>>> >>>> I would like to contribute a fix for JDK-6372077 which is about >>>> JarFile.getManifest() should handle manifest attribute name[s >>>> longer than] 70 bytes. >>>> >>>> It looks like the bug is caused by Manifest.make72Safe breaking >>>> lines at 70 bytes instead of 72 despite its comment and name >>>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Manifest.java#l176).The >>>> resulting StringBuffer has then lines of 72 bytes maximum each >>>> including the following line break. Without the line break that >>>> leaves 70 bytes of characters per line. On the other hand, header >>>> names can be up to 70 bytes (only single-byte utf-8 characters) and >>>> cannot be broken across a line break and need to be followed by a >>>> colon and a space which must be on the same line too according to >>>> the specification. When breaking at 70 bytes excluding the line >>>> break, however, long header names don't fit in one line together >>>> with the colon space delimiter because there is not sufficient space. >>>> Manifests with names up to 70 bytes long can still be written >>>> without immediate exception but the resulting manifests are illegal >>>> in my opinion. When later reading such manifests >>>> (http://hg.openjdk.java.net/jdk10/master/file/tip/src/java.base/share/classes/java/util/jar/Attributes.java#l406), >>>> an error occurs as a consequence of the bad manifest. This is more >>>> or less the current situation and also what JDK-6372077 already knew. >>>> >>>> ?--> After all, in order to fix it, i'd like to propose to make >>>> manifest file lines wider by two bytes. >>>> >>>> The only proper alternative that came into my mind would be to >>>> change the manifest specification and reduce the maximum header >>>> name length there by two and also in the code. If that would break >>>> any existing code i guess that would affect code only that produced >>>> invalid manifests and would be acceptable. >>>> >>>> Supporting all existing and possibly invalid manifests would mean >>>> to add support for reading headers the delimiters of which are >>>> broken onto the next line which I consider too complex with respect >>>> to the value added and even more so considering that those invalid >>>> manifest can be assumed to have been detected as such by reading >>>> them and also because it would be a feature that would be used less >>>> and less over time if the code to write manifest is changed at the >>>> same time to produce only valid manifests in the discussed respect >>>> here. I don't think this should be actually done. >>>> >>>> Before I actually do the leg work, i'd like to ask, if there are >>>> concerns or objections or tips for such a change or if anyone can >>>> or cannot follow the reasoning and the conclusion to make manifests >>>> 2 bytes wider or if i missed an important point altogether. >>>> >>>> In case there will be a consent about how to solve this, would >>>> someone volunteer to sponsor? That may be less urgent at the moment >>>> than the question above about how to proceed. >>>> >>>> Philipp >>>> >>>> >>>> On 12.10.2017 22:32, Sean Mullan wrote: >>>>> Hi Phillip, >>>>> >>>>> All of these bugs are in the core-libs area, so I am copying the >>>>> core-libs-dev list since that is where they should be discussed >>>>> and reviewed. I have -bcc-ed security-dev (where this was >>>>> originally posted). >>>>> >>>>> --Sean >>>>> >>>>> On 10/2/17 1:24 PM, Philipp Kunz wrote: >>>>>> Hi, >>>>>> >>>>>> While fixing JDK-6695402 I came across other related bugs to >>>>>> manifests such as JDK-6372077, JDK-6202130, JDK-8176843, >>>>>> JDK-4842483, and JDK-6443578 which all relate to manifest reading >>>>>> and writing. Somewhere bug 7071055 is mentioned but I cannot find >>>>>> anywhere. Another group of bugs, JDK-6910466, JDK-4935610, and >>>>>> JDK-4271239 concern the mandatory manifest main attributes >>>>>> Manifest-Version or Signature-Version and at first glance are >>>>>> duplicates. If you know of more known bugs, not necessarily >>>>>> present in jira, I'd be glad to get notified. >>>>>> >>>>>> There are also some comments about utf handling and line breaking >>>>>> in the code of Manifest: >>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l299 >>>>>> >>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l327 >>>>>> >>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l370 >>>>>> >>>>>> >>>>>> Furthermore, Attributes.map should declare appropriate type >>>>>> parameters: >>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l61 >>>>>> >>>>>> The specification would also require that `header names must not >>>>>> start with _, - or "From"` >>>>>> (http://docs.oracle.com/javase/7/docs/technotes/guides/jar/jar.html#Section-Specification) >>>>>> but I would opt not to add this as a hard restriction because I >>>>>> guess it can be assumed that such names are in use now after >>>>>> having been working for years. A warning to a logger might be >>>>>> conceivable such as in >>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Attributes.java#l424 >>>>>> >>>>>> Attribute values are never checked at all and invalid characters >>>>>> such as line breaks are never detected except that when reading >>>>>> the manifest again the values are cut off. >>>>>> The tab character (U+0008) does not work in manifest values. >>>>>> I suspect that there are also issues regarding the iteration >>>>>> order but I haven't got a prove yet unlike for the other points >>>>>> above: >>>>>> http://hg.openjdk.java.net/jdk10/master/file/a0116bcc65b7/src/java.base/share/classes/java/util/jar/Manifest.java#l54 >>>>>> >>>>>> There is duplicated or very similar code in Attributes and >>>>>> Manifest: Attributes.write-Manifest.write-Attributes.writeMain >>>>>> and Attributes.read-Manifest.read. >>>>>> Resolving JDK-6202130 would have the additional benefit to be >>>>>> able to view manifests with any utf-8 capable editor even if >>>>>> multi-byte characters break across lines. >>>>>> >>>>>> Fixing these issues individually looks like more complicated work >>>>>> than fixing them all at once, I guess, also because of a very low >>>>>> current test coverage. So I'd suggest to add some thorough tests >>>>>> along with fixing these issues. But before I start I would like >>>>>> to get some guidance, assistance or at least confirmation on how >>>>>> to proceed. I'm new to open jdk and have submitted only one patch >>>>>> so far. >>>>>> >>>>>> Is it ok to add tests for things that have worked before? >>>>>> Is it ok to refactor duplicated code just for the added value to >>>>>> reduce effort for testing? >>>>>> I assume compatibility to and from existing manifests is the >>>>>> highest priority, correct? This would also be the hard part in >>>>>> adding as complete test coverage as possible. What would be >>>>>> acceptable criteria to meet? >>>>>> Why does Attributes not extend LinkedHashMap and why does >>>>>> Manifest not extend HashMap? Any objection? >>>>>> While I would not want to write code that looks slow or change >>>>>> more than necessary one can never know before having performance >>>>>> actually measured. Is there some way this is dealt with or should >>>>>> I wait for such feedback until after patch submission? >>>>>> >>>>>> Would someone sponsor? >>>>>> >>>>>> Regards, >>>>>> Philipp >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ------------------------------------------------------------------------ >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Paratix GmbH >>>>>> St Peterhofstatt 11 >>>>>> 8001 Z?rich >>>>>> >>>>>> +41 (0)76 397 79 35 >>>>>> philipp.kunz at paratix.ch >>>> >>>> -- >>>> >>>> >>>> Gruss Philipp >>>> >>>> >>>> >>>> ------------------------------------------------------------------------ >>>> >>>> >>>> >>>> >>>> Paratix GmbH >>>> St Peterhofstatt 11 >>>> 8001 Z?rich >>>> >>>> +41 (0)76 397 79 35 >>>> philipp.kunz at paratix.ch >>> >> > From brian.burkhalter at oracle.com Mon Jan 22 20:49:35 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 22 Jan 2018 12:49:35 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> Message-ID: <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> All of the comments included below are addressed in [1]. The difference versus webrev.01 are in [2]. The CSR [3] will have to move back to Draft, updated, and re-finalized but I will hold off on that until there is a final consensus on the verbiage. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8139206/webrev.02/ [2] http://cr.openjdk.java.net/~bpb/8139206/webrev.01-02/ [3] https://bugs.openjdk.java.net/browse/JDK-8194956 On Jan 19, 2018, at 11:49 AM, Roger Riggs wrote: > A pre-existing typo: > line 67 "{@code skip()}" should be "{@code skip(*long*)}". > > Since the public readNBytes suffices for readAllBytes, I would rename the private readAtMostNBytes > to readNBytes and avoid the duplication of javadoc. > > Keeping the existing readAllBytes before readNBytes in the source file will make the diff easier to follow > and the methods be in alphabetical order. On Jan 22, 2018, at 12:44 AM, Peter Levart wrote: > The delegation to public method (readAllBytes -> readNBytes) should then be documented so that subclasses know that overriding readNBytes, if needed, is sufficient. On Jan 22, 2018, at 12:52 AM, Alan Bateman wrote: > The updated version looks good. I just wonde about the "For example ..." in the @throws OOME description. The API can't be called with a len > Integer.MAX_VALUE so this example could be confusing - I think just drop that sentence. > > Minor formatting in passing. At L128 and L339 it would be easier to read if the "throws IOException" were on the previous line. Also L355 might be a bit clear if the Math.min was indented (have to look twice to see that it's not in the while body). On Jan 22, 2018, at 7:56 AM, Adam Petcher wrote: > Possible wording, if this method can be called with large length values: > > "The total amount of memory allocated by this method is proportional to the number of bytes read from the stream. Therefore, the method may be safely called with very large values of {@code len}. From andrey.x.nazarov at oracle.com Mon Jan 22 20:58:15 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Mon, 22 Jan 2018 12:58:15 -0800 Subject: RFR: JDK-8161348 Several tools/jlink tests failed due to time out Message-ID: Hi, please review Jlink tests. I?ve increased default timeout to 5 minutes and skip tests in ?-Xcomp? VM mode. CR: http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8161348 ?Andrei From joe.darcy at oracle.com Mon Jan 22 21:07:29 2018 From: joe.darcy at oracle.com (Joseph D. Darcy) Date: Mon, 22 Jan 2018 13:07:29 -0800 Subject: RFR: JDK-8161348 Several tools/jlink tests failed due to time out In-Reply-To: References: Message-ID: <5A665291.4020706@oracle.com> Hello, I'm wary of increasing the timeout to 5 minutes. When such tests are run on CI servers, the effective timeout can be 10 minutes or more given a timeout factor used for the test run. Should using 1GB of memory on the @run line have an earlier @requires guard for an amount of memory on the system? Cheers, -Joe On 1/22/2018 12:58 PM, Andrey Nazarov wrote: > Hi, > please review Jlink tests. > > I?ve increased default timeout to 5 minutes and skip tests in ?-Xcomp? VM mode. > > CR: http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8161348 > > ?Andrei From Alan.Bateman at oracle.com Mon Jan 22 21:09:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Jan 2018 21:09:22 +0000 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> Message-ID: <220f0779-4a22-b0f4-2e3c-f2090c7615dc@oracle.com> On 22/01/2018 20:49, Brian Burkhalter wrote: > All of the comments included below are addressed in [1]. The > difference versus webrev.01 are in [2]. The CSR [3] will have to move > back to Draft, updated, and re-finalized but I will hold off on that > until there is a final consensus on the verbiage. "has the same effect as .." is testable but I think it needs be something like "This method invokes readNBytes with a length of MAX_VALUE" to avoid needing to override readAllBytes. -Alan From brian.burkhalter at oracle.com Mon Jan 22 21:12:26 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 22 Jan 2018 13:12:26 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <784AE060-0FE4-4107-9410-6478B08542E1@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <220f0779-4a22-b0f4-2e3c-f2090c7615dc@oracle.com> <784AE060-0FE4-4107-9410-6478B08542E1@oracle.com> Message-ID: <266774A7-C2FA-429D-8C6C-DC0283364B65@oracle.com> On Jan 22, 2018, at 1:09 PM, Alan Bateman wrote: > On 22/01/2018 20:49, Brian Burkhalter wrote: >> All of the comments included below are addressed in [1]. The difference versus webrev.01 are in [2]. The CSR [3] will have to move back to Draft, updated, and re-finalized but I will hold off on that until there is a final consensus on the verbiage. > "has the same effect as .." is testable but I think it needs be something like "This method invokes readNBytes with a length of MAX_VALUE" to avoid needing to override readAllBytes. I can change it. The reason I used that verbiage was to match that of read(byte[]): "The read(b) method for class InputStream has the same effect as: read(b, 0, b.length)" Brian From Alan.Bateman at oracle.com Mon Jan 22 21:22:54 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Mon, 22 Jan 2018 21:22:54 +0000 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <266774A7-C2FA-429D-8C6C-DC0283364B65@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <220f0779-4a22-b0f4-2e3c-f2090c7615dc@oracle.com> <784AE060-0FE4-4107-9410-6478B08542E1@oracle.com> <266774A7-C2FA-429D-8C6C-DC0283364B65@oracle.com> Message-ID: <1876a2b5-a7bd-3a97-a6b0-1d54179151f2@oracle.com> On 22/01/2018 21:12, Brian Burkhalter wrote: > > I can change it. The reason I used that verbiage was to match that of > read(byte[]): > > ?"The |read(b)|?method for class |InputStream|?has the same effect as: > ?read(b, 0, b.length)" > True but the equivalent in readAllBytes won't guarantee that readNBytes is called (which I think is what you were trying to do in the latest version). -Alan From jason_mehrens at hotmail.com Mon Jan 22 21:30:09 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Mon, 22 Jan 2018 21:30:09 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com>, <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: Daniel, Fantastic! The patch looks good as is. The only thing that sticks out is that both the 'if' and the 'else if' have the same code inside them. That could be refactored a bit. Jason ________________________________________ From: core-libs-dev on behalf of Daniel Fuchs Sent: Monday, January 22, 2018 10:47 AM To: Mark Thomas; core-libs-dev at openjdk.java.net Subject: Re: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat Hi guys, Here is my second take at solving the issue. There's no change in the public API (no CSR, Yay!), and it shouldn't require any changes on the Tomcat side. However I haven't tested my changes with Tomcat yet. Mark - would you be able to help me with that? http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.01/ best regards, -- daniel On 20/01/2018 13:17, Daniel Fuchs wrote: > Hi Mark, > > Thanks for these links and for trying out a solution. > > > On 19/01/18 20:14, Mark Thomas wrote: > > Thanks again for the hints. I have a work-around that seems to work: > > https://svn.apache.org/viewvc?view=revision&revision=1821708 > > I see... so what I think is happening is that in JDK 8, the > handlers from ".handlers" where added to the root logger from > within addLogger(rootLogger) and because ClassLoaderLogManager > overrides LogManager::addLogger, then that part was skipped. > > In JDK 9 - that behavior changed - the handlers from ".handlers" > were no longer added because the root logger was already present > in the context when addLogger got called (that was a regression), > but since ClassLoaderLogManager overrides LogManager::addLogger > anyway - it made no difference for ClassLoaderLogManager. > > In JDK 10, to fix the issue, I forced a call that would > add the handlers from ".handlers" to the root logger, but > because this now happens outside of addLogger(rootLogger) (just after) > then ClassLoaderLogManager feels the difference. > > hmmm... let me think on this a bit more. > > best regards, > > -- daniel From brian.burkhalter at oracle.com Mon Jan 22 21:27:47 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 22 Jan 2018 13:27:47 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <1876a2b5-a7bd-3a97-a6b0-1d54179151f2@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <220f0779-4a22-b0f4-2e3c-f2090c7615dc@oracle.com> <784AE060-0FE4-4107-9410-6478B08542E1@oracle.com> <266774A7-C2FA-429D-8C6C-DC0283364B65@oracle.com> <1876a2b5-a7bd-3a97-a6b0-1d54179151f2@oracle.com> Message-ID: <731C3472-6913-49F7-95AF-E36DB0B76B7D@oracle.com> On Jan 22, 2018, at 1:22 PM, Alan Bateman wrote: > On 22/01/2018 21:12, Brian Burkhalter wrote: >> >> I can change it. The reason I used that verbiage was to match that of read(byte[]): >> >> "The read(b) method for class InputStream has the same effect as: >> read(b, 0, b.length)" >> > True but the equivalent in readAllBytes won't guarantee that readNBytes is called (which I think is what you were trying to do in the latest version). I have updated this verbiage in place in webrev.02 and webrev.01-02. Thanks, Brian From david.holmes at oracle.com Mon Jan 22 22:15:46 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Jan 2018 08:15:46 +1000 Subject: RFR: 8193710 - jcmd -l and jps commands do not list Java processes running in Docker containers In-Reply-To: References: Message-ID: Thanks Bob. Seems okay. David On 23/01/2018 3:20 AM, Bob Vandette wrote: > Please review this change that resolves the detection of Java processes that are running in cgroup > based containers. > > This latest (and hopefully final) update of this fix addresses comments from David Holmes and Mandy Chung. > > Bug: > > https://bugs.openjdk.java.net/browse/JDK-8193710 > > Webrev: > > http://cr.openjdk.java.net/~bobv/8193710/webrev.02/ > > Summary: > > This changeset enables the ability to use jcmd and jps running on a Host to > list the java processes that are running in docker (cgroup based) containers. > > I?ve tested this change by examining processes running as root on both host and in > docker containers as well as under my userid using ?jps and jcmd -l?. > I?ve also tested updates to the getFile functions with a small example program that I wrote. > > > Here are some implementation details that I?ve added to the Linux specific implementation class: > > src/jdk.internal.jvmstat/linux/classes/sun/jvmstat/PlatformSupportImpl.java > > /* Implementation Details: > * > * Java processes that run in docker containers are typically running > * under cgroups with separate pid namespaces which means that pids > * within the container are different that the pid which is visible > * from the host. The container pids typically start with 1 and > * increase. The java process running in the container will use these > * pids when creating the hsperfdata files. In order to locate java > * processes that are running in containers, we take advantage of > * the Linux proc file system which maps the containers tmp directory > * to the hosts under /proc/{hostpid}/root/tmp. We use the /proc status > * file /proc/{hostpid}/status to determine the containers pid and > * then access the hsperfdata file. The status file contains an > * entry "NSPid:" which shows the mapping from the hostpid to the > * containers pid. > * > * Example: > * > * NSPid: 24345 11 > * > * In this example process 24345 is visible from the host, > * is running under the PID namespace and has a container specific > * pid of 11. > * > * The search for Java processes is done by first looking in the > * traditional /tmp for host process hsperfdata files and then > * the search will container in every /proc/*/root/tmp directory. > * There are of course added complications to this search that > * need to be taken into account. > * > * 1. duplication of tmp directories > * > * /proc/{hostpid}/root/tmp directories exist for many processes > * that are running on a Linux kernel that has cgroups enabled even > * if they are not running in a container. To avoid this duplication, > * we compare the inode of the /proc tmp directories to /tmp and > * skip these duplicated directories. > * > * 2. Containerized processes without PID namespaces being enabled. > * > * If a container is running a Java process without namespaces being > * enabled, an hsperfdata file will only be located at > * /proc/{hostpid}/root/tmp/{hostpid}. This is handled by > * checking the last component in the path for both the hostpid > * and potential namespacepids (if one exists). > */ > > Bob. > From andrey.x.nazarov at oracle.com Mon Jan 22 23:33:02 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Mon, 22 Jan 2018 15:33:02 -0800 Subject: RFR: JDK-8161348 Several tools/jlink tests failed due to time out In-Reply-To: <5A665291.4020706@oracle.com> References: <5A665291.4020706@oracle.com> Message-ID: <42B07061-0508-459E-85F6-FCCAD99B6E44@oracle.com> > On 22 Jan 2018, at 13:07, Joseph D. Darcy wrote: > > Hello, > > I'm wary of increasing the timeout to 5 minutes. When such tests are run on CI servers, the effective timeout can be 10 minutes or more given a timeout factor used for the test run. We can move them to later tiers. On my Macbook pro 2015 + SSD JlinkTest.java runs around 40 seconds. It could easily be more than 2 min with HDD, working McAfee/indexer, slower hardware. So I think default timeout = 2 minutes is not enough. > > Should using 1GB of memory on the @run line have an earlier @requires guard for an amount of memory on the system? I think it should. Although it?s not related to the original issue, i?ll update patch with @requires > > Cheers, > > -Joe > > On 1/22/2018 12:58 PM, Andrey Nazarov wrote: >> Hi, >> please review Jlink tests. >> >> I?ve increased default timeout to 5 minutes and skip tests in ?-Xcomp? VM mode. >> >> CR: http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8161348 >> >> ?Andrei > From mandy.chung at oracle.com Mon Jan 22 23:52:14 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 22 Jan 2018 15:52:14 -0800 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: Hi Daniel, On 1/22/18 8:47 AM, Daniel Fuchs wrote: > > http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.01/ > : >> I see... so what I think is happening is that in JDK 8, the >> handlers from ".handlers" where added to the root logger from >> within addLogger(rootLogger) and because ClassLoaderLogManager >> overrides LogManager::addLogger, then that part was skipped. So the cause of JDK-8191033 is because cx::addLocalLogger returns false when addLogger is called the first time to add the root and global logger and it causes that ".handlers" is not read and loaded for the root logger. It took me some time to understand what it's happening.? When addLocalLogger is called to add a logger, it calls ensureAllDefaultLoggers to ensure the root and global loggers are added first.? Do you think the handling of ".handlers" should be done in LoggerContext::ensureDefaultLogger?? Would that avoid the check to call loadLoggerHandlers if it's called from ensureLogManagerInitialized??? The code is getting quite complicated and if moving it closer to the place it adds the root logger would help. thanks Mandy From andrey.x.nazarov at oracle.com Tue Jan 23 00:12:07 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Mon, 22 Jan 2018 16:12:07 -0800 Subject: RFR: JDK-8161348 Several tools/jlink tests failed due to time out In-Reply-To: <42B07061-0508-459E-85F6-FCCAD99B6E44@oracle.com> References: <5A665291.4020706@oracle.com> <42B07061-0508-459E-85F6-FCCAD99B6E44@oracle.com> Message-ID: > On 22 Jan 2018, at 15:33, Andrey Nazarov wrote: > > > >> On 22 Jan 2018, at 13:07, Joseph D. Darcy wrote: >> >> Hello, >> >> I'm wary of increasing the timeout to 5 minutes. When such tests are run on CI servers, the effective timeout can be 10 minutes or more given a timeout factor used for the test run. > We can move them to later tiers. On my Macbook pro 2015 + SSD JlinkTest.java runs around 40 seconds. It could easily be more than 2 min with HDD, working McAfee/indexer, slower hardware. So I think default timeout = 2 minutes is not enough. >> >> Should using 1GB of memory on the @run line have an earlier @requires guard for an amount of memory on the system? > I think it should. Although it?s not related to the original issue, i?ll update patch with @requires Updated http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.01/ >> >> Cheers, >> >> -Joe >> >> On 1/22/2018 12:58 PM, Andrey Nazarov wrote: >>> Hi, >>> please review Jlink tests. >>> >>> I?ve increased default timeout to 5 minutes and skip tests in ?-Xcomp? VM mode. >>> >>> CR: http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.00/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8161348 >>> >>> ?Andrei >> > From kak at google.com Tue Jan 23 05:23:09 2018 From: kak at google.com (Kurt Alfred Kluever) Date: Tue, 23 Jan 2018 00:23:09 -0500 Subject: Microsecond support in java.time.Duration/Instant? In-Reply-To: References: Message-ID: Thanks for the responses, Stephen + Roger,. As noted, a line definitely has to be drawn somewhere. In case anyone is looking for some data, here are current relative usage stats inside of Google for the various APIs, grouped by functionality (creating/decomposing Instants/Durations): Instant.ofEpochMilli(long): 67% Instant.ofEpochSecond(long): 29% * Instants.ofEpochMicros(long): 4%* Instant.toEpochMilli(): 83% Instant.getEpochSecond(): 10% * Instants.toEpochMicros(Instant): 7%* Duration.ofSeconds(long): 30% Duration.ofDays(long): 24% Duration.ofMillis(long): 21% Duration.ofMinutes(long): 18% Duration.ofHours(long): 7% Duration.ofNanos(long): < 1% * Durations.ofMicros(long): < 1%* Duration.toMillis(): 73% Duration.getSeconds(): 16% Duration.toMinutes(): 3% Duration.toNanos(): 3% Duration.toDays(): 3% * Durations.toMicros(Duration): 2%* Duration.toHours(): 1% So yea, it's definitely towards the bottom of the usage stats, but that also might be partially because of the discoverability issue (people are much more likely to find an instance method directly on the type than a static method on our Durations class). Anyway, I'm not claiming these numbers motivate any sort of change, but given a proposal to add microsecond support directly to the APIs, I think I'd be in favor :-) Or perhaps Google is unique in it's usage of microsecond precision (many of our storage systems have timestamps using microsecond precision). PS - and thanks for the ".NET ticks" reference, I hadn't heard of that before. And maybe here's a new one for you that just popped up in the news --- a Flick (1/705600000 of a second). On Mon, Jan 22, 2018 at 10:00 AM, Stephen Colebourne wrote: > On 22 January 2018 at 02:58, Kurt Alfred Kluever wrote: > > I'm curious how these sets of units were chosen or decided upon? I > > understand that the line must be drawn somewhere (or else someone may > come > > along asking for centisecond support), but I'm curious as to the > rational. > > Nanos have to be supported as they are the smallest available. > Millis are supported as they are the historic form. > > Micros is only one of the other possible ones - .NET ticks might be > another. A line has to be drawn somewhere... > > Stephen > -- kak From rachna.goel at oracle.com Tue Jan 23 06:15:34 2018 From: rachna.goel at oracle.com (Rachna Goel) Date: Tue, 23 Jan 2018 11:45:34 +0530 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: <46a67401-92e4-09a2-dfdf-2c10a977a71e@oracle.com> References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> <46a67401-92e4-09a2-dfdf-2c10a977a71e@oracle.com> Message-ID: Hi, Kindly review updated patch for this doc fix: patch: http://cr.openjdk.java.net/%7Ergoel/8146656/webrev.02/ Approved CSR : https://bugs.openjdk.java.net/browse/JDK-8191414 Thanks, Rachna On 20/12/17 10:44 PM, joe darcy wrote: > > Hi Rachna, > > I think the revised version with the @implSpec tag switch is > acceptable, but also think providing more text to describe this > situation would be helpful to programmers unaware of a 13 month > possibility. > > Cheers, > > -Joe > > > On 12/19/2017 2:08 AM, Rachna Goel wrote: >> >> Hi Joe, >> >> Thanks for the comments. >> >> I have updated the CSR to have @implSpec in place of @implNote. >> >> https://bugs.openjdk.java.net/browse/JDK-8191414 >> >> Regarding "An array with either 12 or 13 elements will be returned >> depending on whether or {@link Calendar.UNDECIMBER} is supported." , >> I would like to go with existing statement as this method always >> returns 13 elements where the 13th element may be empty string or may >> contain Calendar.UNDECIMBER, depending upon whether its supported by >> the Calendar instance. >> >> kindly suggest whether this looks fine! >> >> Thanks, >> Rachna >> >> >> On 19/12/17 2:55 PM, joe darcy wrote: >>> Hi Rachna, >>> >>> On 12/19/2017 1:13 AM, Rachna Goel wrote: >>>> >>>> Hello Joe, >>>> >>>> Thanks for the review. >>>> >>>> Reason I added @implNote is that it's the case for the default >>>> implementation. Not added as a part of spec, as some implementation >>>> can just return 12 element array for same methods through the >>>> "java.text.spi.DateFormatSymbolsProvider" SPI. >>>> >>>> >>> >>> That is precisely the sort of situation the @implSpec tag is >>> intended for. It allows the specification to say DateFormatSymbols >>> must behave this way while allowing subclasses to behave differently. >>> >>> Perhaps some general text can be added as normal specification, >>> something like >>> >>> "An array with either 12 or 13 elements will be returned depending >>> on whether or {@link Calendar.UNDECIMBER} is supported." >>> >>> paired with >>> >>> @implSpec This method returns 13 elements since @link >>> Calendar.UNDECIMBER} is supported. >>> >>> HTH, >>> >>> -Joe >>> >> >> -- >> Thanks, >> Rachna > -- Thanks, Rachna From joe.darcy at oracle.com Tue Jan 23 06:53:31 2018 From: joe.darcy at oracle.com (joe darcy) Date: Mon, 22 Jan 2018 22:53:31 -0800 Subject: JDK 11 RFR of JDK-8195987,,Problem list tools/launcher/HelpFlagsTest.java fails on windows Message-ID: Hello, The test ??? tools/launcher/HelpFlagsTest.java has been failing on windows (JDK-8195824) since the push for JDK-8195663. The test needs to be problem listed on windows until a full fix is developed. Please review the patch below. Cheers, -Joe --- a/test/jdk/ProblemList.txt??? Tue Jan 16 08:48:34 2018 +0100 +++ b/test/jdk/ProblemList.txt??? Mon Jan 22 22:52:53 2018 -0800 @@ -316,6 +316,8 @@ ?tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all +tools/launcher/HelpFlagsTest.java 8195824 windows-all + ?tools/jimage/JImageExtractTest.java 8170120 generic-all ?tools/jimage/JImageListTest.java 8170120 generic-all ?tools/jimage/JImageVerifyTest.java 8170120 generic-all From xu.y.yin at oracle.com Tue Jan 23 07:01:45 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 23 Jan 2018 15:01:45 +0800 Subject: RFR: 8195976: Add JNDI test javax/naming/dns/AttributeTests/GetAny.java Message-ID: <9CAEA585-EB5C-4F55-88D3-124E589778EB@oracle.com> Please review the added JNDI test javax/naming/dns/AttributeTests/GetAny.java, thanks It?s to cover the case that we can get the attributes of a DNS entry using special qualifiers. This test will use DNSServer to playback dump dns message (which created by DNSTracer when run test against real dns server) to simulate test environment. bug: https://bugs.openjdk.java.net/browse/JDK-8195976 webrev: http://cr.openjdk.java.net/~xiaofeya/8195976/webrev.00/ Regards Chris From david.holmes at oracle.com Tue Jan 23 07:02:41 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Jan 2018 17:02:41 +1000 Subject: JDK 11 RFR of JDK-8195987,,Problem list tools/launcher/HelpFlagsTest.java fails on windows In-Reply-To: References: Message-ID: Looks good! I hope this is the last failure we see from this one! Thanks, David On 23/01/2018 4:53 PM, joe darcy wrote: > Hello, > > The test > > ??? tools/launcher/HelpFlagsTest.java > > has been failing on windows (JDK-8195824) since the push for > JDK-8195663. The test needs to be problem listed on windows until a full > fix is developed. > > Please review the patch below. > > Cheers, > > -Joe > > --- a/test/jdk/ProblemList.txt??? Tue Jan 16 08:48:34 2018 +0100 > +++ b/test/jdk/ProblemList.txt??? Mon Jan 22 22:52:53 2018 -0800 > @@ -316,6 +316,8 @@ > > ?tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all > > +tools/launcher/HelpFlagsTest.java 8195824 windows-all > + > ?tools/jimage/JImageExtractTest.java 8170120 generic-all > ?tools/jimage/JImageListTest.java 8170120 generic-all > ?tools/jimage/JImageVerifyTest.java 8170120 generic-all > From goetz.lindenmaier at sap.com Tue Jan 23 07:33:42 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 23 Jan 2018 07:33:42 +0000 Subject: JDK 11 RFR of JDK-8195987,,Problem list tools/launcher/HelpFlagsTest.java fails on windows In-Reply-To: References: Message-ID: <7da1b20ce2de4cd3bc20a8514e1a81cd@sap.com> Hi, I recommend pushing the real fix instead. javacpl seems not to have a help message, so it just needs to be excluded from the test: http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev/ Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Dienstag, 23. Januar 2018 08:03 > To: joe darcy ; core-libs-dev dev at openjdk.java.net> > Cc: Kumar Srinivasan ; Lindenmaier, Goetz > ; Jesper Wilhelmsson > > Subject: Re: JDK 11 RFR of JDK-8195987,,Problem list > tools/launcher/HelpFlagsTest.java fails on windows > > Looks good! > > I hope this is the last failure we see from this one! > > Thanks, > David > > On 23/01/2018 4:53 PM, joe darcy wrote: > > Hello, > > > > The test > > > > ??? tools/launcher/HelpFlagsTest.java > > > > has been failing on windows (JDK-8195824) since the push for > > JDK-8195663. The test needs to be problem listed on windows until a full > > fix is developed. > > > > Please review the patch below. > > > > Cheers, > > > > -Joe > > > > --- a/test/jdk/ProblemList.txt??? Tue Jan 16 08:48:34 2018 +0100 > > +++ b/test/jdk/ProblemList.txt??? Mon Jan 22 22:52:53 2018 -0800 > > @@ -316,6 +316,8 @@ > > > > ?tools/launcher/FXLauncherTest.java 8068049 linux-all,macosx-all > > > > +tools/launcher/HelpFlagsTest.java 8195824 windows-all > > + > > ?tools/jimage/JImageExtractTest.java 8170120 generic-all > > ?tools/jimage/JImageListTest.java 8170120 generic-all > > ?tools/jimage/JImageVerifyTest.java 8170120 generic-all > > From goetz.lindenmaier at sap.com Tue Jan 23 07:36:01 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 23 Jan 2018 07:36:01 +0000 Subject: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError Message-ID: Hi, javacpl seems not to have a help message, so it just needs to be excluded from the test. Please review. http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev/ (I sent similar mail in reply to RE: JDK 11 RFR of JDK-8195987,,Problem list tools/launcher/HelpFlagsTest.java fails on windows) Best regards, Goetz. From david.holmes at oracle.com Tue Jan 23 07:50:02 2018 From: david.holmes at oracle.com (David Holmes) Date: Tue, 23 Jan 2018 17:50:02 +1000 Subject: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError In-Reply-To: References: Message-ID: <595de57b-50cb-ea86-0faa-ab393b672036@oracle.com> Hi Goetz, On 23/01/2018 5:36 PM, Lindenmaier, Goetz wrote: > Hi, > > javacpl seems not to have a help message, so it just needs to > be excluded from the test. Please review. > http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev/ That seems okay. Did you test it on Windows? Also ... > (I sent similar mail in reply to RE: JDK 11 RFR of JDK-8195987,,Problem list tools/launcher/HelpFlagsTest.java fails on windows) ... the ProblemList change was pushed so will have to be reverted by this change. Has this test now been tested on all platforms with both the OpenJDK and OracleJDK? Thanks, David > Best regards, > Goetz. > From Alan.Bateman at oracle.com Tue Jan 23 07:53:31 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Jan 2018 07:53:31 +0000 Subject: RFR: 8195976: Add JNDI test javax/naming/dns/AttributeTests/GetAny.java In-Reply-To: <9CAEA585-EB5C-4F55-88D3-124E589778EB@oracle.com> References: <9CAEA585-EB5C-4F55-88D3-124E589778EB@oracle.com> Message-ID: <441789eb-a0ed-5eee-1d74-af5a888e9e82@oracle.com> On 23/01/2018 07:01, Chris Yin wrote: > Please review the added JNDI test javax/naming/dns/AttributeTests/GetAny.java, thanks > > You may want to move it to com/sun/jndi/dns so that it's with the other tests for the DNS provider (as there is no javax.naming.dns API). Also I suspect you don't need "@modules jdk.naming.dns/com.sun.jndi.dns" as it doesn't appear to make direct use of the classes in the implementation. -Alan From Alan.Bateman at oracle.com Tue Jan 23 08:18:00 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 23 Jan 2018 08:18:00 +0000 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <731C3472-6913-49F7-95AF-E36DB0B76B7D@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <220f0779-4a22-b0f4-2e3c-f2090c7615dc@oracle.com> <784AE060-0FE4-4107-9410-6478B08542E1@oracle.com> <266774A7-C2FA-429D-8C6C-DC0283364B65@oracle.com> <1876a2b5-a7bd-3a97-a6b0-1d54179151f2@oracle.com> <731C3472-6913-49F7-95AF-E36DB0B76B7D@oracle.com> Message-ID: <14d855f2-2fe7-c432-b4f5-76d09a695ffa@oracle.com> On 22/01/2018 21:27, Brian Burkhalter wrote: > > I have updated this verbiage in place in webrev.02 and webrev.01-02. > Updated wording looks okay to me. -Alan From goetz.lindenmaier at sap.com Tue Jan 23 08:23:39 2018 From: goetz.lindenmaier at sap.com (Lindenmaier, Goetz) Date: Tue, 23 Jan 2018 08:23:39 +0000 Subject: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError In-Reply-To: <595de57b-50cb-ea86-0faa-ab393b672036@oracle.com> References: <595de57b-50cb-ea86-0faa-ab393b672036@oracle.com> Message-ID: <98a73ac560274baca08c8f9adc31ee31@sap.com> Hi, I can not do proper tests with OracleJDK, sorry for that. This webrev removes the problem from the ProblemList: http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev.02/ Actually, you should have an internal ProblemList for things that fail with the OracleJDK. We do so for SAP JVM and our OpenJDK builds, too. Best regards, Goetz. > -----Original Message----- > From: David Holmes [mailto:david.holmes at oracle.com] > Sent: Dienstag, 23. Januar 2018 08:50 > To: Lindenmaier, Goetz ; core-libs-dev libs-dev at openjdk.java.net> > Subject: Re: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java fails with > java.lang.AssertionError > > Hi Goetz, > > On 23/01/2018 5:36 PM, Lindenmaier, Goetz wrote: > > Hi, > > > > javacpl seems not to have a help message, so it just needs to > > be excluded from the test. Please review. > > http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev/ > > That seems okay. Did you test it on Windows? Also ... > > > (I sent similar mail in reply to RE: JDK 11 RFR of JDK-8195987,,Problem list > tools/launcher/HelpFlagsTest.java fails on windows) > > ... the ProblemList change was pushed so will have to be reverted by > this change. > > Has this test now been tested on all platforms with both the OpenJDK and > OracleJDK? > > Thanks, > David > > > Best regards, > > Goetz. > > From weijun.wang at oracle.com Tue Jan 23 09:19:06 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 23 Jan 2018 17:19:06 +0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> Message-ID: <3BE43426-8CA8-4B1F-B442-229083AFAF25@oracle.com> + * Therefore, the method may be safely called with very large values of + * {@code len} provided sufficient memory is available. What does "sufficient memory" mean? For len, or the number of available bytes? --Max > On Jan 23, 2018, at 4:49 AM, Brian Burkhalter wrote: > > All of the comments included below are addressed in [1]. The difference versus webrev.01 are in [2]. The CSR [3] will have to move back to Draft, updated, and re-finalized but I will hold off on that until there is a final consensus on the verbiage. > > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8139206/webrev.02/ > [2] http://cr.openjdk.java.net/~bpb/8139206/webrev.01-02/ > [3] https://bugs.openjdk.java.net/browse/JDK-8194956 > > On Jan 19, 2018, at 11:49 AM, Roger Riggs wrote: > >> A pre-existing typo: >> line 67 "{@code skip()}" should be "{@code skip(*long*)}". >> >> Since the public readNBytes suffices for readAllBytes, I would rename the private readAtMostNBytes >> to readNBytes and avoid the duplication of javadoc. >> >> Keeping the existing readAllBytes before readNBytes in the source file will make the diff easier to follow >> and the methods be in alphabetical order. > > > On Jan 22, 2018, at 12:44 AM, Peter Levart wrote: > >> The delegation to public method (readAllBytes -> readNBytes) should then be documented so that subclasses know that overriding readNBytes, if needed, is sufficient. > > > On Jan 22, 2018, at 12:52 AM, Alan Bateman wrote: > >> The updated version looks good. I just wonde about the "For example ..." in the @throws OOME description. The API can't be called with a len > Integer.MAX_VALUE so this example could be confusing - I think just drop that sentence. >> >> Minor formatting in passing. At L128 and L339 it would be easier to read if the "throws IOException" were on the previous line. Also L355 might be a bit clear if the Math.min was indented (have to look twice to see that it's not in the while body). > > > On Jan 22, 2018, at 7:56 AM, Adam Petcher wrote: > >> Possible wording, if this method can be called with large length values: >> >> "The total amount of memory allocated by this method is proportional to the number of bytes read from the stream. Therefore, the method may be safely called with very large values of {@code len}. > > From xu.y.yin at oracle.com Tue Jan 23 10:14:37 2018 From: xu.y.yin at oracle.com (Chris Yin) Date: Tue, 23 Jan 2018 18:14:37 +0800 Subject: RFR: 8195976: Add JNDI test javax/naming/dns/AttributeTests/GetAny.java In-Reply-To: <441789eb-a0ed-5eee-1d74-af5a888e9e82@oracle.com> References: <9CAEA585-EB5C-4F55-88D3-124E589778EB@oracle.com> <441789eb-a0ed-5eee-1d74-af5a888e9e82@oracle.com> Message-ID: Thank you Alan, I just moved it to com/sun/jndi/dns/ as you suggested and removed unused "@modules jdk.naming.dns/com.sun.jndi.dns?, updated webrev as below, thanks http://cr.openjdk.java.net/~xiaofeya/8195976/webrev.00/ Regards, Chris > On 23 Jan 2018, at 3:53 PM, Alan Bateman wrote: > > > > On 23/01/2018 07:01, Chris Yin wrote: >> Please review the added JNDI test javax/naming/dns/AttributeTests/GetAny.java, thanks >> >> > You may want to move it to com/sun/jndi/dns so that it's with the other tests for the DNS provider (as there is no javax.naming.dns API). Also I suspect you don't need "@modules jdk.naming.dns/com.sun.jndi.dns" as it doesn't appear to make direct use of the classes in the implementation. > > -Alan From daniel.fuchs at oracle.com Tue Jan 23 10:27:34 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 23 Jan 2018 10:27:34 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: <5bbc2dde-ff8e-c971-b88a-b99cde160aab@oracle.com> Hi Jason, On 22/01/2018 21:30, Jason Mehrens wrote: > Daniel, > > Fantastic! The patch looks good as is. The only > thing that sticks out is that both the 'if' and > the 'else if' have the same code inside them. > That could be refactored a bit. I had the same feeling - but then when looking at possible refactoring I came to the conclusion that I liked the explicit if / else if / better. The duplicated code is just one line + one return so it's not that bad. best regards, -- daniel > > Jason From daniel.fuchs at oracle.com Tue Jan 23 10:49:23 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 23 Jan 2018 10:49:23 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: Hi Mandy, Thanks for looking into this. On 22/01/2018 23:52, mandy chung wrote: > So the cause of JDK-8191033 is because cx::addLocalLogger returns false > when addLogger is called the first time to add the root and global > logger and it causes that ".handlers" is not read and loaded for the > root logger. Yes. "global" suffers the same fate. > It took me some time to understand what it's happening. > When addLocalLogger is called to add a logger, it calls > ensureAllDefaultLoggers to ensure the root and global loggers are added > first.? Do you think the handling of ".handlers" should be done in > LoggerContext::ensureDefaultLogger?? Would that avoid the check to call > loadLoggerHandlers if it's called from ensureLogManagerInitialized? > The code is getting quite complicated and if moving it closer to the > place it adds the root logger would help. I did give it some consideration. I think this would be a mistake. The whole point of this fix is that loadLoggerHandlers *must* be called *within* addLogger - so that a subclass of LogManager that overrides addLogger can take over. The fact that loadLoggerHandlers was called outside of addLogger was what caused the issue with ClassLoaderLogMaanger. In 10 we see that the root logger is added to the context outside of the call to addLogger, so doing what you suggest would - I think - trigger the issue again. In addition the lazy code that ensure that the root and global loggers are added first to the contexts is complex enough - I would be very reluctant to fiddle too much with it. The condition in the / else if / can be true only once when addLogger is called after reading the primordial configuration - so I believe the fix is sound. The other possibility would be to manage a state for the creation of handlers (from name + ".handlers") for both root & global, but I believe this would be more complicated. best regards, -- daniel > On 1/22/18 8:47 AM, Daniel Fuchs wrote: >> >> http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.01/ From Roger.Riggs at Oracle.com Tue Jan 23 14:54:13 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 23 Jan 2018 09:54:13 -0500 Subject: Microsecond support in java.time.Duration/Instant? In-Reply-To: References: Message-ID: Hi Kurt, I created an enhancement request in the Jira and linked the core-libs emails in. ??? https://bugs.openjdk.java.net/browse/JDK-8196003 Thanks for the frequency usage info.? Its hard to guess whether if micro APIs were available whether they would have been used instead of millis. Roger On 1/23/2018 12:23 AM, Kurt Alfred Kluever wrote: > Thanks for the responses, Stephen + Roger,. > > As noted, a line definitely has to be drawn somewhere. In case anyone > is looking for some data, here are current relative usage stats inside > of Google for the various APIs, grouped by functionality > (creating/decomposing Instants/Durations): > > ? Instant.ofEpochMilli(long): 67% > ? Instant.ofEpochSecond(long): 29% > *? Instants.ofEpochMicros(long): 4% > * > > ? Instant.toEpochMilli(): 83% > ? Instant.getEpochSecond(): 10% > *? Instants.toEpochMicros(Instant): 7%* > > ? Duration.ofSeconds(long): 30% > Duration.ofDays(long): 24% > ? Duration.ofMillis(long): 21% > Duration.ofMinutes(long): 18% > Duration.ofHours(long): 7% > ? Duration.ofNanos(long): < 1% > *? Durations.ofMicros(long): < 1%* > > ? Duration.toMillis(): 73% > Duration.getSeconds(): 16% > ? Duration.toMinutes(): 3% > ? Duration.toNanos(): 3% > Duration.toDays(): 3% > *Durations.toMicros(Duration): 2%* > Duration.toHours(): 1% > > So yea, it's definitely towards the bottom of the usage stats, but > that also might be partially because of the discoverability issue > (people are much more likely to find an instance method directly on > the type than a static method on our Durations class). Anyway, I'm not > claiming these numbers motivate any sort of change, but given a > proposal to add microsecond support directly to the APIs, I think I'd > be in favor :-) Or perhaps Google is unique in it's usage of > microsecond precision (many of our storage systems have timestamps > using microsecond precision). > > PS - and thanks for the ".NET ticks" reference, I hadn't heard of that > before. And maybe here's a new one for you that just popped up in the > news --- a Flick ?(1/705600000 of > a second). > > On Mon, Jan 22, 2018 at 10:00 AM, Stephen Colebourne > > wrote: > > On 22 January 2018 at 02:58, Kurt Alfred Kluever > wrote: > > I'm curious how these sets of units were chosen or decided upon? I > > understand that the line must be drawn somewhere (or else > someone may come > > along asking for centisecond support), but I'm curious as to the > rational. > > Nanos have to be supported as they are the smallest available. > Millis are supported as they are the historic form. > > Micros is only one of the other possible ones - .NET ticks might be > another. A line has to be drawn somewhere... > > Stephen > > > > > -- > kak From Roger.Riggs at Oracle.com Tue Jan 23 14:57:40 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 23 Jan 2018 09:57:40 -0500 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> <46a67401-92e4-09a2-dfdf-2c10a977a71e@oracle.com> Message-ID: <96b475f5-dbfc-27ef-eaea-c3ebc9a6a814@Oracle.com> Looks fine. Regards, Roger On 1/23/2018 1:15 AM, Rachna Goel wrote: > Hi, > > Kindly review updated patch for this doc fix: > > patch: http://cr.openjdk.java.net/%7Ergoel/8146656/webrev.02/ > > Approved CSR : https://bugs.openjdk.java.net/browse/JDK-8191414 > > Thanks, > > Rachna > > > On 20/12/17 10:44 PM, joe darcy wrote: >> >> Hi Rachna, >> >> I think the revised version with the @implSpec tag switch is >> acceptable, but also think providing more text to describe this >> situation would be helpful to programmers unaware of a 13 month >> possibility. >> >> Cheers, >> >> -Joe >> >> >> On 12/19/2017 2:08 AM, Rachna Goel wrote: >>> >>> Hi Joe, >>> >>> Thanks for the comments. >>> >>> I have updated the CSR to have @implSpec in place of @implNote. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8191414 >>> >>> Regarding? "An array with either 12 or 13 elements will be returned >>> depending on whether or {@link Calendar.UNDECIMBER} is supported." >>> ,? I would like to go with existing statement as this method always >>> returns 13 elements where the 13th element may be empty string or >>> may contain Calendar.UNDECIMBER, depending upon whether its >>> supported by the Calendar instance. >>> >>> kindly suggest whether this looks fine! >>> >>> Thanks, >>> Rachna >>> >>> >>> On 19/12/17 2:55 PM, joe darcy wrote: >>>> Hi Rachna, >>>> >>>> On 12/19/2017 1:13 AM, Rachna Goel wrote: >>>>> >>>>> Hello Joe, >>>>> >>>>> Thanks for the review. >>>>> >>>>> Reason I added @implNote is that it's the case for the default >>>>> implementation. Not added as a part of spec, as some >>>>> implementation can just return 12 element array for same methods >>>>> through the "java.text.spi.DateFormatSymbolsProvider" SPI. >>>>> >>>>> >>>> >>>> That is precisely the sort of situation the @implSpec tag is >>>> intended for. It allows the specification to say DateFormatSymbols >>>> must behave this way while allowing subclasses to behave differently. >>>> >>>> Perhaps some general text can be added as normal specification, >>>> something like >>>> >>>> "An array with either 12 or 13 elements will be returned depending >>>> on whether or {@link Calendar.UNDECIMBER} is supported." >>>> >>>> paired with >>>> >>>> @implSpec This method returns 13 elements since @link >>>> Calendar.UNDECIMBER} is supported. >>>> >>>> HTH, >>>> >>>> -Joe >>>> >>> >>> -- >>> Thanks, >>> Rachna >> > From kak at google.com Tue Jan 23 15:05:12 2018 From: kak at google.com (Kurt Alfred Kluever) Date: Tue, 23 Jan 2018 10:05:12 -0500 Subject: Microsecond support in java.time.Duration/Instant? In-Reply-To: References: Message-ID: Excellent, thanks Roger - I'll follow along there. On Tue, Jan 23, 2018 at 9:54 AM, Roger Riggs wrote: > Hi Kurt, > > I created an enhancement request in the Jira and linked the core-libs > emails in. > https://bugs.openjdk.java.net/browse/JDK-8196003 > > Thanks for the frequency usage info. Its hard to guess whether if micro > APIs were > available whether they would have been used instead of millis. > > Roger > > > > On 1/23/2018 12:23 AM, Kurt Alfred Kluever wrote: > > Thanks for the responses, Stephen + Roger,. > > As noted, a line definitely has to be drawn somewhere. In case anyone is > looking for some data, here are current relative usage stats inside of > Google for the various APIs, grouped by functionality (creating/decomposing > Instants/Durations): > > Instant.ofEpochMilli(long): 67% > Instant.ofEpochSecond(long): 29% > > * Instants.ofEpochMicros(long): 4% * > > Instant.toEpochMilli(): 83% > Instant.getEpochSecond(): 10% > * Instants.toEpochMicros(Instant): 7%* > > Duration.ofSeconds(long): 30% > Duration.ofDays(long): 24% > Duration.ofMillis(long): 21% > Duration.ofMinutes(long): 18% > Duration.ofHours(long): 7% > Duration.ofNanos(long): < 1% > * Durations.ofMicros(long): < 1%* > > Duration.toMillis(): 73% > Duration.getSeconds(): 16% > Duration.toMinutes(): 3% > Duration.toNanos(): 3% > Duration.toDays(): 3% > * Durations.toMicros(Duration): 2%* > Duration.toHours(): 1% > > So yea, it's definitely towards the bottom of the usage stats, but that > also might be partially because of the discoverability issue (people are > much more likely to find an instance method directly on the type than a > static method on our Durations class). Anyway, I'm not claiming these > numbers motivate any sort of change, but given a proposal to add > microsecond support directly to the APIs, I think I'd be in favor :-) Or > perhaps Google is unique in it's usage of microsecond precision (many of > our storage systems have timestamps using microsecond precision). > > PS - and thanks for the ".NET ticks" reference, I hadn't heard of that > before. And maybe here's a new one for you that just popped up in the news > --- a Flick (1/705600000 of a > second). > > On Mon, Jan 22, 2018 at 10:00 AM, Stephen Colebourne > wrote: > >> On 22 January 2018 at 02:58, Kurt Alfred Kluever wrote: >> > I'm curious how these sets of units were chosen or decided upon? I >> > understand that the line must be drawn somewhere (or else someone may >> come >> > along asking for centisecond support), but I'm curious as to the >> rational. >> >> Nanos have to be supported as they are the smallest available. >> Millis are supported as they are the historic form. >> >> Micros is only one of the other possible ones - .NET ticks might be >> another. A line has to be drawn somewhere... >> >> Stephen >> > > > > -- > kak > > > -- kak From mandy.chung at oracle.com Tue Jan 23 15:44:18 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 23 Jan 2018 07:44:18 -0800 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> Message-ID: <91404607-d569-10c3-138c-8ba5124810ac@oracle.com> On 1/23/18 2:49 AM, Daniel Fuchs wrote: > I did give it some consideration. I think this would be a mistake. > > The whole point of this fix is that loadLoggerHandlers *must* be > called *within* addLogger - so that a subclass of LogManager that > overrides addLogger can take over. Isn't addLocalLogger called only when LogManager does not override addLogger? Mandy From daniel.fuchs at oracle.com Tue Jan 23 15:54:40 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 23 Jan 2018 15:54:40 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <91404607-d569-10c3-138c-8ba5124810ac@oracle.com> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> <91404607-d569-10c3-138c-8ba5124810ac@oracle.com> Message-ID: <84ea0b79-6ed9-a6b3-2318-77dbdc3040c9@oracle.com> On 23/01/2018 15:44, mandy chung wrote: > Isn't addLocalLogger called only when LogManager does not override > addLogger? It's also called by ensureDefaultLogger(). -- daniel > > Mandy From kumar.x.srinivasan at oracle.com Tue Jan 23 17:35:32 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Tue, 23 Jan 2018 09:35:32 -0800 Subject: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError In-Reply-To: <98a73ac560274baca08c8f9adc31ee31@sap.com> References: <595de57b-50cb-ea86-0faa-ab393b672036@oracle.com> <98a73ac560274baca08c8f9adc31ee31@sap.com> Message-ID: <5A677264.80108@oracle.com> The last time around, I was missing the deploy repo and thus the build/test system did not take deploy into account. My error. I will retest and report. Kumar > Hi, > > I can not do proper tests with OracleJDK, sorry for that. > > This webrev removes the problem from the ProblemList: > http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev.02/ > > Actually, you should have an internal ProblemList for things > that fail with the OracleJDK. We do so for SAP JVM and our OpenJDK > builds, too. > > Best regards, > Goetz. > >> -----Original Message----- >> From: David Holmes [mailto:david.holmes at oracle.com] >> Sent: Dienstag, 23. Januar 2018 08:50 >> To: Lindenmaier, Goetz ; core-libs-dev > libs-dev at openjdk.java.net> >> Subject: Re: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java fails with >> java.lang.AssertionError >> >> Hi Goetz, >> >> On 23/01/2018 5:36 PM, Lindenmaier, Goetz wrote: >>> Hi, >>> >>> javacpl seems not to have a help message, so it just needs to >>> be excluded from the test. Please review. >>> http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev/ >> That seems okay. Did you test it on Windows? Also ... >> >>> (I sent similar mail in reply to RE: JDK 11 RFR of JDK-8195987,,Problem list >> tools/launcher/HelpFlagsTest.java fails on windows) >> >> ... the ProblemList change was pushed so will have to be reverted by >> this change. >> >> Has this test now been tested on all platforms with both the OpenJDK and >> OracleJDK? >> >> Thanks, >> David >> >>> Best regards, >>> Goetz. >>> From naoto.sato at oracle.com Tue Jan 23 18:12:11 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Tue, 23 Jan 2018 10:12:11 -0800 Subject: [11] RFR of 8146656: Wrong Months Array for DateFormatSymbols In-Reply-To: References: <428d88c5-8e79-bae6-f9e0-d2b8fa9c1d46@oracle.com> <286bbd77-08ed-bf4c-0ebe-bf7d383e0a90@oracle.com> <719413d5-89e4-7ecd-3420-8a2a6ff63910@oracle.com> <46a67401-92e4-09a2-dfdf-2c10a977a71e@oracle.com> Message-ID: +1 Naoto On 1/22/18 10:15 PM, Rachna Goel wrote: > Hi, > > Kindly review updated patch for this doc fix: > > patch: http://cr.openjdk.java.net/%7Ergoel/8146656/webrev.02/ > > Approved CSR : https://bugs.openjdk.java.net/browse/JDK-8191414 > > Thanks, > > Rachna > > > On 20/12/17 10:44 PM, joe darcy wrote: >> >> Hi Rachna, >> >> I think the revised version with the @implSpec tag switch is >> acceptable, but also think providing more text to describe this >> situation would be helpful to programmers unaware of a 13 month >> possibility. >> >> Cheers, >> >> -Joe >> >> >> On 12/19/2017 2:08 AM, Rachna Goel wrote: >>> >>> Hi Joe, >>> >>> Thanks for the comments. >>> >>> I have updated the CSR to have @implSpec in place of @implNote. >>> >>> https://bugs.openjdk.java.net/browse/JDK-8191414 >>> >>> Regarding? "An array with either 12 or 13 elements will be returned >>> depending on whether or {@link Calendar.UNDECIMBER} is supported." , >>> I would like to go with existing statement as this method always >>> returns 13 elements where the 13th element may be empty string or may >>> contain Calendar.UNDECIMBER, depending upon whether its supported by >>> the Calendar instance. >>> >>> kindly suggest whether this looks fine! >>> >>> Thanks, >>> Rachna >>> >>> >>> On 19/12/17 2:55 PM, joe darcy wrote: >>>> Hi Rachna, >>>> >>>> On 12/19/2017 1:13 AM, Rachna Goel wrote: >>>>> >>>>> Hello Joe, >>>>> >>>>> Thanks for the review. >>>>> >>>>> Reason I added @implNote is that it's the case for the default >>>>> implementation. Not added as a part of spec, as some implementation >>>>> can just return 12 element array for same methods through the >>>>> "java.text.spi.DateFormatSymbolsProvider" SPI. >>>>> >>>>> >>>> >>>> That is precisely the sort of situation the @implSpec tag is >>>> intended for. It allows the specification to say DateFormatSymbols >>>> must behave this way while allowing subclasses to behave differently. >>>> >>>> Perhaps some general text can be added as normal specification, >>>> something like >>>> >>>> "An array with either 12 or 13 elements will be returned depending >>>> on whether or {@link Calendar.UNDECIMBER} is supported." >>>> >>>> paired with >>>> >>>> @implSpec This method returns 13 elements since @link >>>> Calendar.UNDECIMBER} is supported. >>>> >>>> HTH, >>>> >>>> -Joe >>>> >>> >>> -- >>> Thanks, >>> Rachna >> > > -- > Thanks, > Rachna > From daniel.fuchs at oracle.com Tue Jan 23 19:13:17 2018 From: daniel.fuchs at oracle.com (Daniel Fuchs) Date: Tue, 23 Jan 2018 19:13:17 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: <5bbc2dde-ff8e-c971-b88a-b99cde160aab@oracle.com> References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> <5bbc2dde-ff8e-c971-b88a-b99cde160aab@oracle.com> Message-ID: Hi Jason, I take it back :-) Mandy helped me finding better names to refactor the code, so here is a (hopefully) better version: http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.02/ best regards, -- daniel On 23/01/2018 10:27, Daniel Fuchs wrote: > Hi Jason, > > On 22/01/2018 21:30, Jason Mehrens wrote: >> Daniel, >> >> Fantastic!? The patch looks good as is.? The only > thing that sticks >> out is that both the 'if' and >> the 'else if' have the same code inside them. >> That could be refactored a bit. > > I had the same feeling - but then when looking at > possible refactoring I came to the conclusion that > I liked the explicit if / else if / better. > > The duplicated code is just one line + one return > so it's not that bad. > > best regards, > > -- daniel > >> >> Jason > From paul.sandoz at oracle.com Tue Jan 23 19:12:58 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 23 Jan 2018 11:12:58 -0800 Subject: [10] RFR 8195970 Crash passing null to a VarHandle Message-ID: <525D78DD-9D1C-4009-901E-63F975AFDB7B@oracle.com> Hi, Please review: http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8195970-varhandle-buffer-view-null/webrev/ The ByteBuffer view VarHandle implements were (embarrassingly) missing a null check on the ByteBuffer argument. Checks were added and so were tests. I am proposing this be fixed in 10 via the fix request process [1]. Thanks, Paul. [1] http://openjdk.java.net/projects/jdk/10/fix-request-process From mandy.chung at oracle.com Tue Jan 23 19:23:26 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 23 Jan 2018 11:23:26 -0800 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> <5bbc2dde-ff8e-c971-b88a-b99cde160aab@oracle.com> Message-ID: On 1/23/18 11:13 AM, Daniel Fuchs wrote: > Hi Jason, > > I take it back :-) > > Mandy helped me finding better names to refactor the code, so > here is a (hopefully) better version: > > http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.02/ I like this better.?? Thank you for the update. 1220 assert logger != null; This is not necessary since addLogger checks for null logger. No need for a new webrev. Thanks Mandy From mandy.chung at oracle.com Tue Jan 23 19:27:03 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 23 Jan 2018 11:27:03 -0800 Subject: [10] RFR 8195970 Crash passing null to a VarHandle In-Reply-To: <525D78DD-9D1C-4009-901E-63F975AFDB7B@oracle.com> References: <525D78DD-9D1C-4009-901E-63F975AFDB7B@oracle.com> Message-ID: <1a4f8d5c-e8bd-ac90-4aeb-738f92a43f82@oracle.com> Looks good to me.? This is very low risk and no objection from me to fix in 10. Mandy On 1/23/18 11:12 AM, Paul Sandoz wrote: > Hi, > > Please review: > > http://cr.openjdk.java.net/~psandoz/jdk10/JDK-8195970-varhandle-buffer-view-null/webrev/ > > The ByteBuffer view VarHandle implements were (embarrassingly) missing a null check on the ByteBuffer argument. Checks were added and so were tests. > > I am proposing this be fixed in 10 via the fix request process [1]. > > Thanks, > Paul. > > [1] http://openjdk.java.net/projects/jdk/10/fix-request-process From kumar.x.srinivasan at oracle.com Tue Jan 23 20:34:05 2018 From: kumar.x.srinivasan at oracle.com (Kumar Srinivasan) Date: Tue, 23 Jan 2018 12:34:05 -0800 Subject: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java fails with java.lang.AssertionError In-Reply-To: <5A677264.80108@oracle.com> References: <595de57b-50cb-ea86-0faa-ab393b672036@oracle.com> <98a73ac560274baca08c8f9adc31ee31@sap.com> <5A677264.80108@oracle.com> Message-ID: <5A679C3D.8030408@oracle.com> Hi, I got the test results and I also manually tested on Windows, everything looks good, you can go ahead and push the changes. Thanks Kumar > The last time around, I was missing the deploy repo and thus the > build/test system did not take deploy into account. My error. > > I will retest and report. > > Kumar > > >> Hi, >> >> I can not do proper tests with OracleJDK, sorry for that. >> >> This webrev removes the problem from the ProblemList: >> http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev.02/ >> >> Actually, you should have an internal ProblemList for things >> that fail with the OracleJDK. We do so for SAP JVM and our OpenJDK >> builds, too. >> >> Best regards, >> Goetz. >> >>> -----Original Message----- >>> From: David Holmes [mailto:david.holmes at oracle.com] >>> Sent: Dienstag, 23. Januar 2018 08:50 >>> To: Lindenmaier, Goetz ; core-libs-dev >>> >> libs-dev at openjdk.java.net> >>> Subject: Re: RFR(XS): 8195824: tools/launcher/HelpFlagsTest.java >>> fails with >>> java.lang.AssertionError >>> >>> Hi Goetz, >>> >>> On 23/01/2018 5:36 PM, Lindenmaier, Goetz wrote: >>>> Hi, >>>> >>>> javacpl seems not to have a help message, so it just needs to >>>> be excluded from the test. Please review. >>>> http://cr.openjdk.java.net/~goetz/wr18/8195824-fixHelpTest2/webrev/ >>> That seems okay. Did you test it on Windows? Also ... >>> >>>> (I sent similar mail in reply to RE: JDK 11 RFR of >>>> JDK-8195987,,Problem list >>> tools/launcher/HelpFlagsTest.java fails on windows) >>> >>> ... the ProblemList change was pushed so will have to be reverted by >>> this change. >>> >>> Has this test now been tested on all platforms with both the OpenJDK >>> and >>> OracleJDK? >>> >>> Thanks, >>> David >>> >>>> Best regards, >>>> Goetz. >>>> > From jason_mehrens at hotmail.com Tue Jan 23 21:46:36 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Tue, 23 Jan 2018 21:46:36 +0000 Subject: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat In-Reply-To: References: <7712f5ce-7d87-985f-3544-939377507ad7@oracle.com> <04aad6af-c91a-a2a0-1b42-e6202cda4b85@apache.org> <66473c59-f4ce-8ecd-e229-b1c86c2db3b0@oracle.com> <1be87b97-4627-0b7f-50b8-c73a40b965ab@oracle.com> <5bbc2dde-ff8e-c971-b88a-b99cde160aab@oracle.com>, Message-ID: Looks great! Personally, these LogManager startup issues make my head hurt. :) Jason ________________________________________ From: Daniel Fuchs Sent: Tuesday, January 23, 2018 1:13 PM To: Jason Mehrens; core-libs-dev at openjdk.java.net Cc: Mandy Chung Subject: Re: [JDK 11] RFR: 8195096: Exception printed on console with custom LogManager on starting Apache Tomcat Hi Jason, I take it back :-) Mandy helped me finding better names to refactor the code, so here is a (hopefully) better version: http://cr.openjdk.java.net/~dfuchs/webrev_8195096/webrev.02/ best regards, -- daniel On 23/01/2018 10:27, Daniel Fuchs wrote: > Hi Jason, > > On 22/01/2018 21:30, Jason Mehrens wrote: >> Daniel, >> >> Fantastic! The patch looks good as is. The only > thing that sticks >> out is that both the 'if' and >> the 'else if' have the same code inside them. >> That could be refactored a bit. > > I had the same feeling - but then when looking at > possible refactoring I came to the conclusion that > I liked the explicit if / else if / better. > > The duplicated code is just one line + one return > so it's not that bad. > > best regards, > > -- daniel > >> >> Jason > From andrey.x.nazarov at oracle.com Tue Jan 23 22:28:24 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Tue, 23 Jan 2018 14:28:24 -0800 Subject: RFR: JDK-8161348 Several tools/jlink tests failed due to time out In-Reply-To: References: <5A665291.4020706@oracle.com> <42B07061-0508-459E-85F6-FCCAD99B6E44@oracle.com> Message-ID: Hi, After offline discussion with Leonid and with Igor in JBS we decided to skip test in case of -Xcomp option presence and do not increase timeout. Issue was faced in our build and test system only with -Xcomp flag. see updated patch http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.02/ ?Andrei > On 22 Jan 2018, at 16:12, Andrey Nazarov wrote: > > > >> On 22 Jan 2018, at 15:33, Andrey Nazarov wrote: >> >> >> >>> On 22 Jan 2018, at 13:07, Joseph D. Darcy wrote: >>> >>> Hello, >>> >>> I'm wary of increasing the timeout to 5 minutes. When such tests are run on CI servers, the effective timeout can be 10 minutes or more given a timeout factor used for the test run. >> We can move them to later tiers. On my Macbook pro 2015 + SSD JlinkTest.java runs around 40 seconds. It could easily be more than 2 min with HDD, working McAfee/indexer, slower hardware. So I think default timeout = 2 minutes is not enough. >>> >>> Should using 1GB of memory on the @run line have an earlier @requires guard for an amount of memory on the system? >> I think it should. Although it?s not related to the original issue, i?ll update patch with @requires > Updated http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.01/ >>> >>> Cheers, >>> >>> -Joe >>> >>> On 1/22/2018 12:58 PM, Andrey Nazarov wrote: >>>> Hi, >>>> please review Jlink tests. >>>> >>>> I?ve increased default timeout to 5 minutes and skip tests in ?-Xcomp? VM mode. >>>> >>>> CR: http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.00/ >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8161348 >>>> >>>> ?Andrei >>> >> > From mandy.chung at oracle.com Tue Jan 23 22:40:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 23 Jan 2018 14:40:22 -0800 Subject: [11] RFR JDK-8191170: Clarify if java.class.path can be undefined Message-ID: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8191170/webrev.00/index.html A small adjustment in the javadoc.? It clarifies what an empty value of "java.class.path" property is interpreted by the built-in application class loader. Mandy From eliasen at mindspring.com Tue Jan 23 22:43:28 2018 From: eliasen at mindspring.com (Alan Eliasen) Date: Tue, 23 Jan 2018 15:43:28 -0700 Subject: Microsecond support in java.time.Duration/Instant? In-Reply-To: References: Message-ID: <41BBEED3-7DE1-44BF-BC90-FFFFC46631D9@mindspring.com> I'm not sure I want to bring this into the discussion, but there's a proposal (pushed largely by Facebook) for a timescale called the "flick" which is exactly 1/705600000 second. Quote: "This unit of time is the smallest time unit which is LARGER than a nanosecond, and can in integer quantities exactly represent a single frame duration for 24hz, 25hz, 30hz, 48hz, 50hz, 60hz, 90hz, 100hz, 120hz, and also 1/1000 divisions of each. This makes it suitable for use via std::chrono::duration and std::ratio for doing timing work against the system high resolution clock, which is in nanoseconds, but doesn't get slightly out of sync when doing common frame rates." I can see useful properties of this when doing multimedia playback. They have C++ libraries implementing this unit. I'm not affiliated with this effort in any way, but saw it recently. https://github.com/OculusVR/Flicks/blob/master/README.md On January 23, 2018 7:54:13 AM MST, Roger Riggs wrote: >Hi Kurt, > >I created an enhancement request in the Jira and linked the core-libs >emails in. > ??? https://bugs.openjdk.java.net/browse/JDK-8196003 > >Thanks for the frequency usage info.? Its hard to guess whether if >micro >APIs were >available whether they would have been used instead of millis. > >Roger > > >On 1/23/2018 12:23 AM, Kurt Alfred Kluever wrote: >> Thanks for the responses, Stephen + Roger,. >> >> As noted, a line definitely has to be drawn somewhere. In case anyone > >> is looking for some data, here are current relative usage stats >inside >> of Google for the various APIs, grouped by functionality >> (creating/decomposing Instants/Durations): >> >> ? Instant.ofEpochMilli(long): 67% >> ? Instant.ofEpochSecond(long): 29% >> *? Instants.ofEpochMicros(long): 4% >> * >> >> ? Instant.toEpochMilli(): 83% >> ? Instant.getEpochSecond(): 10% >> *? Instants.toEpochMicros(Instant): 7%* >> >> ? Duration.ofSeconds(long): 30% >> Duration.ofDays(long): 24% >> ? Duration.ofMillis(long): 21% >> Duration.ofMinutes(long): 18% >> Duration.ofHours(long): 7% >> ? Duration.ofNanos(long): < 1% >> *? Durations.ofMicros(long): < 1%* >> >> ? Duration.toMillis(): 73% >> Duration.getSeconds(): 16% >> ? Duration.toMinutes(): 3% >> ? Duration.toNanos(): 3% >> Duration.toDays(): 3% >> *Durations.toMicros(Duration): 2%* >> Duration.toHours(): 1% >> >> So yea, it's definitely towards the bottom of the usage stats, but >> that also might be partially because of the discoverability issue >> (people are much more likely to find an instance method directly on >> the type than a static method on our Durations class). Anyway, I'm >not >> claiming these numbers motivate any sort of change, but given a >> proposal to add microsecond support directly to the APIs, I think I'd > >> be in favor :-) Or perhaps Google is unique in it's usage of >> microsecond precision (many of our storage systems have timestamps >> using microsecond precision). >> >> PS - and thanks for the ".NET ticks" reference, I hadn't heard of >that >> before. And maybe here's a new one for you that just popped up in the > >> news --- a Flick ?(1/705600000 of > >> a second). >> >> On Mon, Jan 22, 2018 at 10:00 AM, Stephen Colebourne >> > wrote: >> >> On 22 January 2018 at 02:58, Kurt Alfred Kluever > > wrote: >> > I'm curious how these sets of units were chosen or decided >upon? I >> > understand that the line must be drawn somewhere (or else >> someone may come >> > along asking for centisecond support), but I'm curious as to >the >> rational. >> >> Nanos have to be supported as they are the smallest available. >> Millis are supported as they are the historic form. >> >> Micros is only one of the other possible ones - .NET ticks might >be >> another. A line has to be drawn somewhere... >> >> Stephen >> >> >> >> >> -- >> kak -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From jkalina at redhat.com Tue Jan 23 22:58:56 2018 From: jkalina at redhat.com (Jan Kalina) Date: Tue, 23 Jan 2018 23:58:56 +0100 Subject: Patch fixing JDK-8176553 Message-ID: Hi, I has prepared trivial patch for bug JDK-8176553, which I has reported. I will welcome if it could be merged into JDK. (The bug is present in JDK9 and JDK8 too.) I am covered by Red Hat OCA. The patch is attached, bug reproducer is already in JIRA: https://bugs.openjdk.java.net/browse/JDK-8176553 Thanks, Jan Kalina -------------- next part -------------- A non-text attachment was scrubbed... Name: jkpatch-ldap-referral-loop.patch Type: text/x-patch Size: 763 bytes Desc: not available URL: From paul.sandoz at oracle.com Wed Jan 24 00:12:52 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 23 Jan 2018 16:12:52 -0800 Subject: [11] RFR JDK-8191170: Clarify if java.class.path can be undefined In-Reply-To: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> References: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> Message-ID: Hi Mandy, What is the ?initial module?? Thanks, Paul. > On Jan 23, 2018, at 2:40 PM, mandy chung wrote: > > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8191170/webrev.00/index.html > > A small adjustment in the javadoc. It clarifies what an empty value of "java.class.path" property is interpreted by the built-in application class loader. > > Mandy From mandy.chung at oracle.com Wed Jan 24 00:28:26 2018 From: mandy.chung at oracle.com (mandy chung) Date: Tue, 23 Jan 2018 16:28:26 -0800 Subject: [11] RFR JDK-8191170: Clarify if java.class.path can be undefined In-Reply-To: References: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> Message-ID: <9b244491-e75e-2311-c314-08d279bd6712@oracle.com> It is the module where the main class is per the value of the "jdk.module.main" system property.? If not set, the initial module is an unnamed module.? What about adding the following to @implNote? ?? ? * The "{@code jdk.module.main}" system property specifies the name ???? * of the initial module if it is a named module.? If not defined, ???? * the initial module is an unnamed module. Mandy On 1/23/18 4:12 PM, Paul Sandoz wrote: > Hi Mandy, > > What is the ?initial module?? > > Thanks, > Paul. > >> On Jan 23, 2018, at 2:40 PM, mandy chung wrote: >> >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8191170/webrev.00/index.html >> >> A small adjustment in the javadoc. It clarifies what an empty value of "java.class.path" property is interpreted by the built-in application class loader. >> >> Mandy From brian.burkhalter at oracle.com Wed Jan 24 00:50:31 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Tue, 23 Jan 2018 16:50:31 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <3BE43426-8CA8-4B1F-B442-229083AFAF25@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <3BE43426-8CA8-4B1F-B442-229083AFAF25@oracle.com> Message-ID: <49B674D6-3736-419F-AD19-E4F3AE12660A@oracle.com> On Jan 23, 2018, at 1:19 AM, Weijun Wang wrote: > + * Therefore, the method may be safely called with very large values of > + * {@code len} provided sufficient memory is available. > > What does "sufficient memory" mean? For len, or the number of available bytes? It means enough bytes for the collectivity of the intermediate and returned buffers. This is already stated to be proportional to ?len?. Brian From Alan.Bateman at oracle.com Wed Jan 24 08:40:36 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 24 Jan 2018 08:40:36 +0000 Subject: RFR: JDK-8161348 Several tools/jlink tests failed due to time out In-Reply-To: References: <5A665291.4020706@oracle.com> <42B07061-0508-459E-85F6-FCCAD99B6E44@oracle.com> Message-ID: On 23/01/2018 22:28, Andrey Nazarov wrote: > Hi, > > After offline discussion with Leonid and with Igor in JBS we decided to skip test in case of -Xcomp option presence and do not increase timeout. Issue was faced in our build and test system only with -Xcomp flag. > > see updated patch http://cr.openjdk.java.net/~anazarov/JDK-8161348/webrev.02/ > This looks okay. On the >= 2g check then we should separately see if these tests can run without needing 1g of heap. -Alan From Alan.Bateman at oracle.com Wed Jan 24 11:36:55 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 24 Jan 2018 11:36:55 +0000 Subject: [11] RFR JDK-8191170: Clarify if java.class.path can be undefined In-Reply-To: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> References: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> Message-ID: <449196ba-cf7a-1d78-d573-ce46947b9b7f@oracle.com> On 23/01/2018 22:40, mandy chung wrote: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8191170/webrev.00/index.html > > > A small adjustment in the javadoc.? It clarifies what an empty value > of "java.class.path" property is interpreted by the built-in > application class loader. It looks okay but I had to read it a few times. A few suggestions that might make it a bit easier to read: "It reads the system property java.class.path ..." => "The system property java.class.path is read ..." "is named or not" => "is named or unnamed". "whether the initial module" => "whether the initial module (the module containing the main class)". -Alan From paul.sandoz at oracle.com Wed Jan 24 16:48:05 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 24 Jan 2018 08:48:05 -0800 Subject: [11] RFR JDK-8191170: Clarify if java.class.path can be undefined In-Reply-To: <449196ba-cf7a-1d78-d573-ce46947b9b7f@oracle.com> References: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> <449196ba-cf7a-1d78-d573-ce46947b9b7f@oracle.com> Message-ID: <6166F01B-E340-4ECD-BD4C-7218388A4E65@oracle.com> > On Jan 24, 2018, at 3:36 AM, Alan Bateman wrote: > > On 23/01/2018 22:40, mandy chung wrote: >> http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8191170/webrev.00/index.html >> >> A small adjustment in the javadoc. It clarifies what an empty value of "java.class.path" property is interpreted by the built-in application class loader. > It looks okay but I had to read it a few times. Same here. > A few suggestions that might make it a bit easier to read: > > "It reads the system property java.class.path ..." => "The system property java.class.path is read ..." > > "is named or not" => "is named or unnamed". > > "whether the initial module" => "whether the initial module (the module containing the main class)?. +1 to those. That sufficiently explains the initial module. "If named, the built-in system class loader will have no class path i.e. only find classes and resources only from the application module path;? => ?If named, the built-in system class loader will have no class path and will search for classes and resources using the application module path;" Paul. From paul.sandoz at oracle.com Wed Jan 24 17:42:53 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 24 Jan 2018 09:42:53 -0800 Subject: The new optimized version of Dual-Pivot Quicksort In-Reply-To: <6defd04f-28e0-64fb-a18c-9bb320203f35@mail.ru> References: <6defd04f-28e0-64fb-a18c-9bb320203f35@mail.ru> Message-ID: Hi Vladimir, Thanks. Quick update for those on the list: Doug and Vladimir are doing some more detailed performance analysis. Paul. > On Jan 19, 2018, at 5:38 AM, Vladimir Yaroslavskiy wrote: > > Hi team, > > In Sept 2009 Josh Bloch, Jon Bentley and I introduced new sorting > algorithm, Dual-Pivot Quicksort, for primitives in JDK 7 and later > I suggested several improvements of Dual-Pivot Quicksort, which > were integrated into JDK 8. > > Now I have more optimized and faster version of Dual-Pivot Quicksort. > I have been working on it for the last 5 years. Please, find the > summary of changes below and sources / diff at webrev [1]. > > All tests and benchmarking were run on the most recent build of JDK 10, > jdk-10-ea+39. The new version shows the better performance on different > inputs and guarantees n*log(n) on any data. > > The new implementation of Dual-Pivot Quicksort is 'all-in-one' version: > it contains one code for both parallel and sequential sorting algorithms. > > Suggested version is 10-20% faster on random data, 1.5-4 times faster > on nearly structured arrays, 1.5-2 times faster on period inputs. > > Parallel Dual-Pivot Quicksort is 1.5-3 times faster than current > algorithm based on merge sort. > > Benchmarking on the test suite, suggested by Jon Bentley, shows the > boost of performance in 1.4 times. This test suite contains several > types of inputs, such as random data, nearly structured arrays, data > with period and so on. Also there are several modifications of inputs > and parameters which are used in data building. We run sorting on all > combinations to compare two algorithms. > > Please let me know if you have any questions / comments. > > Summary of changes: > > DualPivotQuicksort class > ------------------------ > * Pivots are chosen with another step, the 1-st and 5-th candidates > are taken as pivots instead of 2-nd and 4-th. > * Splitting into parts is related to the golden ratio > * Pivot candidates are sorted by combination of 5-element > network sorting + insertion sort > * New backwards partitioning is simpler and more efficient > * Quicksort tuning parameters were updated > * Merging sort is invoked on each iteration from Quicksort > * Additional steps for float/double were fully rewritten > * Heap sort is invoked on the leftmost part > * Heap sort is used as a guard against quadratic time > * Parallel sorting is based on Dual-Pivot Quicksort > instead of merge sort > > SortingAlgorithms class > ----------------------- > * Merging sort and pair insertion sort were moved from > DualPivotQuicksort class > * Pair insertion sort was simplified and optimized > * New nano insertion sort was introduced for tiny arrays > * Merging sort was fully rewritten > * Optimized merging partitioning is used > * Merging parameters were updated > * Merging of runs was fully rewritten > * Fast version of heap sort was introduced > * Parallel merging sort was also provided > > Sorting / ParallelSorting classes > --------------------------------- > * New test cases were added > * Sources of these classes were unified > > Arrays class > ------------ > * Calls of Dual-Pivot Quicksort were updated > * Parallel sorting of primitives was switched to parallel > Dual-Pivot Quicksort > * Javadoc was modified > > ArraysParallelSortHelpers class > ------------------------------- > * Old implementation of parallel sorting for primitives was removed > * Javadoc was updated > > Thank you, > Vladimir > > -------------------------------------------------------------------- > [1] http://cr.openjdk.java.net/~alanb/DualPivotSortUpdate/webrev.01/ > -------------------------------------------------------------------- From mandy.chung at oracle.com Wed Jan 24 18:02:28 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 24 Jan 2018 10:02:28 -0800 Subject: [11] RFR JDK-8191170: Clarify if java.class.path can be undefined In-Reply-To: <6166F01B-E340-4ECD-BD4C-7218388A4E65@oracle.com> References: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> <449196ba-cf7a-1d78-d573-ce46947b9b7f@oracle.com> <6166F01B-E340-4ECD-BD4C-7218388A4E65@oracle.com> Message-ID: <0e605ecc-fe99-ec64-ffda-f2541ad95ada@oracle.com> On 1/24/18 8:48 AM, Paul Sandoz wrote: >> A few suggestions that might make it a bit easier to read: >> >> "It reads the system property java.class.path ..." => "The system property java.class.path is read ..." >> >> "is named or not" => "is named or unnamed". >> >> "whether the initial module" => "whether the initial module (the module containing the main class)?. > +1 to those. > > That sufficiently explains the initial module. > > "If named, the built-in system class loader will have no class path > i.e. only find classes and resources only from the application module > path;? > => > ?If named, the built-in system class loader will have no class path and > will search for classes and resources using the application module > path;" > Thanks for the suggestion.? Updated version: ???? *

The name of the built-in system class loader is {@code "app"}. ???? * The system property "{@code java.class.path}" is read during early ???? * initialization of the VM to determine the class path. ???? * An empty value of "{@code java.class.path}" property is interpreted ???? * differently depending on whether the initial module (the module ???? * containing the main class) is named or unnamed: ???? * If named, the built-in system class loader will have no class path and ???? * will search for classes and resources using the application module path; ???? * otherwise, if unnamed, it will set the class path to the current ???? * working directory. Mandy From mandy.chung at oracle.com Wed Jan 24 18:15:50 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 24 Jan 2018 10:15:50 -0800 Subject: [11] RFR JDK-8168682: jdk/test/java/lang/ClassLoader/forNameLeak/ClassForNameLeak.java fails with -Xcomp Message-ID: <1a0ee233-88bd-0530-7ef6-b29fd8bcdc7b@oracle.com> This patch revises the test to make the phantom reference reachable in order to ensure that the reference is enqueued for verification. Webrev at: http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8168682/webrev.00/ thanks Mandy From Alan.Bateman at oracle.com Wed Jan 24 18:25:48 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 24 Jan 2018 18:25:48 +0000 Subject: [11] RFR JDK-8191170: Clarify if java.class.path can be undefined In-Reply-To: <0e605ecc-fe99-ec64-ffda-f2541ad95ada@oracle.com> References: <22bccf84-85cd-7b8f-cc4f-fd775e4704d5@oracle.com> <449196ba-cf7a-1d78-d573-ce46947b9b7f@oracle.com> <6166F01B-E340-4ECD-BD4C-7218388A4E65@oracle.com> <0e605ecc-fe99-ec64-ffda-f2541ad95ada@oracle.com> Message-ID: <41fa4afa-3a59-ae78-68cf-43b8923fc992@oracle.com> On 24/01/2018 18:02, mandy chung wrote: > : > > Thanks for the suggestion.? Updated version: > > ???? *

The name of the built-in system class loader is {@code "app"}. > ???? * The system property "{@code java.class.path}" is read during early > ???? * initialization of the VM to determine the class path. > ???? * An empty value of "{@code java.class.path}" property is interpreted > ???? * differently depending on whether the initial module (the module > ???? * containing the main class) is named or unnamed: > ???? * If named, the built-in system class loader will have no class > path and > ???? * will search for classes and resources using the application > module path; > ???? * otherwise, if unnamed, it will set the class path to the current > ???? * working directory. This looks good. -Alan From brian.burkhalter at oracle.com Wed Jan 24 19:50:55 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Wed, 24 Jan 2018 11:50:55 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <49B674D6-3736-419F-AD19-E4F3AE12660A@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <3BE43426-8CA8-4B1F-B442-229083AFAF25@oracle.com> <49B674D6-3736-419F-AD19-E4F3AE12660A@oracle.com> Message-ID: <2CB4460B-2C06-4043-BF4A-061304FBB681@oracle.com> On Jan 23, 2018, at 4:50 PM, Brian Burkhalter wrote: > On Jan 23, 2018, at 1:19 AM, Weijun Wang wrote: > >> + * Therefore, the method may be safely called with very large values of >> + * {@code len} provided sufficient memory is available. >> >> What does "sufficient memory" mean? For len, or the number of available bytes? > > It means enough bytes for the collectivity of the intermediate and returned buffers. This is already stated to be proportional to ?len?. All right, to make sure this horse is truly dead here?s one more revision. The changes with respect to the previous revision are [1] and the overall changes versus the SCM base are [2]. The .02-.03 differences are: A) Add an @implNote at line 368. B) Minor memory use improvement at line 392. Thanks, Brian [1] http://cr.openjdk.java.net/~bpb/8139206/webrev.02-03/ [2] http://cr.openjdk.java.net/~bpb/8139206/webrev.03/ From Roger.Riggs at Oracle.com Wed Jan 24 21:06:57 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 24 Jan 2018 16:06:57 -0500 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: <2CB4460B-2C06-4043-BF4A-061304FBB681@oracle.com> References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <3BE43426-8CA8-4B1F-B442-229083AFAF25@oracle.com> <49B674D6-3736-419F-AD19-E4F3AE12660A@oracle.com> <2CB4460B-2C06-4043-BF4A-061304FBB681@oracle.com> Message-ID: +1 On 1/24/2018 2:50 PM, Brian Burkhalter wrote: > On Jan 23, 2018, at 4:50 PM, Brian Burkhalter > > wrote: > >> On Jan 23, 2018, at 1:19 AM, Weijun Wang > > wrote: >> >>> + ????* Therefore, the method may be safely called with very large >>> values of >>> + ????* {@code len} provided sufficient memory is available. >>> >>> What does "sufficient memory" mean? For len, or the number of >>> available bytes? >> >> It means enough bytes for the collectivity of the intermediate and >> returned buffers. This is already stated to be proportional to ?len?. > > All right, to make sure this horse is truly dead here?s one more > revision. The changes with respect to the previous revision are [1] > and the overall changes versus the SCM base are [2]. The .02-.03 > differences are: > > A) Add an @implNote at line 368. > B) Minor memory use improvement at line 392. > > Thanks, > > Brian > > [1] http://cr.openjdk.java.net/~bpb/8139206/webrev.02-03/ > > [2] http://cr.openjdk.java.net/~bpb/8139206/webrev.03/ > From amy.lu at oracle.com Thu Jan 25 03:30:55 2018 From: amy.lu at oracle.com (Amy Lu) Date: Thu, 25 Jan 2018 11:30:55 +0800 Subject: [JDK 11] RFR 8157903: (prop) move Properties tests into OpenJDK Message-ID: <82d6608a-be88-0e12-da82-43124dd3f347@oracle.com> Please review the patch to move Properties tests into OpenJDK. bug: https://bugs.openjdk.java.net/browse/JDK-8157903 webrev: http://cr.openjdk.java.net/~amlu/8157903/webrev.00/ Thanks, Amy From mandy.chung at oracle.com Thu Jan 25 05:29:05 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 24 Jan 2018 21:29:05 -0800 Subject: [JDK 11] RFR 8157903: (prop) move Properties tests into OpenJDK In-Reply-To: <82d6608a-be88-0e12-da82-43124dd3f347@oracle.com> References: <82d6608a-be88-0e12-da82-43124dd3f347@oracle.com> Message-ID: Looks okay.? The copyright start year in BlankLines.java is different than the others. You can fix it before you push. Mandy On 1/24/18 7:30 PM, Amy Lu wrote: > Please review the patch to move Properties tests into OpenJDK. > > bug: https://bugs.openjdk.java.net/browse/JDK-8157903 > webrev: http://cr.openjdk.java.net/~amlu/8157903/webrev.00/ > > Thanks, > Amy From ecki at zusammenkunft.net Thu Jan 25 11:58:28 2018 From: ecki at zusammenkunft.net (Bernd) Date: Thu, 25 Jan 2018 12:58:28 +0100 Subject: Oracle Java 8u161 regression in XML Schema Factory Message-ID: Hello, some of our unit tests (using PowerMock and xmlunit) fail with 8u161 (and u162) but work with 8u152. I cant reproduce the fault in a stand-alone program so it seems to be related to classloader/reflection magic of those tools, sorry. Is this a regression of 8159240 (not public?) Here is the stacktrace anyway: com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema factory class com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl does not extend from SchemaDVFactory. at com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.getInstance(SchemaDVFactory.java:75) at com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.getInstance(SchemaDVFactory.java:57) at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.reset(XMLSchemaLoader.java:1024) at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.loadGrammar(XMLSchemaLoader.java:556) at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.loadGrammar(XMLSchemaLoader.java:535) at com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory.newSchema(XMLSchemaFactory.java:254) at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.java:638) at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.java:654) at com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchemaValidator(BuilderTestHelper.java:57) at com.seeburger.api.test.helpers.BuilderTestHelper.validateAndCompare(BuilderTestHelper.java:73) at com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuilderTest.java:196) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:310) at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:89) at org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:97) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(PowerMockJUnit44RunnerDelegateImpl.java:294) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestInSuper(PowerMockJUnit47RunnerDelegateImpl.java:127) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(PowerMockJUnit47RunnerDelegateImpl.java:82) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThenTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:87) at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:207) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:146) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) at org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:34) at org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) at org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) at org.powermock.modules.junit4.common.internal.impl.JUnit4TestSuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) at org.powermock.modules.junit4.common.internal.impl.AbstractCommonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) at org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:59) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:539) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:761) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:461) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:207) on the classpath jaxb-impl-2.2.5.jar but the specific packages are only loaded from rt.jar and redefined. I asume the later is done by Powermock. Line 611: [Loaded com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] Line 616: [Loaded com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] Line 617: [Loaded com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] Line 618: [Loaded com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from __JVM_DefineClass__] Line 619: [Loaded com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from __JVM_DefineClass__] Line 620: [Loaded com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from __JVM_DefineClass__] Is that something you are concerned? Gruss Bernd From sean.coffey at oracle.com Thu Jan 25 16:41:40 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Thu, 25 Jan 2018 16:41:40 +0000 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: References: Message-ID: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> On 25/01/2018 11:58, Bernd wrote: > Hello, > > some of our unit tests (using PowerMock and xmlunit) fail with 8u161 (and > u162) but work with 8u152. > > I cant reproduce the fault in a stand-alone program so it seems to be > related to classloader/reflection magic of those tools, sorry. > > Is this a regression of 8159240 > (not public?) Fixes in the CPU releases are kept private - hence the above bug is not public. The changesets do become public once the release is made public though. See : http://hg.openjdk.java.net/jdk8u/jdk8u/jaxws/rev/06086cb6c349 I don't think it's a factor for what you're seeing. Classes nearer to those below were touched via JDK-8186080: Transform XML interfaces http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 This may be connected with some tools trying to redefine the classes perhaps. Needs more investigating. Perhaps the XMLSchemaLoader changes are a factor ? regards, Sean. > > Here is the stacktrace anyway: > > com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema > factory class > com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl does not > extend from SchemaDVFactory. > at > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.getInstance(SchemaDVFactory.java:75) > at > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.getInstance(SchemaDVFactory.java:57) > at > com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.reset(XMLSchemaLoader.java:1024) > at > com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.loadGrammar(XMLSchemaLoader.java:556) > at > com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.loadGrammar(XMLSchemaLoader.java:535) > at > com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory.newSchema(XMLSchemaFactory.java:254) > at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.java:638) > at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.java:654) > at > com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchemaValidator(BuilderTestHelper.java:57) > at > com.seeburger.api.test.helpers.BuilderTestHelper.validateAndCompare(BuilderTestHelper.java:73) > at > com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuilderTest.java:196) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:310) > at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:89) > at > org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:97) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(PowerMockJUnit44RunnerDelegateImpl.java:294) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestInSuper(PowerMockJUnit47RunnerDelegateImpl.java:127) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(PowerMockJUnit47RunnerDelegateImpl.java:82) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThenTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) > at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:87) > at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:207) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:146) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) > at > org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:34) > at > org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) > at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) > at > org.powermock.modules.junit4.common.internal.impl.JUnit4TestSuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) > at > org.powermock.modules.junit4.common.internal.impl.AbstractCommonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) > at > org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:59) > at > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) > at > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:539) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:761) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:461) > at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:207) > > on the classpath jaxb-impl-2.2.5.jar but the specific packages are only > loaded from rt.jar and redefined. I asume the later is done by Powermock. > > Line 611: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from C:\Program > Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > Line 616: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from > C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > Line 617: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from > C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > Line 618: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > __JVM_DefineClass__] > Line 619: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from > __JVM_DefineClass__] > Line 620: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from > __JVM_DefineClass__] > > Is that something you are concerned? > > Gruss > Bernd > From brian.burkhalter at oracle.com Thu Jan 25 19:22:22 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Thu, 25 Jan 2018 11:22:22 -0800 Subject: RFR 8139206: Add InputStream readNBytes(int len) In-Reply-To: References: <39A73062-3E46-46F6-86B3-FF52AB6CD1D0@oracle.com> <83e1b0a6-b5be-64e2-5d6d-9811f64a1fe0@Oracle.com> <443A39F9-273C-48F3-ACC5-37307B240FEA@oracle.com> <3BE43426-8CA8-4B1F-B442-229083AFAF25@oracle.com> <49B674D6-3736-419F-AD19-E4F3AE12660A@oracle.com> <2CB4460B-2C06-4043-BF4A-061304FBB681@oracle.com> Message-ID: I have moved the CSR [1] back to Draft and updated it according to the content of webrev.03. If there are no more comments by tomorrow I will move it once again to Finalized. After that, once the CSR has been re-approved, then if there are no further comments on the changes I will go ahead and push the fix, assuming there are no unexpected failures in rerunning the regression tests. Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8194956 On Jan 24, 2018, at 1:06 PM, Roger Riggs wrote: > +1 > > On 1/24/2018 2:50 PM, Brian Burkhalter wrote: >> On Jan 23, 2018, at 4:50 PM, Brian Burkhalter wrote: >> >>> On Jan 23, 2018, at 1:19 AM, Weijun Wang wrote: >>> >>>> + * Therefore, the method may be safely called with very large values of >>>> + * {@code len} provided sufficient memory is available. >>>> >>>> What does "sufficient memory" mean? For len, or the number of available bytes? >>> >>> It means enough bytes for the collectivity of the intermediate and returned buffers. This is already stated to be proportional to ?len?. >> >> All right, to make sure this horse is truly dead here?s one more revision. The changes with respect to the previous revision are [1] and the overall changes versus the SCM base are [2]. The .02-.03 differences are: >> >> A) Add an @implNote at line 368. >> B) Minor memory use improvement at line 392. >> >> Thanks, >> >> Brian >> >> [1] http://cr.openjdk.java.net/~bpb/8139206/webrev.02-03/ >> [2] http://cr.openjdk.java.net/~bpb/8139206/webrev.03/ From brent.christian at oracle.com Thu Jan 25 19:30:40 2018 From: brent.christian at oracle.com (Brent Christian) Date: Thu, 25 Jan 2018 11:30:40 -0800 Subject: [11] RFR JDK-8168682: jdk/test/java/lang/ClassLoader/forNameLeak/ClassForNameLeak.java fails with -Xcomp In-Reply-To: <1a0ee233-88bd-0530-7ef6-b29fd8bcdc7b@oracle.com> References: <1a0ee233-88bd-0530-7ef6-b29fd8bcdc7b@oracle.com> Message-ID: <0b6ea12b-7fea-e6fb-36dc-42a43549b4bb@oracle.com> Hi, Mandy Interesting problem. Your changes look fine. (The static finals could be all caps, but I wouldn't bother with further changes just for that.) Thanks, -Brent On 1/24/18 10:15 AM, mandy chung wrote: > This patch revises the test to make the phantom reference reachable > in order to ensure that the reference is enqueued for verification. > > Webrev at: > http://cr.openjdk.java.net/~mchung/jdk11/webrevs/8168682/webrev.00/ > > thanks > Mandy From mandy.chung at oracle.com Thu Jan 25 19:33:22 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 25 Jan 2018 11:33:22 -0800 Subject: [11] RFR JDK-8196127: Dead code in VersionProps.java.template Message-ID: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> Trivial fix to remove unused isHeadless variable. $ hg diff src/java.base/share/classes/java/lang/VersionProps.java.template diff --git a/src/java.base/share/classes/java/lang/VersionProps.java.template b/src/java.base/share/classes/java/lang/VersionProps.java.template --- a/src/java.base/share/classes/java/lang/VersionProps.java.template +++ b/src/java.base/share/classes/java/lang/VersionProps.java.template @@ -166,15 +166,8 @@ ????? * Print version info. ????? */ ???? private static void print(boolean err, boolean newln) { -??????? boolean isHeadless = false; ???????? PrintStream ps = err ? System.err : System.out; -??????? /* Report that we're running headless if the property is true */ -??????? String headless = System.getProperty("java.awt.headless"); -??????? if ( (headless != null) && (headless.equalsIgnoreCase("true")) ) { -??????????? isHeadless = true; -??????? } - ???????? /* First line: platform version. */ ???????? if (err) { ???????????? ps.println(launcher_name + " version \"" + java_version + "\"" Mandy From paul.sandoz at oracle.com Thu Jan 25 19:35:29 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Thu, 25 Jan 2018 11:35:29 -0800 Subject: [11] RFR JDK-8196127: Dead code in VersionProps.java.template In-Reply-To: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> References: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> Message-ID: <474587CE-BA65-421A-AA99-835226F8129D@oracle.com> +1 Paul. > On Jan 25, 2018, at 11:33 AM, mandy chung wrote: > > Trivial fix to remove unused isHeadless variable. > > $ hg diff src/java.base/share/classes/java/lang/VersionProps.java.template > diff --git a/src/java.base/share/classes/java/lang/VersionProps.java.template b/src/java.base/share/classes/java/lang/VersionProps.java.template > --- a/src/java.base/share/classes/java/lang/VersionProps.java.template > +++ b/src/java.base/share/classes/java/lang/VersionProps.java.template > @@ -166,15 +166,8 @@ > * Print version info. > */ > private static void print(boolean err, boolean newln) { > - boolean isHeadless = false; > PrintStream ps = err ? System.err : System.out; > > - /* Report that we're running headless if the property is true */ > - String headless = System.getProperty("java.awt.headless"); > - if ( (headless != null) && (headless.equalsIgnoreCase("true")) ) { > - isHeadless = true; > - } > - > /* First line: platform version. */ > if (err) { > ps.println(launcher_name + " version \"" + java_version + "\"" > > Mandy From lance.andersen at oracle.com Thu Jan 25 19:35:52 2018 From: lance.andersen at oracle.com (Lance Andersen) Date: Thu, 25 Jan 2018 14:35:52 -0500 Subject: [11] RFR JDK-8196127: Dead code in VersionProps.java.template In-Reply-To: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> References: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> Message-ID: <9694FF20-AACF-4E1E-83A9-E2B270D4BDD6@oracle.com> +1 > On Jan 25, 2018, at 2:33 PM, mandy chung wrote: > > Trivial fix to remove unused isHeadless variable. > > $ hg diff src/java.base/share/classes/java/lang/VersionProps.java.template > diff --git a/src/java.base/share/classes/java/lang/VersionProps.java.template b/src/java.base/share/classes/java/lang/VersionProps.java.template > --- a/src/java.base/share/classes/java/lang/VersionProps.java.template > +++ b/src/java.base/share/classes/java/lang/VersionProps.java.template > @@ -166,15 +166,8 @@ > * Print version info. > */ > private static void print(boolean err, boolean newln) { > - boolean isHeadless = false; > PrintStream ps = err ? System.err : System.out; > > - /* Report that we're running headless if the property is true */ > - String headless = System.getProperty("java.awt.headless"); > - if ( (headless != null) && (headless.equalsIgnoreCase("true")) ) { > - isHeadless = true; > - } > - > /* First line: platform version. */ > if (err) { > ps.println(launcher_name + " version \"" + java_version + "\"" > > Mandy Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037 Oracle Java Engineering 1 Network Drive Burlington, MA 01803 Lance.Andersen at oracle.com From sergei.tsypanov at yandex.ru Thu Jan 25 22:02:40 2018 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Fri, 26 Jan 2018 00:02:40 +0200 Subject: [PATCH] Inefficient ArrayList.subList().toArray() Message-ID: <520081516917760@web45g.yandex.ru> Hi, I've run into poor performance of toArray() method called on result of subList() taken from java.util.ArrayList. Consider simple benchmark: @BenchmarkMode(Mode.AverageTime) @OutputTimeUnit(TimeUnit.NANOSECONDS) @Fork(jvmArgsAppend = {"-XX:+UseParallelGC", "-Xms1g", "-Xmx1g"}) public class SubListToArrayBenchmark { @Benchmark public Integer[] list_typeChecked(Data holder) { return holder.list.toArray(new Integer[0]); } @Benchmark public Integer[] subList_typeChecked(Data holder) { return holder.list.subList(0, holder.size).toArray(new Integer[0]); } @State(Scope.Thread) public static class Data { ArrayList list; @Param({"0", "10", "100", "1000"}) int size; @Setup public void setup() { list = IntStream .range(0, size) .boxed() .collect(toCollection(ArrayList::new)); } } } With Java 1.8.0_162 on my PC (Intel i5-4690 3,5 GHz) it yields these results: Benchmark (size) Mode Cnt Score Error Units list_typeChecked 0 avgt 50 4,630 ? 0,062 ns/op list_typeChecked 10 avgt 50 16,629 ? 0,140 ns/op list_typeChecked 100 avgt 50 79,783 ? 1,676 ns/op list_typeChecked 1000 avgt 50 742,757 ? 10,357 ns/op subList_typeChecked 0 avgt 50 11,833 ? 0,801 ns/op subList_typeChecked 10 avgt 50 42,713 ? 0,590 ns/op subList_typeChecked 100 avgt 50 197,336 ? 3,560 ns/op subList_typeChecked 1000 avgt 50 1765,187 ? 19,729 ns/op With Java 9 subList performs slightly better but still much worse than list. Despite the fact that amount of data transfered from list to array is the same for both methods there's a huge difference in absolute values. Instantiation of SubList costs in average only 9.591 ? 0.650 ns and is independent on the size of its source so it couldn't be root cause. Here SubLists taken from ArrayList calls AbstractCollection.toArray() method while implementing RandomAccess and being array-based by its nature. Array-based ArrayList provides efficient implementation of toArray(T[]) and we can apply the same approach for ArrayList.SubList. Here's the patch for JDK 9: -------------------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/src/java.base/share/classes/java/util/ArrayList.java b/src/java.base/share/classes/java/util/ArrayList.java --- a/src/java.base/share/classes/java/util/ArrayList.java +++ b/src/java.base/share/classes/java/util/ArrayList.java @@ -1363,6 +1363,20 @@ } }; } + + public Object[] toArray() { + return Arrays.copyOfRange(root.elementData, offset, size); + } + + @SuppressWarnings("unchecked") + public T[] toArray(T[] a) { + if (a.length < size) + return (T[]) Arrays.copyOfRange(root.elementData, offset, size, a.getClass()); + System.arraycopy(root.elementData, offset, a, 0, size); + if (a.length > size) + a[size] = null; + return a; + } } /** -------------------------------------------------------------------------------------------------------------------------------------------------------- Best regards, Sergey Tsypanov From rednaxelafx at gmail.com Thu Jan 25 22:30:31 2018 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 25 Jan 2018 14:30:31 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: <520081516917760@web45g.yandex.ru> References: <520081516917760@web45g.yandex.ru> Message-ID: Hi Sergey, Not a Reviewer but I like this patch a lot. Thanks for contributing it to OpenJDK! Best regards, Kris On Thu, Jan 25, 2018 at 2:02 PM, ?????? ??????? wrote: > Hi, > > I've run into poor performance of toArray() method called on result of > subList() taken from java.util.ArrayList. > > Consider simple benchmark: > > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @Fork(jvmArgsAppend = {"-XX:+UseParallelGC", "-Xms1g", "-Xmx1g"}) > public class SubListToArrayBenchmark { > > @Benchmark > public Integer[] list_typeChecked(Data holder) { > return holder.list.toArray(new Integer[0]); > } > > @Benchmark > public Integer[] subList_typeChecked(Data holder) { > return holder.list.subList(0, holder.size).toArray(new Integer[0]); > } > > @State(Scope.Thread) > public static class Data { > ArrayList list; > > @Param({"0", "10", "100", "1000"}) > int size; > > @Setup > public void setup() { > list = IntStream > .range(0, size) > .boxed() > .collect(toCollection(ArrayList::new)); > } > } > } > > > With Java 1.8.0_162 on my PC (Intel i5-4690 3,5 GHz) it yields these > results: > > Benchmark (size) Mode Cnt Score Error Units > list_typeChecked 0 avgt 50 4,630 ? 0,062 > ns/op > list_typeChecked 10 avgt 50 16,629 ? 0,140 ns/op > list_typeChecked 100 avgt 50 79,783 ? 1,676 ns/op > list_typeChecked 1000 avgt 50 742,757 ? 10,357 ns/op > > subList_typeChecked 0 avgt 50 11,833 ? 0,801 ns/op > subList_typeChecked 10 avgt 50 42,713 ? 0,590 ns/op > subList_typeChecked 100 avgt 50 197,336 ? 3,560 ns/op > subList_typeChecked 1000 avgt 50 1765,187 ? 19,729 ns/op > > With Java 9 subList performs slightly better but still much worse than > list. > > > Despite the fact that amount of data transfered from list to array is the > same for both methods there's a huge difference in absolute values. > > Instantiation of SubList costs in average only 9.591 ? 0.650 ns and is > independent on the size of its source so it couldn't be root cause. > > Here SubLists taken from ArrayList calls AbstractCollection.toArray() > method while implementing RandomAccess and being array-based by its nature. > Array-based ArrayList provides efficient implementation of toArray(T[]) > and we can apply the same approach for ArrayList.SubList. > > Here's the patch for JDK 9: > > ------------------------------------------------------------ > ------------------------------------------------------------ > -------------------------------- > > diff --git a/src/java.base/share/classes/java/util/ArrayList.java > b/src/java.base/share/classes/java/util/ArrayList.java > --- a/src/java.base/share/classes/java/util/ArrayList.java > +++ b/src/java.base/share/classes/java/util/ArrayList.java > @@ -1363,6 +1363,20 @@ > } > }; > } > + > + public Object[] toArray() { > + return Arrays.copyOfRange(root.elementData, offset, size); > + } > + > + @SuppressWarnings("unchecked") > + public T[] toArray(T[] a) { > + if (a.length < size) > + return (T[]) Arrays.copyOfRange(root.elementData, > offset, size, a.getClass()); > + System.arraycopy(root.elementData, offset, a, 0, size); > + if (a.length > size) > + a[size] = null; > + return a; > + } > } > > /** > > ------------------------------------------------------------ > ------------------------------------------------------------ > -------------------------------- > > Best regards, > Sergey Tsypanov > From andrey.x.nazarov at oracle.com Fri Jan 26 00:05:06 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Thu, 25 Jan 2018 16:05:06 -0800 Subject: RFR: 8186009 tools launcher test AddExportsAndOpensInManifest.java fails intermittently: AccessDeniedException Message-ID: Hi, Could you review fix in launcher tests. Fix changes ?file.jar? name to unique name for every test case. It eliminates dependency between test cases on the file. http://cr.openjdk.java.net/~anazarov/JDK-8186009/webrev.03 https://bugs.openjdk.java.net/browse/JDK-8186009 ?Andrei From andrey.x.nazarov at oracle.com Fri Jan 26 00:19:41 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Thu, 25 Jan 2018 16:19:41 -0800 Subject: RFR: 8186009 tools launcher test AddExportsAndOpensInManifest.java fails intermittently: AccessDeniedException In-Reply-To: References: Message-ID: Screenshot with new layout http://cr.openjdk.java.net/~anazarov/JDK-8186009/Screen%20Shot%202018-01-25%20at%2015.40.15.png CR: http://cr.openjdk.java.net/~anazarov/JDK-8186009/webrev.04/ ?Thanks, Andrei > On 25 Jan 2018, at 16:05, Andrey Nazarov wrote: > > Hi, > > Could you review fix in launcher tests. Fix changes ?file.jar? name to unique name for every test case. It eliminates dependency between test cases on the file. > > http://cr.openjdk.java.net/~anazarov/JDK-8186009/webrev.03 > > https://bugs.openjdk.java.net/browse/JDK-8186009 > > ?Andrei From rednaxelafx at gmail.com Fri Jan 26 00:23:54 2018 From: rednaxelafx at gmail.com (Krystal Mok) Date: Thu, 25 Jan 2018 16:23:54 -0800 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> Message-ID: Hi guys, A coworker of mine had hit this issue last night on 8u161 and it worked fine on 8u151: ERROR: /home/myuser/.cache/bazel/_bazel_myuser/some_hash_code/external/jackson_datatype_joda_shaded/BUILD:5:1: Building external/jackson_datatype_joda_shaded/libjackson-datatype-joda-class.jar (35 source files) failed (Exit 1) java.lang.InternalError: Cannot find requested resource bundle for locale en_US at com.sun.tools.javac.util.JavacMessages.getBundles(JavacMessages.java:128) at com.sun.tools.javac.util.JavacMessages.getLocalizedString(JavacMessages.java:147) at com.sun.tools.javac.util.JavacMessages.getLocalizedString(JavacMessages.java:140) at com.sun.tools.javac.util.Log.localize(Log.java:788) at com.sun.tools.javac.util.Log.printLines(Log.java:586) at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:170) at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:96) at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:90) at com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:105) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder$1.invokeJavac(SimpleJavaLibraryBuilder.java:106) at com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:53) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:109) at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:117) at com.google.devtools.build.buildjar.BazelJavaBuilder.processRequest(BazelJavaBuilder.java:105) at com.google.devtools.build.buildjar.BazelJavaBuilder.runPersistentWorker(BazelJavaBuilder.java:67) at com.google.devtools.build.buildjar.BazelJavaBuilder.main(BazelJavaBuilder.java:45) Caused by: java.util.MissingResourceException: Can't find bundle for base name com.google.errorprone.errors, locale en_US at java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1573) at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1396) at java.util.ResourceBundle.getBundle(ResourceBundle.java:854) at com.sun.tools.javac.util.JavacMessages.lambda$add$0(JavacMessages.java:106) at com.sun.tools.javac.util.JavacMessages.getBundles(JavacMessages.java:125) ... 15 more Resource bundle loading issue...could that be related to this XML issue here? Thanks, Kris On Thu, Jan 25, 2018 at 8:41 AM, Se?n Coffey wrote: > On 25/01/2018 11:58, Bernd wrote: > > Hello, >> >> some of our unit tests (using PowerMock and xmlunit) fail with 8u161 (and >> u162) but work with 8u152. >> >> I cant reproduce the fault in a stand-alone program so it seems to be >> related to classloader/reflection magic of those tools, sorry. >> >> Is this a regression of 8159240 >> (not public?) >> > Fixes in the CPU releases are kept private - hence the above bug is not > public. The changesets do become public once the release is made public > though. See : http://hg.openjdk.java.net/jdk8u/jdk8u/jaxws/rev/06086cb6c34 > 9 > > I don't think it's a factor for what you're seeing. > > Classes nearer to those below were touched via JDK-8186080: Transform XML > interfaces > http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 > http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 > > This may be connected with some tools trying to redefine the classes > perhaps. Needs more investigating. Perhaps the XMLSchemaLoader changes are > a factor ? > > regards, > Sean. > > >> Here is the stacktrace anyway: >> >> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema >> factory class >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl does >> not >> extend from SchemaDVFactory. >> at >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. >> getInstance(SchemaDVFactory.java:75) >> at >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory. >> getInstance(SchemaDVFactory.java:57) >> at >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >> reset(XMLSchemaLoader.java:1024) >> at >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >> loadGrammar(XMLSchemaLoader.java:556) >> at >> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader. >> loadGrammar(XMLSchemaLoader.java:535) >> at >> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema >> Factory.newSchema(XMLSchemaFactory.java:254) >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. >> java:638) >> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory. >> java:654) >> at >> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema >> Validator(BuilderTestHelper.java:57) >> at >> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd >> Compare(BuilderTestHelper.java:73) >> at >> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild >> erTest.java:196) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >> ssorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >> thodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:498) >> at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod >> (PowerMockJUnit44RunnerDelegateImpl.java:310) >> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie. >> java:89) >> at >> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh >> enAfters(MethodRoadie.java:97) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P >> owerMockJUnit44RunnerDelegateImpl.java:294) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn >> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P >> owerMockJUnit47RunnerDelegateImpl.java:82) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe >> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) >> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie >> .java:87) >> at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl.invokeTestMethod(PowerMockJUni >> t44RunnerDelegateImpl.java:207) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:146) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) >> at >> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR >> oadie.java:34) >> at >> org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) >> at >> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >> unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) >> at >> org.powermock.modules.junit4.common.internal.impl.JUnit4Test >> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) >> at >> org.powermock.modules.junit4.common.internal.impl.AbstractCo >> mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) >> at >> org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:59) >> at >> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. >> run(JUnit4TestReference.java:86) >> at >> org.eclipse.jdt.internal.junit.runner.TestExecution.run( >> TestExecution.java:38) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >> sts(RemoteTestRunner.java:539) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >> sts(RemoteTestRunner.java:761) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( >> RemoteTestRunner.java:461) >> at >> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( >> RemoteTestRunner.java:207) >> >> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only >> loaded from rt.jar and redefined. I asume the later is done by Powermock. >> >> Line 611: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >> C:\Program >> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >> Line 616: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >> Line 617: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from >> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >> Line 618: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >> __JVM_DefineClass__] >> Line 619: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from >> __JVM_DefineClass__] >> Line 620: [Loaded >> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from >> __JVM_DefineClass__] >> >> Is that something you are concerned? >> >> Gruss >> Bernd >> >> > > From martinrb at google.com Fri Jan 26 00:24:16 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jan 2018 16:24:16 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: <520081516917760@web45g.yandex.ru> References: <520081516917760@web45g.yandex.ru> Message-ID: Thanks, ?????? I filed a bug for you https://bugs.openjdk.java.net/browse/JDK-8196207 On Thu, Jan 25, 2018 at 2:02 PM, ?????? ??????? wrote: > Hi, > > I've run into poor performance of toArray() method called on result of > subList() taken from java.util.ArrayList. > > Consider simple benchmark: > > @BenchmarkMode(Mode.AverageTime) > @OutputTimeUnit(TimeUnit.NANOSECONDS) > @Fork(jvmArgsAppend = {"-XX:+UseParallelGC", "-Xms1g", "-Xmx1g"}) > public class SubListToArrayBenchmark { > > @Benchmark > public Integer[] list_typeChecked(Data holder) { > return holder.list.toArray(new Integer[0]); > } > > @Benchmark > public Integer[] subList_typeChecked(Data holder) { > return holder.list.subList(0, holder.size).toArray(new Integer[0]); > } > > @State(Scope.Thread) > public static class Data { > ArrayList list; > > @Param({"0", "10", "100", "1000"}) > int size; > > @Setup > public void setup() { > list = IntStream > .range(0, size) > .boxed() > .collect(toCollection(ArrayList::new)); > } > } > } > > > With Java 1.8.0_162 on my PC (Intel i5-4690 3,5 GHz) it yields these > results: > > Benchmark (size) Mode Cnt Score Error Units > list_typeChecked 0 avgt 50 4,630 ? 0,062 > ns/op > list_typeChecked 10 avgt 50 16,629 ? 0,140 ns/op > list_typeChecked 100 avgt 50 79,783 ? 1,676 ns/op > list_typeChecked 1000 avgt 50 742,757 ? 10,357 ns/op > > subList_typeChecked 0 avgt 50 11,833 ? 0,801 ns/op > subList_typeChecked 10 avgt 50 42,713 ? 0,590 ns/op > subList_typeChecked 100 avgt 50 197,336 ? 3,560 ns/op > subList_typeChecked 1000 avgt 50 1765,187 ? 19,729 ns/op > > With Java 9 subList performs slightly better but still much worse than > list. > > > Despite the fact that amount of data transfered from list to array is the > same for both methods there's a huge difference in absolute values. > > Instantiation of SubList costs in average only 9.591 ? 0.650 ns and is > independent on the size of its source so it couldn't be root cause. > > Here SubLists taken from ArrayList calls AbstractCollection.toArray() > method while implementing RandomAccess and being array-based by its nature. > Array-based ArrayList provides efficient implementation of toArray(T[]) > and we can apply the same approach for ArrayList.SubList. > > Here's the patch for JDK 9: > > ------------------------------------------------------------ > ------------------------------------------------------------ > -------------------------------- > > diff --git a/src/java.base/share/classes/java/util/ArrayList.java > b/src/java.base/share/classes/java/util/ArrayList.java > --- a/src/java.base/share/classes/java/util/ArrayList.java > +++ b/src/java.base/share/classes/java/util/ArrayList.java > @@ -1363,6 +1363,20 @@ > } > }; > } > + > + public Object[] toArray() { > + return Arrays.copyOfRange(root.elementData, offset, size); > + } > + > + @SuppressWarnings("unchecked") > + public T[] toArray(T[] a) { > + if (a.length < size) > + return (T[]) Arrays.copyOfRange(root.elementData, > offset, size, a.getClass()); > + System.arraycopy(root.elementData, offset, a, 0, size); > + if (a.length > size) > + a[size] = null; > + return a; > + } > } > > /** > > ------------------------------------------------------------ > ------------------------------------------------------------ > -------------------------------- > > Best regards, > Sergey Tsypanov > From john.r.rose at oracle.com Fri Jan 26 00:41:04 2018 From: john.r.rose at oracle.com (John Rose) Date: Thu, 25 Jan 2018 16:41:04 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: <520081516917760@web45g.yandex.ru> References: <520081516917760@web45g.yandex.ru> Message-ID: On Jan 25, 2018, at 2:02 PM, ?????? ??????? wrote: > > + return (T[]) Arrays.copyOfRange(root.elementData, offset, size, a.getClass()); Giving this a quick glance: I think you may want s/size/offset+size/. There should be calls to checkForComodification. From david.holmes at oracle.com Fri Jan 26 02:24:59 2018 From: david.holmes at oracle.com (David Holmes) Date: Fri, 26 Jan 2018 12:24:59 +1000 Subject: [11] RFR JDK-8196127: Dead code in VersionProps.java.template In-Reply-To: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> References: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> Message-ID: <045bd1c6-e43a-17af-9c13-44a128c20c4f@oracle.com> Wow that was fast! :) Looks good. Thanks, David On 26/01/2018 5:33 AM, mandy chung wrote: > Trivial fix to remove unused isHeadless variable. > > $ hg diff src/java.base/share/classes/java/lang/VersionProps.java.template > diff --git > a/src/java.base/share/classes/java/lang/VersionProps.java.template > b/src/java.base/share/classes/java/lang/VersionProps.java.template > --- a/src/java.base/share/classes/java/lang/VersionProps.java.template > +++ b/src/java.base/share/classes/java/lang/VersionProps.java.template > @@ -166,15 +166,8 @@ > ????? * Print version info. > ????? */ > ???? private static void print(boolean err, boolean newln) { > -??????? boolean isHeadless = false; > ???????? PrintStream ps = err ? System.err : System.out; > > -??????? /* Report that we're running headless if the property is true */ > -??????? String headless = System.getProperty("java.awt.headless"); > -??????? if ( (headless != null) && (headless.equalsIgnoreCase("true")) ) { > -??????????? isHeadless = true; > -??????? } > - > ???????? /* First line: platform version. */ > ???????? if (err) { > ???????????? ps.println(launcher_name + " version \"" + java_version + > "\"" > > Mandy From huizhe.wang at oracle.com Fri Jan 26 03:00:07 2018 From: huizhe.wang at oracle.com (Joe Wang) Date: Thu, 25 Jan 2018 19:00:07 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer Message-ID: <5A6A99B7.1050703@oracle.com> Hi, Adding methods for comparing CharSequence, StringBuilder, and StringBuffer. The Comparable implementations for StringBuilder/Buffer are similar to that of String, allowing comparison operations between two StringBuilders/Buffers, e.g. aStringBuilder.compareTo(anotherStringBuilder). For CharSequence however, refer to the comments in JIRA, a static method 'compare' is added instead of implementing the Comparable interface. This 'compare' method may take CharSequence implementations such as String, StringBuilder and StringBuffer, making it possible to perform comparison among them. The previous example for example is equivalent to CharSequence.compare(aStringBuilder, anotherStringBuilder). Tests for java.base have been independent from each other. The new tests are therefore created to have no dependency on each other or sharing any code. JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ Thanks, Joe From mandy.chung at oracle.com Fri Jan 26 03:41:58 2018 From: mandy.chung at oracle.com (mandy chung) Date: Thu, 25 Jan 2018 19:41:58 -0800 Subject: [11] RFR JDK-8196127: Dead code in VersionProps.java.template In-Reply-To: <045bd1c6-e43a-17af-9c13-44a128c20c4f@oracle.com> References: <6d0f66f7-3133-e69f-9658-f2794c64481b@oracle.com> <045bd1c6-e43a-17af-9c13-44a128c20c4f@oracle.com> Message-ID: <9f596e80-cb2d-d159-16ae-54bb69bdbdc2@oracle.com> I like code deletion! Mandy On 1/25/18 6:24 PM, David Holmes wrote: > Wow that was fast! :) > > Looks good. > > Thanks, > David > > On 26/01/2018 5:33 AM, mandy chung wrote: >> Trivial fix to remove unused isHeadless variable. >> >> $ hg diff >> src/java.base/share/classes/java/lang/VersionProps.java.template >> diff --git >> a/src/java.base/share/classes/java/lang/VersionProps.java.template >> b/src/java.base/share/classes/java/lang/VersionProps.java.template >> --- a/src/java.base/share/classes/java/lang/VersionProps.java.template >> +++ b/src/java.base/share/classes/java/lang/VersionProps.java.template >> @@ -166,15 +166,8 @@ >> ?????? * Print version info. >> ?????? */ >> ????? private static void print(boolean err, boolean newln) { >> -??????? boolean isHeadless = false; >> ????????? PrintStream ps = err ? System.err : System.out; >> >> -??????? /* Report that we're running headless if the property is >> true */ >> -??????? String headless = System.getProperty("java.awt.headless"); >> -??????? if ( (headless != null) && >> (headless.equalsIgnoreCase("true")) ) { >> -??????????? isHeadless = true; >> -??????? } >> - >> ????????? /* First line: platform version. */ >> ????????? if (err) { >> ????????????? ps.println(launcher_name + " version \"" + java_version >> + "\"" >> >> Mandy From martinrb at google.com Fri Jan 26 06:20:52 2018 From: martinrb at google.com (Martin Buchholz) Date: Thu, 25 Jan 2018 22:20:52 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> Message-ID: Yes. public Object[] toArray() { checkForComodification(); return Arrays.copyOfRange(root.elementData, offset, offset + size); } @SuppressWarnings("unchecked") public T[] toArray(T[] a) { checkForComodification(); if (a.length < size) return (T[]) Arrays.copyOfRange( root.elementData, offset, offset + size, a.getClass()); System.arraycopy(root.elementData, offset, a, 0, size); if (a.length > size) a[size] = null; return a; } It's still possible to find simple performance improvements in classes like ArrayList. On Thu, Jan 25, 2018 at 4:41 PM, John Rose wrote: > On Jan 25, 2018, at 2:02 PM, ?????? ??????? > wrote: > > > > + return (T[]) Arrays.copyOfRange(root.elementData, > offset, size, a.getClass()); > > > Giving this a quick glance: > I think you may want s/size/offset+size/. > There should be calls to checkForComodification. > > From amy.lu at oracle.com Fri Jan 26 07:45:27 2018 From: amy.lu at oracle.com (Amy Lu) Date: Fri, 26 Jan 2018 15:45:27 +0800 Subject: [JDK 11] RFR 8196211: Move two sun/nio/cs tests into OpenJDK Message-ID: <74868834-8095-d530-ff16-66ec257f24ee@oracle.com> Please review the patch to move two sun/nio/cs tests into OpenJDK. bug: https://bugs.openjdk.java.net/browse/JDK-8196211 webrev: http://cr.openjdk.java.net/~amlu/8196211/webrev.00/ Thanks, Amy From Alan.Bateman at oracle.com Fri Jan 26 07:47:04 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Fri, 26 Jan 2018 07:47:04 +0000 Subject: RFR: 8186009 tools launcher test AddExportsAndOpensInManifest.java fails intermittently: AccessDeniedException In-Reply-To: References: Message-ID: <97940f65-b5b8-0cc9-3c95-b09df62be6fd@oracle.com> On 26/01/2018 00:19, Andrey Nazarov wrote: > Screenshot with new layout http://cr.openjdk.java.net/~anazarov/JDK-8186009/Screen%20Shot%202018-01-25%20at%2015.40.15.png > > CR: http://cr.openjdk.java.net/~anazarov/JDK-8186009/webrev.04/ > Looks okay to me (although I'm not a fan of using @BeforeMethod in this way). -Alan From sean.coffey at oracle.com Fri Jan 26 08:06:06 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Fri, 26 Jan 2018 08:06:06 +0000 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> Message-ID: I don't see them being related with data given below. Maybe there's a classpath configuration issue ? If you feel there's a bug in the JDK libraries, please report it via https://bugs.java.com/ regards, Sean. On 26/01/2018 00:23, Krystal Mok wrote: > Hi guys, > > A coworker of mine had hit this issue last night on 8u161 and it > worked fine on 8u151: > > ERROR: > /home/myuser/.cache/bazel/_bazel_myuser/some_hash_code/external/jackson_datatype_joda_shaded/BUILD:5:1: > Building > external/jackson_datatype_joda_shaded/libjackson-datatype-joda-class.jar > (35 source files) failed (Exit 1) > java.lang.InternalError: Cannot find requested resource bundle for > locale en_US > ? ? ? ? at > com.sun.tools.javac.util.JavacMessages.getBundles(JavacMessages.java:128) > ? ? ? ? at > com.sun.tools.javac.util.JavacMessages.getLocalizedString(JavacMessages.java:147) > ? ? ? ? at > com.sun.tools.javac.util.JavacMessages.getLocalizedString(JavacMessages.java:140) > ? ? ? ? at com.sun.tools.javac.util.Log.localize(Log.java:788) > ? ? ? ? at com.sun.tools.javac.util.Log.printLines(Log.java:586) > ? ? ? ? at > com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:170) > ? ? ? ? at > com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:96) > ? ? ? ? at > com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:90) > ? ? ? ? at > com.google.devtools.build.buildjar.javac.BlazeJavacMain.compile(BlazeJavacMain.java:105) > ? ? ? ? at > com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder$1.invokeJavac(SimpleJavaLibraryBuilder.java:106) > ? ? ? ? at > com.google.devtools.build.buildjar.ReducedClasspathJavaLibraryBuilder.compileSources(ReducedClasspathJavaLibraryBuilder.java:53) > ? ? ? ? at > com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.compileJavaLibrary(SimpleJavaLibraryBuilder.java:109) > ? ? ? ? at > com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:117) > ? ? ? ? at > com.google.devtools.build.buildjar.BazelJavaBuilder.processRequest(BazelJavaBuilder.java:105) > ? ? ? ? at > com.google.devtools.build.buildjar.BazelJavaBuilder.runPersistentWorker(BazelJavaBuilder.java:67) > ? ? ? ? at > com.google.devtools.build.buildjar.BazelJavaBuilder.main(BazelJavaBuilder.java:45) > Caused by: java.util.MissingResourceException: Can't find bundle for > base name com.google.errorprone.errors, locale en_US > ? ? ? ? at > java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1573) > ? ? ? ? at > java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1396) > ? ? ? ? at java.util.ResourceBundle.getBundle(ResourceBundle.java:854) > ? ? ? ? at > com.sun.tools.javac.util.JavacMessages.lambda$add$0(JavacMessages.java:106) > ? ? ? ? at > com.sun.tools.javac.util.JavacMessages.getBundles(JavacMessages.java:125) > ? ? ? ? ... 15 more > > Resource bundle loading issue...could that be related to this XML > issue here? > > Thanks, > Kris > > On Thu, Jan 25, 2018 at 8:41 AM, Se?n Coffey > wrote: > > On 25/01/2018 11:58, Bernd wrote: > > Hello, > > some of our unit tests (using PowerMock and xmlunit) fail with > 8u161 (and > u162) but work with 8u152. > > I cant reproduce the fault in a stand-alone program so it > seems to be > related to classloader/reflection magic of those tools, sorry. > > Is this a regression of 8159240 > > (not > public?) > > Fixes in the CPU releases are kept private - hence the above bug > is not public. The changesets do become public once the release is > made public though. See : > http://hg.openjdk.java.net/jdk8u/jdk8u/jaxws/rev/06086cb6c349 > > > I don't think it's a factor for what you're seeing. > > Classes nearer to those below were touched via JDK-8186080: > Transform XML interfaces > http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 > > http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 > > > This may be connected with some tools trying to redefine the > classes perhaps. Needs more investigating. Perhaps the > XMLSchemaLoader changes are a factor ? > > regards, > Sean. > > > Here is the stacktrace anyway: > > com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: > Schema > factory class > com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > does not > extend from SchemaDVFactory. > ? ? ?at > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.getInstance(SchemaDVFactory.java:75) > ? ? ?at > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.getInstance(SchemaDVFactory.java:57) > ? ? ?at > com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.reset(XMLSchemaLoader.java:1024) > ? ? ?at > com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.loadGrammar(XMLSchemaLoader.java:556) > ? ? ?at > com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.loadGrammar(XMLSchemaLoader.java:535) > ? ? ?at > com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchemaFactory.newSchema(XMLSchemaFactory.java:254) > ? ? ?at > javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.java:638) > ? ? ?at > javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.java:654) > ? ? ?at > com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchemaValidator(BuilderTestHelper.java:57) > ? ? ?at > com.seeburger.api.test.helpers.BuilderTestHelper.validateAndCompare(BuilderTestHelper.java:73) > ? ? ?at > com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuilderTest.java:196) > ? ? ?at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > ? ? ?at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > ? ? ?at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ? ? ?at java.lang.reflect.Method.invoke(Method.java:498) > ? ? ?at > org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:310) > ? ? ?at > org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.java:89) > ? ? ?at > org.junit.internal.runners.MethodRoadie.runBeforesThenTestThenAfters(MethodRoadie.java:97) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(PowerMockJUnit44RunnerDelegateImpl.java:294) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestInSuper(PowerMockJUnit47RunnerDelegateImpl.java:127) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit47RunnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(PowerMockJUnit47RunnerDelegateImpl.java:82) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThenTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) > ? ? ?at > org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie.java:87) > ? ? ?at > org.junit.internal.runners.MethodRoadie.run(MethodRoadie.java:50) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDelegateImpl.java:207) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateImpl.java:146) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) > ? ? ?at > org.junit.internal.runners.ClassRoadie.runUnprotected(ClassRoadie.java:34) > ? ? ?at > org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) > ? ? ?at > org.powermock.modules.junit4.internal.impl.PowerMockJUnit44RunnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) > ? ? ?at > org.powermock.modules.junit4.common.internal.impl.JUnit4TestSuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) > ? ? ?at > org.powermock.modules.junit4.common.internal.impl.AbstractCommonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) > ? ? ?at > org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRunner.java:59) > ? ? ?at > org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) > ? ? ?at > org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) > ? ? ?at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:539) > ? ? ?at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:761) > ? ? ?at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:461) > ? ? ?at > org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:207) > > on the classpath jaxb-impl-2.2.5.jar but the specific packages > are only > loaded from rt.jar and redefined. I asume the later is done by > Powermock. > > ? ? ?Line 611: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory > from C:\Program > Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > ? ? ?Line 616: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > from > C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > ? ? ?Line 617: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > from > C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] > ? ? ?Line 618: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from > __JVM_DefineClass__] > ? ? ?Line 619: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory > from > __JVM_DefineClass__] > ? ? ?Line 620: [Loaded > com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl > from > __JVM_DefineClass__] > > Is that something you are concerned? > > Gruss > Bernd > > > > > From rednaxelafx at gmail.com Fri Jan 26 08:14:05 2018 From: rednaxelafx at gmail.com (Krystal Mok) Date: Fri, 26 Jan 2018 00:14:05 -0800 Subject: Oracle Java 8u161 regression in XML Schema Factory In-Reply-To: References: <14b3a54d-dfd3-b9a8-7d66-ebfe7feee1a1@oracle.com> Message-ID: Thanks Sean. It's more likely a JDK issue because my coworker simply swapped the JAVA_HOME from 8u161 which failed with that error to 8u151 and it worked fine. I'll ask if he's got the cycles to distill a reproducer. If he does I'll open a JBS ticket. Thanks, Kris On Fri, Jan 26, 2018 at 12:06 AM, Se?n Coffey wrote: > I don't see them being related with data given below. Maybe there's a > classpath configuration issue ? > > If you feel there's a bug in the JDK libraries, please report it via > https://bugs.java.com/ > > regards, > Sean. > On 26/01/2018 00:23, Krystal Mok wrote: > > Hi guys, > > A coworker of mine had hit this issue last night on 8u161 and it worked > fine on 8u151: > > ERROR: /home/myuser/.cache/bazel/_bazel_myuser/some_hash_code/ > external/jackson_datatype_joda_shaded/BUILD:5:1: Building > external/jackson_datatype_joda_shaded/libjackson-datatype-joda-class.jar > (35 source files) failed (Exit 1) > java.lang.InternalError: Cannot find requested resource bundle for locale > en_US > at com.sun.tools.javac.util.JavacMessages.getBundles( > JavacMessages.java:128) > at com.sun.tools.javac.util.JavacMessages.getLocalizedString( > JavacMessages.java:147) > at com.sun.tools.javac.util.JavacMessages.getLocalizedString( > JavacMessages.java:140) > at com.sun.tools.javac.util.Log.localize(Log.java:788) > at com.sun.tools.javac.util.Log.printLines(Log.java:586) > at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions( > JavacTaskImpl.java:170) > at com.sun.tools.javac.api.JavacTaskImpl.doCall( > JavacTaskImpl.java:96) > at com.sun.tools.javac.api.JavacTaskImpl.call( > JavacTaskImpl.java:90) > at com.google.devtools.build.buildjar.javac.BlazeJavacMain. > compile(BlazeJavacMain.java:105) > at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder$1. > invokeJavac(SimpleJavaLibraryBuilder.java:106) > at com.google.devtools.build.buildjar. > ReducedClasspathJavaLibraryBuilder.compileSources( > ReducedClasspathJavaLibraryBuilder.java:53) > at com.google.devtools.build.buildjar.SimpleJavaLibraryBuilder. > compileJavaLibrary(SimpleJavaLibraryBuilder.java:109) > at com.google.devtools.build.buildjar. > SimpleJavaLibraryBuilder.run(SimpleJavaLibraryBuilder.java:117) > at com.google.devtools.build.buildjar.BazelJavaBuilder. > processRequest(BazelJavaBuilder.java:105) > at com.google.devtools.build.buildjar.BazelJavaBuilder. > runPersistentWorker(BazelJavaBuilder.java:67) > at com.google.devtools.build.buildjar.BazelJavaBuilder. > main(BazelJavaBuilder.java:45) > Caused by: java.util.MissingResourceException: Can't find bundle for base > name com.google.errorprone.errors, locale en_US > at java.util.ResourceBundle.throwMissingResourceException( > ResourceBundle.java:1573) > at java.util.ResourceBundle.getBundleImpl(ResourceBundle. > java:1396) > at java.util.ResourceBundle.getBundle(ResourceBundle.java:854) > at com.sun.tools.javac.util.JavacMessages.lambda$add$0( > JavacMessages.java:106) > at com.sun.tools.javac.util.JavacMessages.getBundles( > JavacMessages.java:125) > ... 15 more > > Resource bundle loading issue...could that be related to this XML issue > here? > > Thanks, > Kris > > On Thu, Jan 25, 2018 at 8:41 AM, Se?n Coffey > wrote: > >> On 25/01/2018 11:58, Bernd wrote: >> >> Hello, >>> >>> some of our unit tests (using PowerMock and xmlunit) fail with 8u161 (and >>> u162) but work with 8u152. >>> >>> I cant reproduce the fault in a stand-alone program so it seems to be >>> related to classloader/reflection magic of those tools, sorry. >>> >>> Is this a regression of 8159240 >>> (not public?) >>> >> Fixes in the CPU releases are kept private - hence the above bug is not >> public. The changesets do become public once the release is made public >> though. See : http://hg.openjdk.java.net/jdk >> 8u/jdk8u/jaxws/rev/06086cb6c349 >> >> I don't think it's a factor for what you're seeing. >> >> Classes nearer to those below were touched via JDK-8186080: Transform XML >> interfaces >> http://hg.openjdk.java.net/jdk8u/jdk8u/jdk/rev/cb84156d54b2 >> http://hg.openjdk.java.net/jdk8u/jdk8u/jaxp/rev/08a44c164993 >> >> This may be connected with some tools trying to redefine the classes >> perhaps. Needs more investigating. Perhaps the XMLSchemaLoader changes are >> a factor ? >> >> regards, >> Sean. >> >> >>> Here is the stacktrace anyway: >>> >>> com.sun.org.apache.xerces.internal.impl.dv.DVFactoryException: Schema >>> factory class >>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl does >>> not >>> extend from SchemaDVFactory. >>> at >>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.g >>> etInstance(SchemaDVFactory.java:75) >>> at >>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory.g >>> etInstance(SchemaDVFactory.java:57) >>> at >>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.r >>> eset(XMLSchemaLoader.java:1024) >>> at >>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.l >>> oadGrammar(XMLSchemaLoader.java:556) >>> at >>> com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaLoader.l >>> oadGrammar(XMLSchemaLoader.java:535) >>> at >>> com.sun.org.apache.xerces.internal.jaxp.validation.XMLSchema >>> Factory.newSchema(XMLSchemaFactory.java:254) >>> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.j >>> ava:638) >>> at javax.xml.validation.SchemaFactory.newSchema(SchemaFactory.j >>> ava:654) >>> at >>> com.seeburger.api.test.helpers.BuilderTestHelper.getCRSchema >>> Validator(BuilderTestHelper.java:57) >>> at >>> com.seeburger.api.test.helpers.BuilderTestHelper.validateAnd >>> Compare(BuilderTestHelper.java:73) >>> at >>> com.seeburger.api.test.KSMBuilderTest.testDeletePGP(KSMBuild >>> erTest.java:196) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >>> ssorImpl.java:62) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >>> thodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at org.junit.internal.runners.TestMethod.invoke(TestMethod.java:68) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runTestMethod >>> (PowerMockJUnit44RunnerDelegateImpl.java:310) >>> at org.junit.internal.runners.MethodRoadie$2.run(MethodRoadie.j >>> ava:89) >>> at >>> org.junit.internal.runners.MethodRoadie.runBeforesThenTestTh >>> enAfters(MethodRoadie.java:97) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>> unnerDelegateImpl$PowerMockJUnit44MethodRunner.executeTest(P >>> owerMockJUnit44RunnerDelegateImpl.java:294) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >>> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTestIn >>> Super(PowerMockJUnit47RunnerDelegateImpl.java:127) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit47R >>> unnerDelegateImpl$PowerMockJUnit47MethodRunner.executeTest(P >>> owerMockJUnit47RunnerDelegateImpl.java:82) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>> unnerDelegateImpl$PowerMockJUnit44MethodRunner.runBeforesThe >>> nTestThenAfters(PowerMockJUnit44RunnerDelegateImpl.java:282) >>> at org.junit.internal.runners.MethodRoadie.runTest(MethodRoadie >>> .java:87) >>> at org.junit.internal.runners.MethodRoadie.run(MethodRoadie.jav >>> a:50) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>> unnerDelegateImpl.invokeTestMethod(PowerMockJUnit44RunnerDel >>> egateImpl.java:207) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>> unnerDelegateImpl.runMethods(PowerMockJUnit44RunnerDelegateI >>> mpl.java:146) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>> unnerDelegateImpl$1.run(PowerMockJUnit44RunnerDelegateImpl.java:120) >>> at >>> org.junit.internal.runners.ClassRoadie.runUnprotected(ClassR >>> oadie.java:34) >>> at >>> org.junit.internal.runners.ClassRoadie.runProtected(ClassRoadie.java:44) >>> at >>> org.powermock.modules.junit4.internal.impl.PowerMockJUnit44R >>> unnerDelegateImpl.run(PowerMockJUnit44RunnerDelegateImpl.java:122) >>> at >>> org.powermock.modules.junit4.common.internal.impl.JUnit4Test >>> SuiteChunkerImpl.run(JUnit4TestSuiteChunkerImpl.java:106) >>> at >>> org.powermock.modules.junit4.common.internal.impl.AbstractCo >>> mmonPowerMockRunner.run(AbstractCommonPowerMockRunner.java:53) >>> at >>> org.powermock.modules.junit4.PowerMockRunner.run(PowerMockRu >>> nner.java:59) >>> at >>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.r >>> un(JUnit4TestReference.java:86) >>> at >>> org.eclipse.jdt.internal.junit.runner.TestExecution.run(Test >>> Execution.java:38) >>> at >>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>> sts(RemoteTestRunner.java:539) >>> at >>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>> sts(RemoteTestRunner.java:761) >>> at >>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(R >>> emoteTestRunner.java:461) >>> at >>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( >>> RemoteTestRunner.java:207) >>> >>> on the classpath jaxb-impl-2.2.5.jar but the specific packages are only >>> loaded from rt.jar and redefined. I asume the later is done by Powermock. >>> >>> Line 611: [Loaded >>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >>> C:\Program >>> Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >>> Line 616: [Loaded >>> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from >>> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >>> Line 617: [Loaded >>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from >>> C:\Program Files\Java\jdk1.8.0_161\jre\lib\rt.jar] >>> Line 618: [Loaded >>> com.sun.org.apache.xerces.internal.impl.dv.SchemaDVFactory from >>> __JVM_DefineClass__] >>> Line 619: [Loaded >>> com.sun.org.apache.xerces.internal.impl.dv.xs.BaseSchemaDVFactory from >>> __JVM_DefineClass__] >>> Line 620: [Loaded >>> com.sun.org.apache.xerces.internal.impl.dv.xs.SchemaDVFactoryImpl from >>> __JVM_DefineClass__] >>> >>> Is that something you are concerned? >>> >>> Gruss >>> Bernd >>> >>> >> >> > > From sergei.tsypanov at yandex.ru Fri Jan 26 09:12:11 2018 From: sergei.tsypanov at yandex.ru (=?utf-8?B?0KHQtdGA0LPQtdC5INCm0YvQv9Cw0L3QvtCy?=) Date: Fri, 26 Jan 2018 11:12:11 +0200 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> Message-ID: <733691516957931@web38o.yandex.ru> Thanks for quick review! Indeed, there must be checkForComodification() at the beginning of each method, my bad. Btw, ArrayList produced by java.util.Arrays.asList() is also RandomAccess and array-based but has no custom implementation of subList(). Could we apply the same approach for java.util.Arrays.ArrayList? Regards, Sergey From claes.redestad at oracle.com Fri Jan 26 13:31:31 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Fri, 26 Jan 2018 14:31:31 +0100 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> Message-ID: On 2018-01-26 07:20, Martin Buchholz wrote: > It's still possible to find simple performance improvements in classes like > ArrayList. Indeed! It seems ArrayList.SubList delegates quite a few of its implementations to AbstractList and AbstractCollection, which uses iterators for things that could be done more efficiently with conventional for-loops. This includes contains, equals, hashCode, indexOf, lastIndexOf... I saw up to 7x improvements on some of these over the inherited implementations when providing specialized ones for ImmutableList.SubList as part of JDK-8193128 I'll file a bug and maybe even tinker with it when I find some time... /Claes From jason_mehrens at hotmail.com Fri Jan 26 14:23:38 2018 From: jason_mehrens at hotmail.com (Jason Mehrens) Date: Fri, 26 Jan 2018 14:23:38 +0000 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A6A99B7.1050703@oracle.com> References: <5A6A99B7.1050703@oracle.com> Message-ID: Joe, 1. Seems odd that StringBuilder and StringBuffer are comparable but don't have an equals/hashcode implementation. Seems like if you are to stick with that you'll have to add the standard "natural ordering is inconsistent with equals" verbiage. 2. For StringBuffer compare, won't you have to either lock the 'another' object too so it is not mutated or perform some sort of copy before compare? Neither sound like a good option. 3. Does CharSequence.compare need to specify that IndexOutOfBoundsException may be thrown under concurrent modification? 4. For CharSequence.compare, instead of creating a special case for string would it make sense to do: ==== if (cs1.getClass() == cs2.getClass() && cs1 instanceof Comparable) { return ((Comparable) cs1).compareTo(cs2); } ==== Given #1 and #2 maybe StringBuilder and StringBuffer shouldn't implement comparable and just rely on users either calling 'sb1.toString().compare(sb2.toString())' or 'CharSequence.compare(sb1, sb2)'. Jason ________________________________________ From: core-libs-dev on behalf of Joe Wang Sent: Thursday, January 25, 2018 9:00 PM To: core-libs-dev Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer Hi, Adding methods for comparing CharSequence, StringBuilder, and StringBuffer. The Comparable implementations for StringBuilder/Buffer are similar to that of String, allowing comparison operations between two StringBuilders/Buffers, e.g. aStringBuilder.compareTo(anotherStringBuilder). For CharSequence however, refer to the comments in JIRA, a static method 'compare' is added instead of implementing the Comparable interface. This 'compare' method may take CharSequence implementations such as String, StringBuilder and StringBuffer, making it possible to perform comparison among them. The previous example for example is equivalent to CharSequence.compare(aStringBuilder, anotherStringBuilder). Tests for java.base have been independent from each other. The new tests are therefore created to have no dependency on each other or sharing any code. JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ Thanks, Joe From scolebourne at joda.org Fri Jan 26 15:01:56 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Fri, 26 Jan 2018 15:01:56 +0000 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A6A99B7.1050703@oracle.com> References: <5A6A99B7.1050703@oracle.com> Message-ID: Just to note that JSR-310 had to add a CharSequence comparison method. Not the same as this one, but a data point, and perhaps a target to become public in a future webrev. See DateTimeParseContext.subSequenceEquals(). Stephen On 26 January 2018 at 03:00, Joe Wang wrote: > Hi, > > Adding methods for comparing CharSequence, StringBuilder, and StringBuffer. > > The Comparable implementations for StringBuilder/Buffer are similar to that > of String, allowing comparison operations between two > StringBuilders/Buffers, e.g. aStringBuilder.compareTo(anotherStringBuilder). > For CharSequence however, refer to the comments in JIRA, a static method > 'compare' is added instead of implementing the Comparable interface. This > 'compare' method may take CharSequence implementations such as String, > StringBuilder and StringBuffer, making it possible to perform comparison > among them. The previous example for example is equivalent to > CharSequence.compare(aStringBuilder, anotherStringBuilder). > > Tests for java.base have been independent from each other. The new tests are > therefore created to have no dependency on each other or sharing any code. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe From rob.mckenna at oracle.com Fri Jan 26 16:40:23 2018 From: rob.mckenna at oracle.com (Rob McKenna) Date: Fri, 26 Jan 2018 16:40:23 +0000 Subject: [8u-dev] RFR - 8156824: com.sun.jndi.ldap.pool.PoolCleaner should clear its context class loader Message-ID: <20180126164023.GA6402@vimes> Hi folks, I'd like to backport this change to 8u-dev. The changes are straightforward enough but 8u needs some changes that were made to InnocuousThread. (strictly it doesn't need all of the changes I've made but I've made an effort to make the code look as close to 9 as possible) bug: https://bugs.openjdk.java.net/browse/JDK-8156824 9u changeset: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/22e704dfa05c 8u webrev: http://cr.openjdk.java.net/~robm/8156824/webrev.01/ -Rob From martinrb at google.com Fri Jan 26 17:12:43 2018 From: martinrb at google.com (Martin Buchholz) Date: Fri, 26 Jan 2018 09:12:43 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> Message-ID: It may be that array-backed lists inheriting from AbstractList is not useful except when creating a prototype (or as a way of testing AbstractList!) because a high-quality implementation should always override. On Fri, Jan 26, 2018 at 5:31 AM, Claes Redestad wrote: > On 2018-01-26 07:20, Martin Buchholz wrote: > >> It's still possible to find simple performance improvements in classes >> like >> ArrayList. >> > > Indeed! > > It seems ArrayList.SubList delegates quite a few of its implementations > to AbstractList and AbstractCollection, which uses iterators for things > that could be done more efficiently with conventional for-loops. > > This includes contains, equals, hashCode, indexOf, lastIndexOf... I saw > up to 7x improvements on some of these over the inherited > implementations when providing specialized ones for ImmutableList.SubList > as part of JDK-8193128 > > I'll file a bug and maybe even tinker with it when I find some time... > > /Claes > From paul.sandoz at oracle.com Fri Jan 26 18:32:30 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 26 Jan 2018 10:32:30 -0800 Subject: [JDK 11] RFR 8196211: Move two sun/nio/cs tests into OpenJDK In-Reply-To: <74868834-8095-d530-ff16-66ec257f24ee@oracle.com> References: <74868834-8095-d530-ff16-66ec257f24ee@oracle.com> Message-ID: <4BF3C2BF-1D61-440F-BFC3-18B2A47234DE@oracle.com> Hi, Quick observation. EUCTWBufferBoundaryDecodeTest uses a different data file on windows and it uses the line separator as a trigger. Is it possible to better formalize this by passing in the argument for the file via jtreg? Paul. > On Jan 25, 2018, at 11:45 PM, Amy Lu wrote: > > Please review the patch to move two sun/nio/cs tests into OpenJDK. > > bug: https://bugs.openjdk.java.net/browse/JDK-8196211 > webrev: http://cr.openjdk.java.net/~amlu/8196211/webrev.00/ > > Thanks, > Amy From huizhe.wang at oracle.com Fri Jan 26 21:40:52 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 26 Jan 2018 13:40:52 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> Message-ID: <970fc635-7f79-baca-ecad-f7b16d133032@oracle.com> Thanks for the comments, Jason. The topic about equals/hashcode had been discussed. They are currently inherited, and by definition of CharSequence, equality is generally undefined. One could argue for it to be a lexicographic comparison, but that means changing the current specification, potentially breaking compatibility. Note that we made subtle wording change to the general description of CharSequence. For #2, I'll add a clarification in the spec similar to those that deal with appending a CharSequence or StringBuffer, that the method does not synchronize on the source.? A better concurrent lock mechanism could have been devised for StringBuffer, but it'd be a fruitless effort to attempt to change its poor state. Given the nature of these classes, in almost all cases, StringBuilder is recommended over StringBuffer. So our effort here is to maintain a consistency between the classes. #3, Concurrency would be better handled outside of the compare method (same as the above). #4, sounds good. I'll look into the modification and probably add some more test cases. Best, Joe On 1/26/2018 6:23 AM, Jason Mehrens wrote: > Joe, > > 1. Seems odd that StringBuilder and StringBuffer are comparable but don't have an equals/hashcode implementation. > Seems like if you are to stick with that you'll have to add the standard "natural ordering is inconsistent with equals" verbiage. > 2. For StringBuffer compare, won't you have to either lock the 'another' object too so it is not mutated or perform some > sort of copy before compare? Neither sound like a good option. > 3. Does CharSequence.compare need to specify that IndexOutOfBoundsException may be thrown under concurrent modification? > 4. For CharSequence.compare, instead of creating a special case for string would it make sense to do: > > ==== > if (cs1.getClass() == cs2.getClass() && cs1 instanceof Comparable) { > return ((Comparable) cs1).compareTo(cs2); > } > ==== > > Given #1 and #2 maybe StringBuilder and StringBuffer shouldn't implement comparable and just rely on users either calling > 'sb1.toString().compare(sb2.toString())' or 'CharSequence.compare(sb1, sb2)'. > > Jason > ________________________________________ > From: core-libs-dev on behalf of Joe Wang > Sent: Thursday, January 25, 2018 9:00 PM > To: core-libs-dev > Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer > > Hi, > > Adding methods for comparing CharSequence, StringBuilder, and StringBuffer. > > The Comparable implementations for StringBuilder/Buffer are similar to > that of String, allowing comparison operations between two > StringBuilders/Buffers, e.g. > aStringBuilder.compareTo(anotherStringBuilder). For CharSequence > however, refer to the comments in JIRA, a static method 'compare' is > added instead of implementing the Comparable interface. This 'compare' > method may take CharSequence implementations such as String, > StringBuilder and StringBuffer, making it possible to perform comparison > among them. The previous example for example is equivalent to > CharSequence.compare(aStringBuilder, anotherStringBuilder). > > Tests for java.base have been independent from each other. The new tests > are therefore created to have no dependency on each other or sharing any > code. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe From huizhe.wang at oracle.com Fri Jan 26 21:48:30 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 26 Jan 2018 13:48:30 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: References: <5A6A99B7.1050703@oracle.com> Message-ID: <32d3e7be-e8b4-6570-f41f-fcb801f0629e@oracle.com> Thanks Stephen for the note. I downloaded the api [1], but don't seem to see a class "DateTimeParseContext". Do you have a pointer to the current webrev? What does it do, and what would be the implication with regards to the CharSequence compare method? [1] https://jcp.org/en/jsr/detail?id=310 Best, Joe On 1/26/2018 7:01 AM, Stephen Colebourne wrote: > Just to note that JSR-310 had to add a CharSequence comparison method. > Not the same as this one, but a data point, and perhaps a target to > become public in a future webrev. See > DateTimeParseContext.subSequenceEquals(). > > Stephen > > > On 26 January 2018 at 03:00, Joe Wang wrote: >> Hi, >> >> Adding methods for comparing CharSequence, StringBuilder, and StringBuffer. >> >> The Comparable implementations for StringBuilder/Buffer are similar to that >> of String, allowing comparison operations between two >> StringBuilders/Buffers, e.g. aStringBuilder.compareTo(anotherStringBuilder). >> For CharSequence however, refer to the comments in JIRA, a static method >> 'compare' is added instead of implementing the Comparable interface. This >> 'compare' method may take CharSequence implementations such as String, >> StringBuilder and StringBuffer, making it possible to perform comparison >> among them. The previous example for example is equivalent to >> CharSequence.compare(aStringBuilder, anotherStringBuilder). >> >> Tests for java.base have been independent from each other. The new tests are >> therefore created to have no dependency on each other or sharing any code. >> >> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >> >> Thanks, >> Joe From xueming.shen at oracle.com Fri Jan 26 22:12:57 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 26 Jan 2018 14:12:57 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <32d3e7be-e8b4-6570-f41f-fcb801f0629e@oracle.com> References: <5A6A99B7.1050703@oracle.com> <32d3e7be-e8b4-6570-f41f-fcb801f0629e@oracle.com> Message-ID: <5A6BA7E9.7090000@oracle.com> java.time.format.DateTimeParseContext.subSequenceEquals On 1/26/18, 1:48 PM, huizhe wang wrote: > Thanks Stephen for the note. > > I downloaded the api [1], but don't seem to see a class > "DateTimeParseContext". Do you have a pointer to the current webrev? > What does it do, and what would be the implication with regards to the > CharSequence compare method? > > [1] https://jcp.org/en/jsr/detail?id=310 > > Best, > Joe > > On 1/26/2018 7:01 AM, Stephen Colebourne wrote: >> Just to note that JSR-310 had to add a CharSequence comparison method. >> Not the same as this one, but a data point, and perhaps a target to >> become public in a future webrev. See >> DateTimeParseContext.subSequenceEquals(). >> >> Stephen >> >> >> On 26 January 2018 at 03:00, Joe Wang wrote: >>> Hi, >>> >>> Adding methods for comparing CharSequence, StringBuilder, and >>> StringBuffer. >>> >>> The Comparable implementations for StringBuilder/Buffer are similar >>> to that >>> of String, allowing comparison operations between two >>> StringBuilders/Buffers, e.g. >>> aStringBuilder.compareTo(anotherStringBuilder). >>> For CharSequence however, refer to the comments in JIRA, a static >>> method >>> 'compare' is added instead of implementing the Comparable interface. >>> This >>> 'compare' method may take CharSequence implementations such as String, >>> StringBuilder and StringBuffer, making it possible to perform >>> comparison >>> among them. The previous example for example is equivalent to >>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>> >>> Tests for java.base have been independent from each other. The new >>> tests are >>> therefore created to have no dependency on each other or sharing any >>> code. >>> >>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>> >>> Thanks, >>> Joe > From huizhe.wang at oracle.com Fri Jan 26 22:52:02 2018 From: huizhe.wang at oracle.com (huizhe wang) Date: Fri, 26 Jan 2018 14:52:02 -0800 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A6BA7E9.7090000@oracle.com> References: <5A6A99B7.1050703@oracle.com> <32d3e7be-e8b4-6570-f41f-fcb801f0629e@oracle.com> <5A6BA7E9.7090000@oracle.com> Message-ID: Thanks Sherman.? Found it now.? Interesting method. What was the motivation? Does any class implement CharSequence in 310? Best, Joe On 1/26/2018 2:12 PM, Xueming Shen wrote: > > java.time.format.DateTimeParseContext.subSequenceEquals > > On 1/26/18, 1:48 PM, huizhe wang wrote: >> Thanks Stephen for the note. >> >> I downloaded the api [1], but don't seem to see a class >> "DateTimeParseContext". Do you have a pointer to the current webrev? >> What does it do, and what would be the implication with regards to >> the CharSequence compare method? >> >> [1] https://jcp.org/en/jsr/detail?id=310 >> >> Best, >> Joe >> >> On 1/26/2018 7:01 AM, Stephen Colebourne wrote: >>> Just to note that JSR-310 had to add a CharSequence comparison method. >>> Not the same as this one, but a data point, and perhaps a target to >>> become public in a future webrev. See >>> DateTimeParseContext.subSequenceEquals(). >>> >>> Stephen >>> >>> >>> On 26 January 2018 at 03:00, Joe Wang wrote: >>>> Hi, >>>> >>>> Adding methods for comparing CharSequence, StringBuilder, and >>>> StringBuffer. >>>> >>>> The Comparable implementations for StringBuilder/Buffer are similar >>>> to that >>>> of String, allowing comparison operations between two >>>> StringBuilders/Buffers, e.g. >>>> aStringBuilder.compareTo(anotherStringBuilder). >>>> For CharSequence however, refer to the comments in JIRA, a static >>>> method >>>> 'compare' is added instead of implementing the Comparable >>>> interface. This >>>> 'compare' method may take CharSequence implementations such as String, >>>> StringBuilder and StringBuffer, making it possible to perform >>>> comparison >>>> among them. The previous example for example is equivalent to >>>> CharSequence.compare(aStringBuilder, anotherStringBuilder). >>>> >>>> Tests for java.base have been independent from each other. The new >>>> tests are >>>> therefore created to have no dependency on each other or sharing >>>> any code. >>>> >>>> JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 >>>> webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ >>>> >>>> Thanks, >>>> Joe >> > From andrey.x.nazarov at oracle.com Fri Jan 26 23:17:36 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Fri, 26 Jan 2018 15:17:36 -0800 Subject: RFR: 8179294 several langtools tests depend on jar but do not declare that dependency Message-ID: <1757A99A-524F-4D1F-B04B-7003446E4FDF@oracle.com> Hi, Please review fix which adds absent module dependencies in Pack200 and Jlink tests. http://cr.openjdk.java.net/~anazarov/JDK-8179294/webrev.00 https://bugs.openjdk.java.net/browse/JDK-8179294 ?Thanks, Andrei From xueming.shen at oracle.com Fri Jan 26 23:32:14 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Fri, 26 Jan 2018 15:32:14 -0800 Subject: RFR JDK-8176379: java.util.Base64 mime encoder behaves incorrectly if initialized with a line length of size 1-3 Message-ID: <5A6BBA7E.3090001@oracle.com> Hi, Please help review the change for JDK-8176379 issue: https://bugs.openjdk.java.net/browse/JDK-8176379 webrev: http://cr.openjdk.java.net/~sherman/8176379/webrev It appears the spec is somewhat misleading and/or incorrect in this corner case. And the implementation of the mime type encoder OutputStream does not expect a "length==0" mime encoder. The difference behavior of the non-stream-type encoder and stream-type encoder comes from in non-stream-type case, we always have a " > 0" guard as if (linemax > 0 && slen > linemax / 4 * 3) slen = linemax / 4 * 3 but in stream-type we don't. private void checkNewline() throws IOException { if (linepos == linemax) { out.write(newline); linepos = 0; } } The proposed change here is to clarify the spec and the implementation to use a non-mime type encoder (no line separator) when the rounded down mime line length is smaller than 4. We probably need a CSR for this spec clarification, if the proposed change is approved. Thanks, Sherman From andrey.x.nazarov at oracle.com Fri Jan 26 23:25:48 2018 From: andrey.x.nazarov at oracle.com (Andrey Nazarov) Date: Fri, 26 Jan 2018 15:25:48 -0800 Subject: RFR: 8179294 several langtools tests depend on jar but do not declare that dependency In-Reply-To: <1757A99A-524F-4D1F-B04B-7003446E4FDF@oracle.com> References: <1757A99A-524F-4D1F-B04B-7003446E4FDF@oracle.com> Message-ID: <344FA5EA-180D-452A-8C0D-FAFB02D75C60@oracle.com> Added test from duplicated issue http://cr.openjdk.java.net/~anazarov/JDK-8179294/webrev.01/ ?Andrei > On 26 Jan 2018, at 15:17, Andrey Nazarov wrote: > > Hi, > > Please review fix which adds absent module dependencies in Pack200 and Jlink tests. > > http://cr.openjdk.java.net/~anazarov/JDK-8179294/webrev.00 > https://bugs.openjdk.java.net/browse/JDK-8179294 > > > ?Thanks, > Andrei From igor.ignatyev at oracle.com Fri Jan 26 23:40:56 2018 From: igor.ignatyev at oracle.com (Igor Ignatyev) Date: Fri, 26 Jan 2018 15:40:56 -0800 Subject: RFR: 8179294 several langtools tests depend on jar but do not declare that dependency In-Reply-To: <344FA5EA-180D-452A-8C0D-FAFB02D75C60@oracle.com> References: <1757A99A-524F-4D1F-B04B-7003446E4FDF@oracle.com> <344FA5EA-180D-452A-8C0D-FAFB02D75C60@oracle.com> Message-ID: <1AE00455-5F5A-49DC-BDC8-BA984A829B8F@oracle.com> the fix looks good to me. -- Igor > On Jan 26, 2018, at 3:25 PM, Andrey Nazarov wrote: > > Added test from duplicated issue http://cr.openjdk.java.net/~anazarov/JDK-8179294/webrev.01/ > > ?Andrei > >> On 26 Jan 2018, at 15:17, Andrey Nazarov wrote: >> >> Hi, >> >> Please review fix which adds absent module dependencies in Pack200 and Jlink tests. >> >> http://cr.openjdk.java.net/~anazarov/JDK-8179294/webrev.00 >> https://bugs.openjdk.java.net/browse/JDK-8179294 >> >> >> ?Thanks, >> Andrei > From paul.sandoz at oracle.com Fri Jan 26 23:53:26 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Fri, 26 Jan 2018 15:53:26 -0800 Subject: RFR JDK-8176379: java.util.Base64 mime encoder behaves incorrectly if initialized with a line length of size 1-3 In-Reply-To: <5A6BBA7E.3090001@oracle.com> References: <5A6BBA7E.3090001@oracle.com> Message-ID: <6DB12EB2-4A10-4091-A811-FCEA8B2B67BB@oracle.com> Base64 ? 138 lineLength = lineLength >> 2 << 2; I know the code was shuffled up but i find the following clearer in terms of bit manipulation: // Round down to nearest mulitple of 4 lineLength &= ~0b11 TestBase64 ? Is there a test passing in a negative value for len? Thanks, Paul. > On Jan 26, 2018, at 3:32 PM, Xueming Shen wrote: > > Hi, > > Please help review the change for JDK-8176379 > > issue: https://bugs.openjdk.java.net/browse/JDK-8176379 > webrev: http://cr.openjdk.java.net/~sherman/8176379/webrev > > It appears the spec is somewhat misleading and/or incorrect in this corner > case. And the implementation of the mime type encoder OutputStream does > not expect a "length==0" mime encoder. The difference behavior of the > non-stream-type encoder and stream-type encoder comes from > > in non-stream-type case, we always have a " > 0" guard as > > if (linemax > 0 && slen > linemax / 4 * 3) > slen = linemax / 4 * 3 > > but in stream-type we don't. > > private void checkNewline() throws IOException { > if (linepos == linemax) { > out.write(newline); > linepos = 0; > } > } > > The proposed change here is to clarify the spec and the implementation > to use a non-mime type encoder (no line separator) when the rounded down > mime line length is smaller than 4. > > We probably need a CSR for this spec clarification, if the proposed change is > approved. > > Thanks, > Sherman From patrick at reini.net Sat Jan 27 08:45:30 2018 From: patrick at reini.net (Patrick Reinhart) Date: Sat, 27 Jan 2018 09:45:30 +0100 Subject: JDK-8196298: Add null Reader and Writer Message-ID: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> Hi there, I saw that great extension on InputStream/OutputStream adding a Null implementation there. Based on those I would propose a similar addition to the Reader/Writer. -Patrick Here is what the API would look like based on the existing from InputStream/OutputStream: ??? /** ???? * Returns a new {@code Reader} that contains no characters. The returned ???? * stream is initially open.? The stream is closed by calling the ???? * {@code close()} method.? Subsequent calls to {@code close()} have no ???? * effect. ???? * ???? *

While the stream is open, the {@code ready?()}, {@code read()}, ???? * {@code read(char[])}, {@code read(char[], int, int)}, {@code skip()}, and ???? * {@code transferTo()} methods all behave as if end of stream has been ???? * reached.? After the stream has been closed, these methods all throw ???? * {@code IOException}. ???? * ???? *

The {@code markSupported()} method returns {@code false}.? The ???? * {@code mark()} method does nothing, and the {@code reset()} method ???? * throws {@code IOException}. ???? * ???? * @return an {@code Reader} which contains no characters ???? * ???? * @since 11 ???? */ ??? public static Reader nullReader() {} ??? /** ???? * Returns a new {@code Writer} which discards all characters.? The ???? * returned stream is initially open.? The stream is closed by calling ???? * the {@code close()} method.? Subsequent calls to {@code close()} have ???? * no effect. ???? * ???? *

While the stream is open, the {@code write(int)}, {@code ???? * write(char[])}, and {@code write(char[], int, int)} methods do nothing. ???? * After the stream has been closed, these methods all throw {@code ???? * IOException}. ???? * ???? *

The {@code flush()} method does nothing. ???? * ???? * @return an {@code Writer} which discards all characters ???? * ???? * @since 11 ???? */ ??? public static Writer nullWriter() {} From jon.chambers at gmail.com Sat Jan 27 16:05:09 2018 From: jon.chambers at gmail.com (Jon Chambers) Date: Sat, 27 Jan 2018 11:05:09 -0500 Subject: Optimizing UUID#fromString(String) Message-ID: Hello! I've recently had reason to take a look at performance around parsing and stringifying UUIDs. In exploring the space, I identified some opportunities to optimize the implementation of UUID#fromString as it currently exists ( http://hg.openjdk.java.net/jdk/jdk/file/fd237da7a113/src/java.base/share/classes/java/util/UUID.java#l196 ). Because UUID strings are of a known structure and length (32 hexadecimal digits and four dashes) and because UUIDs are exactly 128 bits in length, we know exactly how each character in a UUID string maps to bits in the parsed UUID. We always know, for example, that the first character in a UUID string maps to the four highest bits in the UUID, the second character maps to the four bits below that, and so on. With that knowledge, we can cut out a lot of the generality and bounds-checking we'd normally expect of a string-to-number parser. I've built an implementation with that in mind: https://github.com/jchambers/fast-uuid/blob/master/src/main/java/com/eatthepath/uuid/FastUUID.java#L108. In benchmarks ( https://github.com/jchambers/fast-uuid/blob/master/benchmark/src/main/java/com/eatthepath/UUIDBenchmark.java#L55-L63), this implementation is about six times faster than the current JDK implementation (9.0.4+11) and 14 times faster than the implementation in 1.8. The experimental implementation is more strict about UUID format (the current JDK implementation allows for variable-length blocks of hex digits between dashes while the experimental one doesn't), and I'll defer to you folks as to whether its handling of technically-malformed UUID strings is acceptable. As discussed via Twitter ( https://twitter.com/cl4es/status/956308599277486080), we might consider using the fixed-length parsing approach if we know the UUID string is exactly 36 characters long and fall back to the looser parser otherwise. I also recognize that this is partially reinventing the wheel when it comes to parsing hex strings, and the tradeoff between consistency and performance is certainly worthy of consideration. Regardless, I wanted to call this optimization opportunity to your attention, and would be happy to offer a proper patch if this seems like a worthwhile change. Cheers, and thank you for your consideration! -Jon From ivan.gerasimov at oracle.com Sat Jan 27 19:37:29 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sat, 27 Jan 2018 11:37:29 -0800 Subject: RFR JDK-8194412 : Adding 256 units of IsoFields.QUARTER_YEARS broken Message-ID: <4e596b86-912c-ce3c-d6c9-e509826c16ff@oracle.com> Hello! It was spotted that adding more then 255 quarter-years to a Temporal leads to a wrong result. BUGURL: https://bugs.openjdk.java.net/browse/JDK-8194412 WEBREV: http://cr.openjdk.java.net/~igerasim/8194412/00/webrev/ Would you please take a look at the fix and see that I'm not missing something? Thanks in advance! -- With kind regards, Ivan Gerasimov From Roger.Riggs at Oracle.com Sat Jan 27 20:24:27 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Sat, 27 Jan 2018 15:24:27 -0500 Subject: Optimizing UUID#fromString(String) In-Reply-To: References: Message-ID: <0859d368-d348-a01f-4097-1890cdd11065@Oracle.com> Hi Jon, Promising based on the cited performance improvements. Can you review a similar thread and improvements from 2013 to see if there are secondary considerations that have already been raised. http://mail.openjdk.java.net/pipermail/core-libs-dev/2013-January/013494.html Please include the patch in the body of the email or attach it to the email (with a .txt extension) to meet OpenJDK IP requirements. Compatibility requirements would rule out tightening the acceptable format so a fallback to a fully compatible parse would be needed. Thanks, Roger On 1/27/2018 11:05 AM, Jon Chambers wrote: > Hello! > > I've recently had reason to take a look at performance around parsing and > stringifying UUIDs. In exploring the space, I identified some opportunities > to optimize the implementation of UUID#fromString as it currently exists ( > http://hg.openjdk.java.net/jdk/jdk/file/fd237da7a113/src/java.base/share/classes/java/util/UUID.java#l196 > ). > > Because UUID strings are of a known structure and length (32 hexadecimal > digits and four dashes) and because UUIDs are exactly 128 bits in length, > we know exactly how each character in a UUID string maps to bits in the > parsed UUID. We always know, for example, that the first character in a > UUID string maps to the four highest bits in the UUID, the second character > maps to the four bits below that, and so on. > > With that knowledge, we can cut out a lot of the generality and > bounds-checking we'd normally expect of a string-to-number parser. I've > built an implementation with that in mind: > https://github.com/jchambers/fast-uuid/blob/master/src/main/java/com/eatthepath/uuid/FastUUID.java#L108. > In benchmarks ( > https://github.com/jchambers/fast-uuid/blob/master/benchmark/src/main/java/com/eatthepath/UUIDBenchmark.java#L55-L63), > this implementation is about six times faster than the current JDK > implementation (9.0.4+11) and 14 times faster than the implementation in > 1.8. > > The experimental implementation is more strict about UUID format (the > current JDK implementation allows for variable-length blocks of hex digits > between dashes while the experimental one doesn't), and I'll defer to you > folks as to whether its handling of technically-malformed UUID strings is > acceptable. As discussed via Twitter ( > https://twitter.com/cl4es/status/956308599277486080), we might consider > using the fixed-length parsing approach if we know the UUID string is > exactly 36 characters long and fall back to the looser parser otherwise. I > also recognize that this is partially reinventing the wheel when it comes > to parsing hex strings, and the tradeoff between consistency and > performance is certainly worthy of consideration. > > Regardless, I wanted to call this optimization opportunity to your > attention, and would be happy to offer a proper patch if this seems like a > worthwhile change. > > Cheers, and thank you for your consideration! > > -Jon From scolebourne at joda.org Sun Jan 28 23:09:35 2018 From: scolebourne at joda.org (Stephen Colebourne) Date: Sun, 28 Jan 2018 23:09:35 +0000 Subject: RFR JDK-8194412 : Adding 256 units of IsoFields.QUARTER_YEARS broken In-Reply-To: <4e596b86-912c-ce3c-d6c9-e509826c16ff@oracle.com> References: <4e596b86-912c-ce3c-d6c9-e509826c16ff@oracle.com> Message-ID: Spelling mistake in tests "qarterYears" Otherwise, looks good. thanks Stephen On 27 January 2018 at 19:37, Ivan Gerasimov wrote: > Hello! > > It was spotted that adding more then 255 quarter-years to a Temporal leads > to a wrong result. > > BUGURL: https://bugs.openjdk.java.net/browse/JDK-8194412 > WEBREV: http://cr.openjdk.java.net/~igerasim/8194412/00/webrev/ > > Would you please take a look at the fix and see that I'm not missing > something? > > Thanks in advance! > > -- > With kind regards, > Ivan Gerasimov > From claes.redestad at oracle.com Sun Jan 28 23:19:43 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 29 Jan 2018 00:19:43 +0100 Subject: Optimizing UUID#fromString(String) In-Reply-To: References: Message-ID: <8312f175-ffc6-eb56-75db-fe4d0fa9f475@oracle.com> Hi Jon, On 2018-01-27 17:05, Jon Chambers wrote: > Regardless, I wanted to call this optimization opportunity to your > attention, and would be happy to offer a proper patch if this seems like a > worthwhile change. this does looks promising - and at least points out there might be some performance issues lurking here! Found another compatibility issue, however: The current UUID.fromString allow use of any unicode digits parseable into digits. Say, Arabic digits (U+0660 - U+0669): jshell> UUID.fromString("\u0669-\u0669-\u0669-\u0669-\u0669"); $2 ==> 00000009-0009-0009-0009-000000000009 That you're getting some improvement from avoiding Character.digit(..) stands out in a way, so I wondered if there's something going on here... and it does seem that this method could be optimized by providing a fast-path in the CharacterDataLatin1 plane... I experimented with a few solutions, and the best I could find was a 256 element byte[] lookup table for digits in CharacterDataLatin1 (spanning the entire 8-bit space of CharacterDataLatin1 allows us to avoid a branch): http://cr.openjdk.java.net/~redestad/scratch/char.digit.00/ This speeds up your micro testing java.util.UUID (UUIDBenchmark.benchmarkUUIDFromString) by ~1.6x. The improvement is there on all counters when looking at -prof perfnorm (less loads, stores, branches, instructions, better CPI etc)[1], so seems like a pretty straightforward thing to consider independently. Your FastUUID implementation is still quite a bit faster, though, but it'd be good to dig around for a bit to see if there are other more general improvements like this to be had.. Thanks! /Claes [1] Baseline: Benchmark??????????????????????????????????????????????????? Mode Cnt???? Score?? Error? Units UUIDBenchmark.benchmarkUUIDFromString??????????????????????? avgt 4???? 0.505 ? 0.066? us/op UUIDBenchmark.benchmarkUUIDFromString:CPI avgt????????? 1.024?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-load-misses avgt????????? 3.000?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-loads avgt??????? 211.128?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-stores avgt???????? 34.894?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-icache-load-misses avgt????????? 0.067?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-load-misses avgt????????? 0.103?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-loads avgt????????? 0.812?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-store-misses avgt????????? 0.004?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-stores avgt????????? 0.054?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:branch-misses avgt???????? 13.269?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:branches avgt??????? 301.931?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:cycles avgt?????? 1585.057?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-load-misses avgt????????? 0.005?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-loads avgt??????? 208.788?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-store-misses avgt???????? ? 10???????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-stores avgt???????? 30.540?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:iTLB-load-misses avgt????????? 0.002?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:iTLB-loads avgt????????? 0.006?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:instructions avgt?????? 1547.568?????????? #/op Results with http://cr.openjdk.java.net/~redestad/scratch/char.digit.00/ -- : Benchmark??????????????????????????????????????????????????? Mode Cnt???? Score?? Error? Units UUIDBenchmark.benchmarkUUIDFromString??????????????????????? avgt 4???? 0.310 ? 0.022? us/op UUIDBenchmark.benchmarkUUIDFromString:CPI avgt????????? 0.832?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-load-misses avgt????????? 2.026?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-loads avgt??????? 208.227?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-stores avgt???????? 30.673?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:L1-icache-load-misses avgt????????? 0.052?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-load-misses avgt????????? 0.042?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-loads avgt????????? 1.027?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-store-misses avgt????????? 0.007?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:LLC-stores avgt????????? 0.044?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:branch-misses avgt????????? 0.040?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:branches avgt??????? 255.547?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:cycles avgt??????? 981.466?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-load-misses avgt????????? 0.003?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-loads avgt??????? 213.797?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-store-misses avgt???????? ? 10???????????? #/op UUIDBenchmark.benchmarkUUIDFromString:dTLB-stores avgt???????? 30.888?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:iTLB-load-misses avgt????????? 0.002?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:iTLB-loads avgt????????? 0.002?????????? #/op UUIDBenchmark.benchmarkUUIDFromString:instructions avgt?????? 1180.129?????????? #/op From ivan.gerasimov at oracle.com Mon Jan 29 00:12:54 2018 From: ivan.gerasimov at oracle.com (Ivan Gerasimov) Date: Sun, 28 Jan 2018 16:12:54 -0800 Subject: RFR JDK-8194412 : Adding 256 units of IsoFields.QUARTER_YEARS broken In-Reply-To: References: <4e596b86-912c-ce3c-d6c9-e509826c16ff@oracle.com> Message-ID: Thank you Stephen! On 1/28/18 3:09 PM, Stephen Colebourne wrote: > Spelling mistake in tests "qarterYears" Yes. Fixed the mistake and updated the webrev in place. > Otherwise, looks good. > thanks > Stephen > > > > On 27 January 2018 at 19:37, Ivan Gerasimov wrote: >> Hello! >> >> It was spotted that adding more then 255 quarter-years to a Temporal leads >> to a wrong result. >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8194412 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8194412/00/webrev/ >> >> Would you please take a look at the fix and see that I'm not missing >> something? >> >> Thanks in advance! >> >> -- >> With kind regards, >> Ivan Gerasimov >> -- With kind regards, Ivan Gerasimov From ramanand.patil at oracle.com Mon Jan 29 08:49:37 2018 From: ramanand.patil at oracle.com (Ramanand Patil) Date: Mon, 29 Jan 2018 00:49:37 -0800 (PST) Subject: RFR[JDK10]: 8195837: (tz) Support tzdata2018c Message-ID: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> Hi all, Please review the latest TZDATA integration (tzdata2018c) into JDK10. Bug: https://bugs.openjdk.java.net/browse/JDK-8195837 Webrev: http://cr.openjdk.java.net/~rpatil/8195837/webrev.00/ All the TimeZone related tests are passed after integration. Regards, Ramanand. From claes.redestad at oracle.com Mon Jan 29 11:15:38 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 29 Jan 2018 12:15:38 +0100 Subject: RFR: 8196331: Optimize Character.digit for latin1 input Message-ID: Hi, for the latin1 block of CharacterData we can improve the digit(int, int) implementation by providing an optimized lookup table. This improves microbenchmarks exercising Integer.parseInt, Long.parseLong and UUID.fromString etc by around 50%for typical inputs. Webrev: http://cr.openjdk.java.net/~redestad/8196331/open.00/ Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 The lookup array is pre-calculated to minimize startup impact (adds 1,027 bytecodes executed during initialization) /Claes From claes.redestad at oracle.com Mon Jan 29 11:30:05 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 29 Jan 2018 12:30:05 +0100 Subject: Optimizing UUID#fromString(String) In-Reply-To: <8312f175-ffc6-eb56-75db-fe4d0fa9f475@oracle.com> References: <8312f175-ffc6-eb56-75db-fe4d0fa9f475@oracle.com> Message-ID: Hi, I've filed two RFEs: - https://bugs.openjdk.java.net/browse/JDK-8196331 for providing a ? fast-path for ASCII/Latin1 input to Character.digit. Patch out ? for review - https://bugs.openjdk.java.net/browse/JDK-8196334 for tracking ? the optimizations specific to UUID proposed here Jon, if you want to provide a patch I'd be happy to sponsor. I have a few ideas on how to retrofit existing behavior that we can discuss offline. Also, if you haven't filed an OCA then this might be a good time to do so[1] Thanks! /Claes [1] http://openjdk.java.net/contribute/ On 2018-01-29 00:19, Claes Redestad wrote: > Hi Jon, > > On 2018-01-27 17:05, Jon Chambers wrote: >> Regardless, I wanted to call this optimization opportunity to your >> attention, and would be happy to offer a proper patch if this seems >> like a >> worthwhile change. > > this does looks promising - and at least points out there might be > some performance issues lurking here! > > Found another compatibility issue, however: The current > UUID.fromString allow use of any unicode digits parseable into > digits. Say, Arabic digits (U+0660 - U+0669): > > jshell> UUID.fromString("\u0669-\u0669-\u0669-\u0669-\u0669"); > $2 ==> 00000009-0009-0009-0009-000000000009 > > That you're getting some improvement from avoiding > Character.digit(..) stands out in a way, so I wondered if there's > something going on here... and it does seem that this method could > be optimized by providing a fast-path in the CharacterDataLatin1 > plane... > > I experimented with a few solutions, and the best I could find was > a 256 element byte[] lookup table for digits in CharacterDataLatin1 > (spanning the entire 8-bit space of CharacterDataLatin1 allows us > to avoid a branch): > > http://cr.openjdk.java.net/~redestad/scratch/char.digit.00/ > > This speeds up your micro testing java.util.UUID > (UUIDBenchmark.benchmarkUUIDFromString) by ~1.6x. The > improvement is there on all counters when looking at -prof perfnorm > (less loads, stores, branches, instructions, better CPI etc)[1], so seems > like a pretty straightforward thing to consider independently. > > Your FastUUID implementation is still quite a bit faster, though, but > it'd be good to dig around for a bit to see if there are other more > general improvements like this to be had.. > > Thanks! > > /Claes > > [1] > Baseline: > Benchmark??????????????????????????????????????????????????? Mode > Cnt???? Score?? Error? Units > UUIDBenchmark.benchmarkUUIDFromString??????????????????????? avgt > 4???? 0.505 ? 0.066? us/op > UUIDBenchmark.benchmarkUUIDFromString:CPI avgt 1.024?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-load-misses > avgt????????? 3.000?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-loads avgt > 211.128?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-stores avgt???????? > 34.894?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-icache-load-misses > avgt????????? 0.067?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-load-misses avgt????????? > 0.103?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-loads avgt 0.812?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-store-misses avgt????????? > 0.004?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-stores avgt 0.054?????????? > #/op > UUIDBenchmark.benchmarkUUIDFromString:branch-misses avgt > 13.269?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:branches avgt 301.931?????????? > #/op > UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 1585.057?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-load-misses avgt????????? > 0.005?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-loads avgt > 208.788?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-store-misses avgt???????? ? > 10???????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-stores avgt > 30.540?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:iTLB-load-misses avgt????????? > 0.002?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:iTLB-loads avgt 0.006?????????? > #/op > UUIDBenchmark.benchmarkUUIDFromString:instructions avgt > 1547.568?????????? #/op > > Results with > http://cr.openjdk.java.net/~redestad/scratch/char.digit.00/ -- : > Benchmark??????????????????????????????????????????????????? Mode > Cnt???? Score?? Error? Units > UUIDBenchmark.benchmarkUUIDFromString??????????????????????? avgt > 4???? 0.310 ? 0.022? us/op > UUIDBenchmark.benchmarkUUIDFromString:CPI avgt 0.832?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-load-misses > avgt????????? 2.026?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-loads avgt > 208.227?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-stores avgt???????? > 30.673?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:L1-icache-load-misses > avgt????????? 0.052?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-load-misses avgt????????? > 0.042?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-loads avgt 1.027?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-store-misses avgt????????? > 0.007?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:LLC-stores avgt 0.044?????????? > #/op > UUIDBenchmark.benchmarkUUIDFromString:branch-misses avgt > 0.040?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:branches avgt 255.547?????????? > #/op > UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 981.466?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-load-misses avgt????????? > 0.003?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-loads avgt > 213.797?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-store-misses avgt???????? ? > 10???????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:dTLB-stores avgt > 30.888?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:iTLB-load-misses avgt????????? > 0.002?????????? #/op > UUIDBenchmark.benchmarkUUIDFromString:iTLB-loads avgt 0.002?????????? > #/op > UUIDBenchmark.benchmarkUUIDFromString:instructions avgt > 1180.129?????????? #/op From amy.lu at oracle.com Mon Jan 29 12:17:55 2018 From: amy.lu at oracle.com (Amy Lu) Date: Mon, 29 Jan 2018 20:17:55 +0800 Subject: [JDK 11] RFR 8196211: Move two sun/nio/cs tests into OpenJDK In-Reply-To: <4BF3C2BF-1D61-440F-BFC3-18B2A47234DE@oracle.com> References: <74868834-8095-d530-ff16-66ec257f24ee@oracle.com> <4BF3C2BF-1D61-440F-BFC3-18B2A47234DE@oracle.com> Message-ID: <19a79a9f-d49f-8163-e486-2963b7a7d5da@oracle.com> Updated on the inputFileName. Please review: http://cr.openjdk.java.net/~amlu/8196211/webrev.01/ Thanks, Amy On 27/01/2018 2:32 AM, Paul Sandoz wrote: > Hi, > > Quick observation. > > EUCTWBufferBoundaryDecodeTest uses a different data file on windows and it uses the line separator as a trigger. Is it possible to better formalize this by passing in the argument for the file via jtreg? > > Paul. > > >> On Jan 25, 2018, at 11:45 PM, Amy Lu wrote: >> >> Please review the patch to move two sun/nio/cs tests into OpenJDK. >> >> bug: https://bugs.openjdk.java.net/browse/JDK-8196211 >> webrev: http://cr.openjdk.java.net/~amlu/8196211/webrev.00/ >> >> Thanks, >> Amy From enasser at in.ibm.com Mon Jan 29 13:08:09 2018 From: enasser at in.ibm.com (Nasser Ebrahim) Date: Mon, 29 Jan 2018 18:38:09 +0530 Subject: Clarification for adding classes shipped with JDK to sun.rmi.registry.registryFilter property in java.security file In-Reply-To: References: Message-ID: Hi Roger, Thank you for your inputs. I agree with you that logging the filtering action can help developers to easily identify the classes which are failed due to the new filtering mechanism. However, it may not be fair to ask the application developers to add the basic JDK classes when Java introduced the filtering mechanism. For application classes, it makes sense for developer to use the logging mechanism to identify the classes to be added to the white list. I think JDK should add all its standard classes (at least the base packages) to white list by default rather than asking each end user to add to the white list. Please clarify whether there is any reason for not adding the standard JDK classes by default to the white list as they are trusted code. Thank you, Nasser Ebrahim From: Roger Riggs To: core-libs-dev at openjdk.java.net Date: 01/19/2018 08:50 PM Subject: Re: Clarification for adding classes shipped with JDK to sun.rmi.registry.registryFilter property in java.security file Sent by: "core-libs-dev" Hi Vipin, A couple of suggestions to make working with serialization filters easier. JEP 290 enabled logging of filtering actions by setting java.io.serialization.level = FINEST. Add that to the logging.properties and supply it on the command line with "-Djava.util.logging.config.file=logging.properties". If set to FINEST, it will log every call to the filter with the arguments of class, array size, etc. If set to FINE, it will log only the cases where the filter rejects based on its arguments. The pattern based filters can allow or disallow entire packages. For example, "java.net.**" will allow any class in any package with a "java.net." prefix. For development purposes, serialized classes for the registry could be identified using a filter of "*" with logging enabled as FINEST and examining the log. Expanding the white list for the registry is a possibility with a very conservative evaluation of specific cases. Regards, Roger [1] https://urldefense.proofpoint.com/v2/url?u=http-3A__openjdk.java.net_jeps_290&d=DwIFaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=yiMdG4h7wT1IHGC_YAWYv5USAuyCyINm1bo82D5y5bY&m=12RWeCnDad-pQO_PzN0K_mAliSGWmNyH_hOob40MLlA&s=HSk5hl04g1T39ZWtVoEpYFo0CYAkuLPyfQlKA2eh5Us&e= On 1/16/2018 7:21 AM, Vipin Mv1 wrote: > > Hi, > > After upgrading to oracle 8u121-b12, I get following Exception when running > the following testcase. > > import java.io.*; > import java.net.InetAddress; > import java.rmi.*; > import java.rmi.registry.*; > import java.security.Security; > > /* @test > * @run main/othervm RegistryFilterTest > */ > > public class RegistryFilterTest { > > private static int port = 12345; > private static Registry registry; > > static class RemoteObj implements Serializable, Remote { > private static final long serialVersionUID = 01L; > > final Object obj; > > RemoteObj(Object obj) { > this.obj = obj; > } > > > } > > > public static void main(String[] args) throws Exception{ > > InetAddress in = InetAddress.getLocalHost(); > LocateRegistry.createRegistry(port); > Registry registry = LocateRegistry.getRegistry("localhost", > port); > registry.bind("InetAddress", new RemoteObj(in)); > registry.unbind("InetAddress"); > System.out.println("RMI Registry Test Passed"); > > } > > } > > TestResults > ----------------------- > Jan 12, 2018 5:32:18 PM java.io.ObjectInputStream filterCheck > INFO: ObjectInputFilter REJECTED: class java.net.InetAddress, array length: > -1, nRefs: 5, depth: 2, bytes: 216, ex: n/a > Caused by: java.io.InvalidClassException: filter status: REJECTED > at java.io.ObjectInputStream.filterCheck(ObjectInputStream.java:1406) > at java.io.ObjectInputStream.readNonProxyDesc > (ObjectInputStream.java:1997) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1877) > at java.io.ObjectInputStream.readOrdinaryObject > (ObjectInputStream.java:2170) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) > at java.io.ObjectInputStream.defaultReadFields > (ObjectInputStream.java:2415) > at java.io.ObjectInputStream.readSerialData > (ObjectInputStream.java:2339) > at java.io.ObjectInputStream.readOrdinaryObject > (ObjectInputStream.java:2197) > at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1698) > at java.io.ObjectInputStream.readObjectImpl(ObjectInputStream.java:540) > at java.io.ObjectInputStream.readObject(ObjectInputStream.java:475) > ... 16 more > > I understand it is due to the new serialization Filtering mechanism > introduced to RMI Registry and Distributed Garbage Collection as a part of > the following fixes. > > JDK-8160108 Implement Serialization Filtering > JDK-8156804: Better constraint checking(not public) > > As a result, a new default white list has been introduced for RMIRegistry > which allows deserialization only to limited set of classes by default. > Which are > > String.class > java.lang.Number.class > java.rmi.Remote.class > java.lang.reflect.Proxy.class > sun.rmi.server.UnicastRef.class > java.rmi.server.RMIClientSocketFactory.class > java.rmi.server.RMIServerSocketFactory.class > java.rmi.activation.ActivationID.class > java.rmi.server.UID.class > and their subclasses. > > It also enabled provisions to increase the default white list by > > 1. updating property "sun.rmi.registry.registryFilter" in java.security > file > 2. adding -Dsun.rmi.registry.registryFilter in java command line > > Though the current solution is straight forward, it is difficult from an > application point of view to identify such classes without running the > application multiple times to come up with comprehensive list to be > allowed. This would be a overhead to the developer and should be avoided at > least for classes shipped with JDK (for example, java.net.InetAddress as > used in testcase). > > I assume that default list is very limited to reduce the chances of getting > exploited by serialization vulnerability. I think, we can spare public > packages shipped with JDK by adding them to > "sun.rmi.registry.registryFilter" in java.security file out of the box. > > Please let me know your views. > > Thanks & Regards, > Vipin MV From jon.chambers at gmail.com Mon Jan 29 13:59:22 2018 From: jon.chambers at gmail.com (Jon Chambers) Date: Mon, 29 Jan 2018 08:59:22 -0500 Subject: Optimizing UUID#fromString(String) In-Reply-To: References: <8312f175-ffc6-eb56-75db-fe4d0fa9f475@oracle.com> Message-ID: Awesome. Excited to see the Character.digit improvements! To set expectations from here, I'll need to do two things to move forward: 1. I need to sign the OCA, as you've pointed out. As a courtesy, I'd like to alert my employer before doing so. This has been a non-issue for similar agreements for other open-source projects in the past. 2. I'll need to work through the OpenJDK build/test process and (I suspect?) do some very preliminary reading on Mercurial. Because this is a personal project, I may not have an opportunity to make much headway there until this weekend. That said, I'm grateful for the sponsorship and look forward to working with you. I'll follow up directly to discuss the details. Cheers! -Jon On Mon, Jan 29, 2018 at 6:30 AM, Claes Redestad wrote: > Hi, > > I've filed two RFEs: > > - https://bugs.openjdk.java.net/browse/JDK-8196331 for providing a > fast-path for ASCII/Latin1 input to Character.digit. Patch out > for review > - https://bugs.openjdk.java.net/browse/JDK-8196334 for tracking > the optimizations specific to UUID proposed here > > Jon, if you want to provide a patch I'd be happy to sponsor. I have > a few ideas on how to retrofit existing behavior that we can discuss > offline. Also, if you haven't filed an OCA then this might be a good > time to do so[1] > > Thanks! > > /Claes > > [1] http://openjdk.java.net/contribute/ > > > On 2018-01-29 00:19, Claes Redestad wrote: > >> Hi Jon, >> >> On 2018-01-27 17:05, Jon Chambers wrote: >> >>> Regardless, I wanted to call this optimization opportunity to your >>> attention, and would be happy to offer a proper patch if this seems like >>> a >>> worthwhile change. >>> >> >> this does looks promising - and at least points out there might be >> some performance issues lurking here! >> >> Found another compatibility issue, however: The current >> UUID.fromString allow use of any unicode digits parseable into >> digits. Say, Arabic digits (U+0660 - U+0669): >> >> jshell> UUID.fromString("\u0669-\u0669-\u0669-\u0669-\u0669"); >> $2 ==> 00000009-0009-0009-0009-000000000009 >> >> That you're getting some improvement from avoiding >> Character.digit(..) stands out in a way, so I wondered if there's >> something going on here... and it does seem that this method could >> be optimized by providing a fast-path in the CharacterDataLatin1 >> plane... >> >> I experimented with a few solutions, and the best I could find was >> a 256 element byte[] lookup table for digits in CharacterDataLatin1 >> (spanning the entire 8-bit space of CharacterDataLatin1 allows us >> to avoid a branch): >> >> http://cr.openjdk.java.net/~redestad/scratch/char.digit.00/ >> >> This speeds up your micro testing java.util.UUID >> (UUIDBenchmark.benchmarkUUIDFromString) by ~1.6x. The >> improvement is there on all counters when looking at -prof perfnorm >> (less loads, stores, branches, instructions, better CPI etc)[1], so seems >> like a pretty straightforward thing to consider independently. >> >> Your FastUUID implementation is still quite a bit faster, though, but >> it'd be good to dig around for a bit to see if there are other more >> general improvements like this to be had.. >> >> Thanks! >> >> /Claes >> >> [1] >> Baseline: >> Benchmark Mode >> Cnt Score Error Units >> UUIDBenchmark.benchmarkUUIDFromString avgt 4 >> 0.505 ? 0.066 us/op >> UUIDBenchmark.benchmarkUUIDFromString:CPI avgt 1.024 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-load-misses >> avgt 3.000 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-loads avgt >> 211.128 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-stores avgt >> 34.894 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-icache-load-misses >> avgt 0.067 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-load-misses avgt >> 0.103 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-loads avgt 0.812 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-store-misses avgt >> 0.004 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-stores avgt 0.054 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:branch-misses avgt >> 13.269 #/op >> UUIDBenchmark.benchmarkUUIDFromString:branches avgt 301.931 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 1585.057 #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-load-misses avgt >> 0.005 #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-loads avgt 208.788 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-store-misses avgt ? >> 10?? #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-stores avgt 30.540 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:iTLB-load-misses avgt >> 0.002 #/op >> UUIDBenchmark.benchmarkUUIDFromString:iTLB-loads avgt 0.006 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:instructions avgt >> 1547.568 #/op >> >> Results with http://cr.openjdk.java.net/~redestad/scratch/char.digit.00/ >> -- : >> Benchmark Mode >> Cnt Score Error Units >> UUIDBenchmark.benchmarkUUIDFromString avgt 4 >> 0.310 ? 0.022 us/op >> UUIDBenchmark.benchmarkUUIDFromString:CPI avgt 0.832 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-load-misses >> avgt 2.026 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-loads avgt >> 208.227 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-dcache-stores avgt >> 30.673 #/op >> UUIDBenchmark.benchmarkUUIDFromString:L1-icache-load-misses >> avgt 0.052 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-load-misses avgt >> 0.042 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-loads avgt 1.027 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-store-misses avgt >> 0.007 #/op >> UUIDBenchmark.benchmarkUUIDFromString:LLC-stores avgt 0.044 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:branch-misses avgt 0.040 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:branches avgt 255.547 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:cycles avgt 981.466 #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-load-misses avgt >> 0.003 #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-loads avgt 213.797 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-store-misses avgt ? >> 10?? #/op >> UUIDBenchmark.benchmarkUUIDFromString:dTLB-stores avgt 30.888 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:iTLB-load-misses avgt >> 0.002 #/op >> UUIDBenchmark.benchmarkUUIDFromString:iTLB-loads avgt 0.002 >> #/op >> UUIDBenchmark.benchmarkUUIDFromString:instructions avgt >> 1180.129 #/op >> > > From Roger.Riggs at Oracle.com Mon Jan 29 14:12:51 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 29 Jan 2018 09:12:51 -0500 Subject: RFR JDK-8194412 : Adding 256 units of IsoFields.QUARTER_YEARS broken In-Reply-To: References: <4e596b86-912c-ce3c-d6c9-e509826c16ff@oracle.com> Message-ID: Hi Ivan, Looks good! Thanks, Roger On 1/28/2018 6:09 PM, Stephen Colebourne wrote: > Spelling mistake in tests "qarterYears" > Otherwise, looks good. > thanks > Stephen > > > > On 27 January 2018 at 19:37, Ivan Gerasimov wrote: >> Hello! >> >> It was spotted that adding more then 255 quarter-years to a Temporal leads >> to a wrong result. >> >> BUGURL: https://bugs.openjdk.java.net/browse/JDK-8194412 >> WEBREV: http://cr.openjdk.java.net/~igerasim/8194412/00/webrev/ >> >> Would you please take a look at the fix and see that I'm not missing >> something? >> >> Thanks in advance! >> >> -- >> With kind regards, >> Ivan Gerasimov >> From sean.coffey at oracle.com Mon Jan 29 14:42:11 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Mon, 29 Jan 2018 14:42:11 +0000 Subject: RFR[JDK10]: 8195837: (tz) Support tzdata2018c In-Reply-To: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> References: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> Message-ID: <2f58265e-fd81-39e8-b1d5-ef0bcad7550f@oracle.com> The changes look fine to me Ramanand. The europe file contains some interesting comments about the rolled back changes that have been made for 2018c. A plan on how to resolve these pending changes can be followed up via JDK-8195595 Regards, Sean. On 29/01/18 08:49, Ramanand Patil wrote: > Hi all, > Please review the latest TZDATA integration (tzdata2018c) into JDK10. > Bug: https://bugs.openjdk.java.net/browse/JDK-8195837 > Webrev: http://cr.openjdk.java.net/~rpatil/8195837/webrev.00/ > > All the TimeZone related tests are passed after integration. > > Regards, > Ramanand. From brian.burkhalter at oracle.com Mon Jan 29 16:14:40 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 29 Jan 2018 08:14:40 -0800 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> Message-ID: Hi Patrick, Conceptually this looks good. There might be some javadoc tweaks needed but those can be worked out while this is a draft CSR (to be filed). Thanks, Brian On Jan 27, 2018, at 12:45 AM, Patrick Reinhart wrote: > I saw that great extension on InputStream/OutputStream adding a Null > implementation there. Based on those I would propose a similar addition > to the Reader/Writer. From gnu.andrew at redhat.com Mon Jan 29 16:26:37 2018 From: gnu.andrew at redhat.com (Andrew Hughes) Date: Mon, 29 Jan 2018 16:26:37 +0000 Subject: RFR[JDK10]: 8195837: (tz) Support tzdata2018c In-Reply-To: <2f58265e-fd81-39e8-b1d5-ef0bcad7550f@oracle.com> References: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> <2f58265e-fd81-39e8-b1d5-ef0bcad7550f@oracle.com> Message-ID: On 29 January 2018 at 14:42, Se?n Coffey wrote: > The changes look fine to me Ramanand. The europe file contains some > interesting comments about the rolled back changes that have been made for > 2018c. A plan on how to resolve these pending changes can be followed up via > JDK-8195595 > Yes... I've been following the discussion on the tzdb list about these changes to the Irish timezone and the breakage they cause. It's good to see that, at least for now, these changes will be reverted locally for OpenJDK. My thanks to those who've put forward the case on that list for the compatibility issues these changes cause, particularly Stephen Colebourne. > Regards, > Sean. > > -- Andrew :) Senior Free Java Software Engineer Red Hat, Inc. (http://www.redhat.com) Web Site: http://fuseyism.com Twitter: https://twitter.com/gnu_andrew_java PGP Key: ed25519/0xCFDA0F9B35964222 (hkp://keys.gnupg.net) Fingerprint = 5132 579D D154 0ED2 3E04 C5A0 CFDA 0F9B 3596 4222 From paul.sandoz at oracle.com Mon Jan 29 16:37:38 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 29 Jan 2018 08:37:38 -0800 Subject: [JDK 11] RFR 8196211: Move two sun/nio/cs tests into OpenJDK In-Reply-To: <19a79a9f-d49f-8163-e486-2963b7a7d5da@oracle.com> References: <74868834-8095-d530-ff16-66ec257f24ee@oracle.com> <4BF3C2BF-1D61-440F-BFC3-18B2A47234DE@oracle.com> <19a79a9f-d49f-8163-e486-2963b7a7d5da@oracle.com> Message-ID: That?s better :-) If you wanna make it static i suggest: s/inputFileName/INPUT_FILE_NAME although there is little benefit here since all access is local to main. Up to you to make local or keep static. No need for another review. Paul. > On Jan 29, 2018, at 4:17 AM, Amy Lu wrote: > > Updated on the inputFileName. > Please review: http://cr.openjdk.java.net/~amlu/8196211/webrev.01/ > > Thanks, > Amy > On 27/01/2018 2:32 AM, Paul Sandoz wrote: >> Hi, >> >> Quick observation. >> >> EUCTWBufferBoundaryDecodeTest uses a different data file on windows and it uses the line separator as a trigger. Is it possible to better formalize this by passing in the argument for the file via jtreg? >> >> Paul. >> >> >>> On Jan 25, 2018, at 11:45 PM, Amy Lu wrote: >>> >>> Please review the patch to move two sun/nio/cs tests into OpenJDK. >>> >>> bug: https://bugs.openjdk.java.net/browse/JDK-8196211 >>> webrev: http://cr.openjdk.java.net/~amlu/8196211/webrev.00/ >>> >>> Thanks, >>> Amy > From martinrb at google.com Mon Jan 29 16:47:42 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jan 2018 08:47:42 -0800 Subject: RFR[JDK10]: 8195837: (tz) Support tzdata2018c In-Reply-To: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> References: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> Message-ID: Google has been successfully running jdk8 and jdk9 with 2018c. On Mon, Jan 29, 2018 at 12:49 AM, Ramanand Patil wrote: > Hi all, > Please review the latest TZDATA integration (tzdata2018c) into JDK10. > Bug: https://bugs.openjdk.java.net/browse/JDK-8195837 > Webrev: http://cr.openjdk.java.net/~rpatil/8195837/webrev.00/ > > All the TimeZone related tests are passed after integration. > > Regards, > Ramanand. > From martinrb at google.com Mon Jan 29 17:44:06 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jan 2018 09:44:06 -0800 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: References: Message-ID: Thanks. I might try to shrink the size of the static array, by only storing values between '0' and 'z', at the cost of slightly greater lookup costs for each char. On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad wrote: > Hi, > > for the latin1 block of CharacterData we can improve the > digit(int, int) implementation by providing an optimized lookup > table. This improves microbenchmarks exercising Integer.parseInt, > Long.parseLong and UUID.fromString etc by around 50%for typical > inputs. > > Webrev: http://cr.openjdk.java.net/~redestad/8196331/open.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 > > The lookup array is pre-calculated to minimize startup impact > (adds 1,027 bytecodes executed during initialization) > > /Claes > From paul.sandoz at oracle.com Mon Jan 29 18:14:44 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 29 Jan 2018 10:14:44 -0800 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: References: Message-ID: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> > On Jan 29, 2018, at 9:44 AM, Martin Buchholz wrote: > > Thanks. I might try to shrink the size of the static array, by only > storing values between '0' and 'z', at the cost of slightly greater lookup > costs for each char. > I was wondering the same, or just clip the end of the array to?z' if (ch <= ?z? && radix ?) { // Might even fold the upper bounds check for DIGITS value = DIGITS[ch]; ... } Paul. > On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad > wrote: > >> Hi, >> >> for the latin1 block of CharacterData we can improve the >> digit(int, int) implementation by providing an optimized lookup >> table. This improves microbenchmarks exercising Integer.parseInt, >> Long.parseLong and UUID.fromString etc by around 50%for typical >> inputs. >> >> Webrev: http://cr.openjdk.java.net/~redestad/8196331/open.00/ >> Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 >> >> The lookup array is pre-calculated to minimize startup impact >> (adds 1,027 bytecodes executed during initialization) >> >> /Claes >> From xueming.shen at oracle.com Mon Jan 29 17:52:46 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 29 Jan 2018 09:52:46 -0800 Subject: RFR JDK-8176379: java.util.Base64 mime encoder behaves incorrectly if initialized with a line length of size 1-3 In-Reply-To: <6DB12EB2-4A10-4091-A811-FCEA8B2B67BB@oracle.com> References: <5A6BBA7E.3090001@oracle.com> <6DB12EB2-4A10-4091-A811-FCEA8B2B67BB@oracle.com> Message-ID: <5A6F5F6E.3020509@oracle.com> On 01/26/2018 03:53 PM, Paul Sandoz wrote: > Base64 > ? > > 138 lineLength = lineLength>> 2<< 2; > > I know the code was shuffled up but i find the following clearer in terms of bit manipulation: > > // Round down to nearest mulitple of 4 > lineLength&= ~0b11 > sure, we can go the straightforward bit manipulation. > TestBase64 > ? > > Is there a test passing in a negative value for len? > Actually I have a dedicated test case for "mime encoder maxlen". *test/jdk/java/util/Base64/Base64GetEncoderTest.java* I have updated it to test the negative and the < 4 maxlen. http://cr.openjdk.java.net/~sherman/8176379/webrev/ webrev has been updated according. Thanks! Sherman From paul.sandoz at oracle.com Mon Jan 29 18:20:17 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 29 Jan 2018 10:20:17 -0800 Subject: RFR JDK-8176379: java.util.Base64 mime encoder behaves incorrectly if initialized with a line length of size 1-3 In-Reply-To: <5A6F5F6E.3020509@oracle.com> References: <5A6BBA7E.3090001@oracle.com> <6DB12EB2-4A10-4091-A811-FCEA8B2B67BB@oracle.com> <5A6F5F6E.3020509@oracle.com> Message-ID: <77B75AD7-1919-494E-A186-3AF495C30AEB@oracle.com> +1 We will require a CSR for the minor adjustment to the specification. Paul. > On Jan 29, 2018, at 9:52 AM, Xueming Shen wrote: > > On 01/26/2018 03:53 PM, Paul Sandoz wrote: >> >> Base64 >> ? >> >> 138 lineLength = lineLength >> 2 << 2; >> >> I know the code was shuffled up but i find the following clearer in terms of bit manipulation: >> >> // Round down to nearest mulitple of 4 >> lineLength &= ~0b11 >> > > sure, we can go the straightforward bit manipulation. > >> TestBase64 >> ? >> >> Is there a test passing in a negative value for len? >> > > Actually I have a dedicated test case for "mime encoder maxlen". > test/jdk/java/util/Base64/Base64GetEncoderTest.java > I have updated it to test the negative and the < 4 maxlen. > > http://cr.openjdk.java.net/~sherman/8176379/webrev/ > > webrev has been updated according. > > Thanks! > Sherman From Roger.Riggs at Oracle.com Mon Jan 29 18:29:07 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Mon, 29 Jan 2018 13:29:07 -0500 Subject: RFR JDK-8176379: java.util.Base64 mime encoder behaves incorrectly if initialized with a line length of size 1-3 In-Reply-To: <77B75AD7-1919-494E-A186-3AF495C30AEB@oracle.com> References: <5A6BBA7E.3090001@oracle.com> <6DB12EB2-4A10-4091-A811-FCEA8B2B67BB@oracle.com> <5A6F5F6E.3020509@oracle.com> <77B75AD7-1919-494E-A186-3AF495C30AEB@oracle.com> Message-ID: <876e77dc-651e-73d5-9c30-29a21d24f7ef@Oracle.com> Hi, Looks good. Base64.java: line 120? typo:? "outputut" -> "output" Roger On 1/29/2018 1:20 PM, Paul Sandoz wrote: > +1 > > We will require a CSR for the minor adjustment to the specification. > > Paul. > >> On Jan 29, 2018, at 9:52 AM, Xueming Shen wrote: >> >> On 01/26/2018 03:53 PM, Paul Sandoz wrote: >>> Base64 >>> ? >>> >>> 138 lineLength = lineLength >> 2 << 2; >>> >>> I know the code was shuffled up but i find the following clearer in terms of bit manipulation: >>> >>> // Round down to nearest mulitple of 4 >>> lineLength &= ~0b11 >>> >> sure, we can go the straightforward bit manipulation. >> >>> TestBase64 >>> ? >>> >>> Is there a test passing in a negative value for len? >>> >> Actually I have a dedicated test case for "mime encoder maxlen". >> test/jdk/java/util/Base64/Base64GetEncoderTest.java >> I have updated it to test the negative and the < 4 maxlen. >> >> http://cr.openjdk.java.net/~sherman/8176379/webrev/ >> >> webrev has been updated according. >> >> Thanks! >> Sherman From claudia at reini.net Sat Jan 27 08:43:41 2018 From: claudia at reini.net (Claudia Reinhart) Date: Sat, 27 Jan 2018 09:43:41 +0100 Subject: JDK-8196298: Add null Reader and Writer Message-ID: Hi there, I saw that great extension on InputStream/OutputStream adding a Null implementation there. Based on those I would propose a similar addition to the Reader/Writer. -Patrick Here is what the API would look like based on the existing from InputStream/OutputStream: ??? /** ???? * Returns a new {@code Reader} that contains no characters. The returned ???? * stream is initially open.? The stream is closed by calling the ???? * {@code close()} method.? Subsequent calls to {@code close()} have no ???? * effect. ???? * ???? *

While the stream is open, the {@code ready?()}, {@code read()}, ???? * {@code read(char[])}, {@code read(char[], int, int)}, {@code skip()}, and ???? * {@code transferTo()} methods all behave as if end of stream has been ???? * reached.? After the stream has been closed, these methods all throw ???? * {@code IOException}. ???? * ???? *

The {@code markSupported()} method returns {@code false}.? The ???? * {@code mark()} method does nothing, and the {@code reset()} method ???? * throws {@code IOException}. ???? * ???? * @return an {@code Reader} which contains no characters ???? * ???? * @since 11 ???? */ ??? public static Reader nullReader() {} ??? /** ???? * Returns a new {@code Writer} which discards all characters.? The ???? * returned stream is initially open.? The stream is closed by calling ???? * the {@code close()} method.? Subsequent calls to {@code close()} have ???? * no effect. ???? * ???? *

While the stream is open, the {@code write(int)}, {@code ???? * write(char[])}, and {@code write(char[], int, int)} methods do nothing. ???? * After the stream has been closed, these methods all throw {@code ???? * IOException}. ???? * ???? *

The {@code flush()} method does nothing. ???? * ???? * @return an {@code Writer} which discards all characters ???? * ???? * @since 11 ???? */ ??? public static Writer nullWriter() {} From claes.redestad at oracle.com Mon Jan 29 19:37:20 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 29 Jan 2018 20:37:20 +0100 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> Message-ID: <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> I ran with a smaller byte[] initially and saw about a 10% improvement from removing the branch, which is why I felt the superfluous bytes were motivated. /Claes Paul Sandoz skrev: (29 januari 2018 19:14:44 CET) > > >> On Jan 29, 2018, at 9:44 AM, Martin Buchholz >wrote: >> >> Thanks. I might try to shrink the size of the static array, by only >> storing values between '0' and 'z', at the cost of slightly greater >lookup >> costs for each char. >> > >I was wondering the same, or just clip the end of the array to?z' > >if (ch <= ?z? && radix ?) { // Might even fold the upper bounds check >for DIGITS > value = DIGITS[ch]; > ... >} > >Paul. > >> On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad > >> wrote: >> >>> Hi, >>> >>> for the latin1 block of CharacterData we can improve the >>> digit(int, int) implementation by providing an optimized lookup >>> table. This improves microbenchmarks exercising Integer.parseInt, >>> Long.parseLong and UUID.fromString etc by around 50%for typical >>> inputs. >>> >>> Webrev: http://cr.openjdk.java.net/~redestad/8196331/open.00/ >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 >>> >>> The lookup array is pre-calculated to minimize startup impact >>> (adds 1,027 bytecodes executed during initialization) >>> >>> /Claes >>> -- Sent from my Android device with K-9 Mail. Please excuse my brevity. From paul.sandoz at oracle.com Mon Jan 29 19:38:41 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 29 Jan 2018 11:38:41 -0800 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> Message-ID: Smaller in only the upper bound? I would an explicit upper bounds check would dominate that of the bounds check for the array itself. Paul. > On Jan 29, 2018, at 11:37 AM, Claes Redestad wrote: > > I ran with a smaller byte[] initially and saw about a 10% improvement from removing the branch, which is why I felt the superfluous bytes were motivated. > > /Claes > > Paul Sandoz skrev: (29 januari 2018 19:14:44 CET) > > > On Jan 29, 2018, at 9:44 AM, Martin Buchholz wrote: > > Thanks. I might try to shrink the size of the static array, by only > storing values between '0' and 'z', at the cost of slightly greater lookup > costs for each char. > > > I was wondering the same, or just clip the end of the array to?z' > > if (ch <= ?z? && radix ?) { // Might even fold the upper bounds check for DIGITS > value = DIGITS[ch]; > ... > } > > Paul. > > On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad > wrote: > > Hi, > > for the latin1 block of CharacterData we can improve the > digit(int, int) implementation by providing an optimized lookup > table. This improves microbenchmarks exercising Integer.parseInt, > Long.parseLong and UUID.fromString etc by around 50%for typical > inputs. > > Webrev: http://cr.openjdk.java.net/~redestad/8196331/open.00/ > Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 > > The lookup array is pre-calculated to minimize startup impact > (adds 1,027 bytecodes executed during initialization) > > /Claes > > > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. From patrick at reini.net Mon Jan 29 19:43:20 2018 From: patrick at reini.net (Patrick Reinhart) Date: Mon, 29 Jan 2018 20:43:20 +0100 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> Message-ID: <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> Just added a new CSR for it based on the one for InputStream/OutputStream... -Patrick Am 29.01.2018 um 17:14 schrieb Brian Burkhalter: > Hi Patrick, > > Conceptually this looks good. There might be some javadoc tweaks > needed but those can be worked out while this is a draft CSR (to be > filed). > > Thanks, > > Brian > > On Jan 27, 2018, at 12:45 AM, Patrick Reinhart > wrote: > >> I saw that great extension on InputStream/OutputStream adding a Null >> implementation there. Based on those I would propose a similar addition >> to the Reader/Writer. > From brian.burkhalter at oracle.com Mon Jan 29 20:58:26 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 29 Jan 2018 12:58:26 -0800 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> Message-ID: <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> Great - thanks - will take a look. Brian On Jan 29, 2018, at 11:43 AM, Patrick Reinhart wrote: > Just added a new CSR for it based on the one for InputStream/OutputStream... From naoto.sato at oracle.com Mon Jan 29 20:56:14 2018 From: naoto.sato at oracle.com (Naoto Sato) Date: Mon, 29 Jan 2018 12:56:14 -0800 Subject: RFR[JDK10]: 8195837: (tz) Support tzdata2018c In-Reply-To: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> References: <0d83a571-fa8b-4f47-8487-c6dff6fd5a03@default> Message-ID: <3dde2e36-3c7a-6a65-cb80-30e0dcfc59fb@oracle.com> Looks good to me. Naoto On 1/29/18 12:49 AM, Ramanand Patil wrote: > Hi all, > Please review the latest TZDATA integration (tzdata2018c) into JDK10. > Bug: https://bugs.openjdk.java.net/browse/JDK-8195837 > Webrev: http://cr.openjdk.java.net/~rpatil/8195837/webrev.00/ > > All the TimeZone related tests are passed after integration. > > Regards, > Ramanand. > From martinrb at google.com Mon Jan 29 21:02:16 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jan 2018 13:02:16 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> Message-ID: I'm going to expedite this a little since we are building further changes on top. RFR: jsr166 jdk integration 2018-02 http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html 8196207: Inefficient ArrayList.subList().toArray() http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ArrayList-subList-toArray/index.html https://bugs.openjdk.java.net/browse/JDK-8196207 On Fri, Jan 26, 2018 at 9:12 AM, Martin Buchholz wrote: > It may be that array-backed lists inheriting from AbstractList is not > useful except when creating a prototype (or as a way of testing > AbstractList!) because a high-quality implementation should always override. > > On Fri, Jan 26, 2018 at 5:31 AM, Claes Redestad > wrote: > >> On 2018-01-26 07:20, Martin Buchholz wrote: >> >>> It's still possible to find simple performance improvements in classes >>> like >>> ArrayList. >>> >> >> Indeed! >> >> It seems ArrayList.SubList delegates quite a few of its implementations >> to AbstractList and AbstractCollection, which uses iterators for things >> that could be done more efficiently with conventional for-loops. >> >> This includes contains, equals, hashCode, indexOf, lastIndexOf... I saw >> up to 7x improvements on some of these over the inherited >> implementations when providing specialized ones for ImmutableList.SubList >> as part of JDK-8193128 >> >> I'll file a bug and maybe even tinker with it when I find some time... >> >> /Claes >> > > From claes.redestad at oracle.com Mon Jan 29 21:04:29 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Mon, 29 Jan 2018 22:04:29 +0100 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> Message-ID: Right, I can't really explain why, but the effect is very visible and reproducible in microbenchmarks. Differences in generated ASM might be indicating that the inlining behavior changes enough to shift the result around; maybe a job for @ForceInline? I'll experiment and analyze a bit more tomorrow to see if I can find a good explanation for the observed difference and/or a solution that allows us to slim down the lookup array. /Claes On 2018-01-29 20:38, Paul Sandoz wrote: > Smaller in only the upper bound? I would an explicit upper bounds > check would dominate that of the bounds check for the array itself. > > Paul. > >> On Jan 29, 2018, at 11:37 AM, Claes Redestad >> > wrote: >> >> I ran with a smaller byte[] initially and saw about a 10% improvement >> from removing the branch, which is why I felt the superfluous bytes >> were motivated. >> >> /Claes >> >> Paul Sandoz > >> skrev: (29 januari 2018 19:14:44 CET) >> >> >> On Jan 29, 2018, at 9:44 AM, Martin Buchholz >> > wrote: >> Thanks. I might try to shrink the size of the static array, >> by only storing values between '0' and 'z', at the cost of >> slightly greater lookup costs for each char. >> >> >> I was wondering the same, or just clip the end of the array to?z' >> >> if (ch <= ?z? && radix ?) { // Might even fold the upper bounds check for DIGITS >> value = DIGITS[ch]; >> ... >> } >> >> Paul. >> >> On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad >> > > wrote: >> >> Hi, for the latin1 block of CharacterData we can improve >> the digit(int, int) implementation by providing an >> optimized lookup table. This improves microbenchmarks >> exercising Integer.parseInt, Long.parseLong and >> UUID.fromString etc by around 50%for typical inputs. >> Webrev: >> http://cr.openjdk.java.net/~redestad/8196331/open.00/ >> >> Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 The >> lookup array is pre-calculated to minimize startup impact >> (adds 1,027 bytecodes executed during initialization) /Claes >> >> >> >> -- >> Sent from my Android device with K-9 Mail. Please excuse my brevity. > From brian.burkhalter at oracle.com Mon Jan 29 22:55:48 2018 From: brian.burkhalter at oracle.com (Brian Burkhalter) Date: Mon, 29 Jan 2018 14:55:48 -0800 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> Message-ID: <01CCBA7D-1815-4A78-A803-A9D18E437B34@oracle.com> I made a few minor changes to the CSR [1] verbiage. Probably the nullWriter() documentation should also be updated to explicitly mention all variants of both append() and write(). Thanks, Brian [1] https://bugs.openjdk.java.net/browse/JDK-8196350 On Jan 29, 2018, at 12:58 PM, Brian Burkhalter wrote: > Great - thanks - will take a look. > > Brian > > On Jan 29, 2018, at 11:43 AM, Patrick Reinhart wrote: > >> Just added a new CSR for it based on the one for InputStream/OutputStream... From paul.sandoz at oracle.com Mon Jan 29 23:10:32 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 29 Jan 2018 15:10:32 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> Message-ID: <3E1289BB-6296-4414-BD40-C2CBA72DF527@oracle.com> > On Jan 29, 2018, at 1:02 PM, Martin Buchholz wrote: > > I'm going to expedite this a little since we are building further changes > on top. > > RFR: jsr166 jdk integration 2018-02 > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > Looks ok, but i personally would not categorize the F/J changes as miscellaneous :-) > 8196207: Inefficient ArrayList.subList().toArray() > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/ArrayList-subList-toArray/index.html > https://bugs.openjdk.java.net/browse/JDK-8196207 > Looks ok too. Paul. From martinrb at google.com Mon Jan 29 23:30:42 2018 From: martinrb at google.com (Martin Buchholz) Date: Mon, 29 Jan 2018 15:30:42 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: <3E1289BB-6296-4414-BD40-C2CBA72DF527@oracle.com> References: <520081516917760@web45g.yandex.ru> <3E1289BB-6296-4414-BD40-C2CBA72DF527@oracle.com> Message-ID: On Mon, Jan 29, 2018 at 3:10 PM, Paul Sandoz wrote: > > > > On Jan 29, 2018, at 1:02 PM, Martin Buchholz > wrote: > > > > I'm going to expedite this a little since we are building further changes > > on top. > > > > RFR: jsr166 jdk integration 2018-02 > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166- > integration/overview.html > > > > Looks ok, but i personally would not categorize the F/J changes as > miscellaneous :-) > Sorry, we're still working on forkjoin; only the ArrayList changes are ready to go. From paul.sandoz at oracle.com Mon Jan 29 23:39:12 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Mon, 29 Jan 2018 15:39:12 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> <3E1289BB-6296-4414-BD40-C2CBA72DF527@oracle.com> Message-ID: <192FB691-8208-47E2-8C2C-C4B8C9D4202E@oracle.com> > On Jan 29, 2018, at 3:30 PM, Martin Buchholz wrote: > > > > On Mon, Jan 29, 2018 at 3:10 PM, Paul Sandoz > wrote: > > > > On Jan 29, 2018, at 1:02 PM, Martin Buchholz > wrote: > > > > I'm going to expedite this a little since we are building further changes > > on top. > > > > RFR: jsr166 jdk integration 2018-02 > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > > > Looks ok, but i personally would not categorize the F/J changes as miscellaneous :-) > > Sorry, we're still working on forkjoin; Ok, split from the misc changes? > only the ArrayList changes are ready to go. +1 Paul. From john.r.rose at oracle.com Tue Jan 30 00:00:59 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 29 Jan 2018 16:00:59 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> <3E1289BB-6296-4414-BD40-C2CBA72DF527@oracle.com> Message-ID: Reviewed (officially). This is a good point-fix. ? John P.S. I still think we have some tech. debt to discharge in finding a way to generically provide zero-copy access to array data, across interoperating collections APIs. If we got the deeper, more general answer right, we would be able to reformulate the present point-fix in terms of a single viewing operation, and then apply it in other places (like Arrays.asList) without more code duplication. I think (though am not sure yet) that the Collection::stream API point is the right place to inject zero-copy array viewing capabilities. A specialized array-backed Stream (SIZED, of course) would provide a general but efficient connection point, that would allow private arrays to be securely read but not written. Then the various array-backed implementations (and their sublists) would simply override their respective stream views. The copyOfRange step would appear, not in a bunch of point fixes, but centrally in the array-backed stream code. A package-private class could mediate secret access to the backing array, if necessary. See also this discussion about generic array-producing terminal methods, near this message: http://mail.openjdk.java.net/pipermail/core-libs-dev/2017-December/050560.html On Jan 29, 2018, at 3:30 PM, Martin Buchholz wrote: > > > > On Mon, Jan 29, 2018 at 3:10 PM, Paul Sandoz > wrote: > > > > On Jan 29, 2018, at 1:02 PM, Martin Buchholz > wrote: > > > > I'm going to expedite this a little since we are building further changes > > on top. > > > > RFR: jsr166 jdk integration 2018-02 > > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166-integration/overview.html > > > > Looks ok, but i personally would not categorize the F/J changes as miscellaneous :-) > > Sorry, we're still working on forkjoin; only the ArrayList changes are ready to go. From john.r.rose at oracle.com Tue Jan 30 00:09:00 2018 From: john.r.rose at oracle.com (John Rose) Date: Mon, 29 Jan 2018 16:09:00 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> <3E1289BB-6296-4414-BD40-C2CBA72DF527@oracle.com> Message-ID: <97736055-55DE-4A68-8BBE-1FCBC5A595AA@oracle.com> On Jan 29, 2018, at 4:00 PM, John Rose wrote: > > Then the various array-backed implementations > (and their sublists) would simply override their respective > stream views. BTW, this notion is more general than you might think. It applies to collections, like the internal spined buffer, which are backed by sequences of arrays. You use the spliterator of the array-backed stream to find the "cracks" between the backing arrays. After subdividing appropriately, you get Arrays.copyOf and its friends at the leaves of the outer loop. As I said, I *think* this can work, and if it does it will address the issue of duplicate or slow copies more systematically than slowly expanding but incomplete set of point-fixes, like the one under review. From Ulf.Zibis at CoSoCo.de Tue Jan 30 00:10:14 2018 From: Ulf.Zibis at CoSoCo.de (Ulf Zibis) Date: Tue, 30 Jan 2018 01:10:14 +0100 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> Message-ID: <54b64c89-3cfe-c4f6-e343-cb9c42f814d3@CoSoCo.de> Hi, you may can construct the lookup table as a static String constant to slim down the footprint and treat it as a byte[] as we do in the Charset coders. -Ulf Am 29.01.2018 um 22:04 schrieb Claes Redestad: > I'll experiment and analyze a bit more tomorrow to see if I can find a > good explanation for the observed difference and/or a solution that > allows us to slim down the lookup array. From weijun.wang at oracle.com Tue Jan 30 00:22:50 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 30 Jan 2018 08:22:50 +0800 Subject: Cannot add attribute into main attributes of a jar if there is no version In-Reply-To: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> References: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> Message-ID: <8F0CFC56-CCBA-4A3E-8B93-7299820E4F00@oracle.com> Ping again. > On Jan 22, 2018, at 1:12 PM, Weijun Wang wrote: > > src/java.base/share/classes/java/util/jar/Attributes.java: > > 329 @SuppressWarnings("deprecation") > 330 void writeMain(DataOutputStream out) throws IOException > 331 { > 332 // write out the *-Version header first, if it exists > 333 String vername = Name.MANIFEST_VERSION.toString(); > 334 String version = getValue(vername); > 335 if (version == null) { > 336 vername = Name.SIGNATURE_VERSION.toString(); > 337 version = getValue(vername); > 338 } > 339 > 340 if (version != null) { > 341 out.writeBytes(vername+": "+version+"\r\n"); > 342 } > 343 > 344 // write out all attributes except for the version > 345 // we wrote out earlier > 346 for (Entry e : entrySet()) { > 347 String name = ((Name) e.getKey()).toString(); > 348 if ((version != null) && !(name.equalsIgnoreCase(vername))) { > > So, if there is no existing MANIFEST_VERSION or SIGNATURE_VERSION, then version is null and the check above will be false for ever and any other attribute cannot be written out. > > Is this intended? If so, we can exit with an else block after line 342. > > Thanks > Max > > p.s. I am writing a test and notice this. > > 349 > 350 StringBuffer buffer = new StringBuffer(name); > 351 buffer.append(": "); > 352 > 353 String value = (String) e.getValue(); > 354 if (value != null) { > 355 byte[] vb = value.getBytes("UTF8"); > 356 value = new String(vb, 0, 0, vb.length); > 357 } > 358 buffer.append(value); > 359 > 360 buffer.append("\r\n"); > 361 Manifest.make72Safe(buffer); > 362 out.writeBytes(buffer.toString()); > 363 } > 364 } > 365 out.writeBytes("\r\n"); > 366 } > From mandy.chung at oracle.com Tue Jan 30 00:57:51 2018 From: mandy.chung at oracle.com (mandy chung) Date: Mon, 29 Jan 2018 16:57:51 -0800 Subject: Cannot add attribute into main attributes of a jar if there is no version In-Reply-To: <8F0CFC56-CCBA-4A3E-8B93-7299820E4F00@oracle.com> References: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> <8F0CFC56-CCBA-4A3E-8B93-7299820E4F00@oracle.com> Message-ID: <5c562c18-4f9a-2809-a120-2194ab4c3d21@oracle.com> On 1/29/18 4:22 PM, Weijun Wang wrote: > Ping again. > >> On Jan 22, 2018, at 1:12 PM, Weijun Wang wrote: >> >> src/java.base/share/classes/java/util/jar/Attributes.java: >> >> 329 @SuppressWarnings("deprecation") >> 330 void writeMain(DataOutputStream out) throws IOException >> 331 { >> 332 // write out the *-Version header first, if it exists >> 333 String vername = Name.MANIFEST_VERSION.toString(); >> 334 String version = getValue(vername); >> 335 if (version == null) { >> 336 vername = Name.SIGNATURE_VERSION.toString(); >> 337 version = getValue(vername); >> 338 } >> 339 >> 340 if (version != null) { >> 341 out.writeBytes(vername+": "+version+"\r\n"); >> 342 } >> 343 >> 344 // write out all attributes except for the version >> 345 // we wrote out earlier >> 346 for (Entry e : entrySet()) { >> 347 String name = ((Name) e.getKey()).toString(); >> 348 if ((version != null) && !(name.equalsIgnoreCase(vername))) { >> >> So, if there is no existing MANIFEST_VERSION or SIGNATURE_VERSION, then version is null and the check above will be false for ever and any other attribute cannot be written out. >> >> Is this intended? If so, we can exit with an else block after line 342. From code inspection, I agree that this method is a nop if there is no Manifest-Version attribute or Signature-Attribute.? This can return quickly without iterating the entry set.?? I don't see any issue to make it an else block. This method is only called from Manifest::write method which does not mention Signature-Version but I don't have the history to tell if this is intended or not. Mandy From weijun.wang at oracle.com Tue Jan 30 01:44:30 2018 From: weijun.wang at oracle.com (Weijun Wang) Date: Tue, 30 Jan 2018 09:44:30 +0800 Subject: Cannot add attribute into main attributes of a jar if there is no version In-Reply-To: <5c562c18-4f9a-2809-a120-2194ab4c3d21@oracle.com> References: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> <8F0CFC56-CCBA-4A3E-8B93-7299820E4F00@oracle.com> <5c562c18-4f9a-2809-a120-2194ab4c3d21@oracle.com> Message-ID: > On Jan 30, 2018, at 8:57 AM, mandy chung wrote: > > > > On 1/29/18 4:22 PM, Weijun Wang wrote: >> Ping again. >> >> >>> On Jan 22, 2018, at 1:12 PM, Weijun Wang >>> wrote: >>> >>> src/java.base/share/classes/java/util/jar/Attributes.java: >>> >>> 329 @SuppressWarnings("deprecation") >>> 330 void writeMain(DataOutputStream out) throws IOException >>> 331 { >>> 332 // write out the *-Version header first, if it exists >>> 333 String vername = Name.MANIFEST_VERSION.toString(); >>> 334 String version = getValue(vername); >>> 335 if (version == null) { >>> 336 vername = Name.SIGNATURE_VERSION.toString(); >>> 337 version = getValue(vername); >>> 338 } >>> 339 >>> 340 if (version != null) { >>> 341 out.writeBytes(vername+": "+version+"\r\n"); >>> 342 } >>> 343 >>> 344 // write out all attributes except for the version >>> 345 // we wrote out earlier >>> 346 for (Entry e : entrySet()) { >>> 347 String name = ((Name) e.getKey()).toString(); >>> 348 if ((version != null) && !(name.equalsIgnoreCase(vername))) { >>> >>> So, if there is no existing MANIFEST_VERSION or SIGNATURE_VERSION, then version is null and the check above will be false for ever and any other attribute cannot be written out. >>> >>> Is this intended? If so, we can exit with an else block after line 342. >>> > > From code inspection, I agree that this method is a nop if there is no Manifest-Version attribute or Signature-Attribute. This can return quickly without iterating the entry set. I don't see any issue to make it an else block. On the other hand, if this is not intended we should fix line 348 and remove the version != null check. I cannot find a place saying a MANIFEST_VERSION or SIGNATURE_VERSION must be provided. Even if so, this should be an error and it's not a good idea to silently drop all other attributes in the main section. Anyway, I filed https://bugs.openjdk.java.net/browse/JDK-8196371. --Max > > This method is only called from Manifest::write method which does not mention Signature-Version but I don't have the history to tell if this is intended or not. > > Mandy > > From xueming.shen at oracle.com Tue Jan 30 02:59:24 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Mon, 29 Jan 2018 18:59:24 -0800 Subject: Cannot add attribute into main attributes of a jar if there is no version In-Reply-To: <5c562c18-4f9a-2809-a120-2194ab4c3d21@oracle.com> References: <79D7D305-6B0E-4298-8950-2986B54C6F52@oracle.com> <8F0CFC56-CCBA-4A3E-8B93-7299820E4F00@oracle.com> <5c562c18-4f9a-2809-a120-2194ab4c3d21@oracle.com> Message-ID: <5A6FDF8C.3030601@oracle.com> On 1/29/18, 4:57 PM, mandy chung wrote: > > > On 1/29/18 4:22 PM, Weijun Wang wrote: >> Ping again. >> >>> On Jan 22, 2018, at 1:12 PM, Weijun Wang >>> wrote: >>> >>> src/java.base/share/classes/java/util/jar/Attributes.java: >>> >>> 329 @SuppressWarnings("deprecation") >>> 330 void writeMain(DataOutputStream out) throws IOException >>> 331 { >>> 332 // write out the *-Version header first, if it exists >>> 333 String vername = Name.MANIFEST_VERSION.toString(); >>> 334 String version = getValue(vername); >>> 335 if (version == null) { >>> 336 vername = Name.SIGNATURE_VERSION.toString(); >>> 337 version = getValue(vername); >>> 338 } >>> 339 >>> 340 if (version != null) { >>> 341 out.writeBytes(vername+": "+version+"\r\n"); >>> 342 } >>> 343 >>> 344 // write out all attributes except for the version >>> 345 // we wrote out earlier >>> 346 for (Entry e : entrySet()) { >>> 347 String name = ((Name) e.getKey()).toString(); >>> 348 if ((version != null) && >>> !(name.equalsIgnoreCase(vername))) { >>> >>> So, if there is no existing MANIFEST_VERSION or SIGNATURE_VERSION, >>> then version is null and the check above will be false for ever and >>> any other attribute cannot be written out. >>> >>> Is this intended? If so, we can exit with an else block after line 342. > > From code inspection, I agree that this method is a nop if there is no > Manifest-Version attribute or Signature-Attribute. This can return > quickly without iterating the entry set. I don't see any issue to > make it an else block. > > This method is only called from Manifest::write method which does not > mention Signature-Version but I don't have the history to tell if this > is intended or not. I would assume the "intention" here is to do if ((version != null) && name.equalsIgnoreCase(vername))) continue; sherman > > Mandy > > From amaembo at gmail.com Tue Jan 30 04:07:40 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Tue, 30 Jan 2018 11:07:40 +0700 Subject: RFR (JDK11) 8137326: Methods for comparing CharSequence, StringBuilder, and StringBuffer In-Reply-To: <5A6A99B7.1050703@oracle.com> References: <5A6A99B7.1050703@oracle.com> Message-ID: Hello! An AbstractStringBuilder#compareTo implementation is wrong. You cannot simply compare the whole byte array. Here's the test-case: public class Test { public static void main(String[] args) { StringBuilder sb1 = new StringBuilder("test1"); StringBuilder sb2 = new StringBuilder("test2"); sb1.setLength(4); sb2.setLength(4); System.out.println(sb1.compareTo(sb2)); System.out.println(sb1.toString().compareTo(sb2.toString())); } } We truncated the stringbuilders making their content equal, so sb1.toString().compareTo(sb2.toString()) is 0, but compareTo compares the original content (before the truncation) as truncation, of course, does not zero the truncated bytes, neither does it reallocate the array (unless explicitly asked via trimToSize). With best regards, Tagir Valeev. On Fri, Jan 26, 2018 at 10:00 AM, Joe Wang wrote: > Hi, > > Adding methods for comparing CharSequence, StringBuilder, and StringBuffer. > > The Comparable implementations for StringBuilder/Buffer are similar to that > of String, allowing comparison operations between two > StringBuilders/Buffers, e.g. aStringBuilder.compareTo(anotherStringBuilder). > For CharSequence however, refer to the comments in JIRA, a static method > 'compare' is added instead of implementing the Comparable interface. This > 'compare' method may take CharSequence implementations such as String, > StringBuilder and StringBuffer, making it possible to perform comparison > among them. The previous example for example is equivalent to > CharSequence.compare(aStringBuilder, anotherStringBuilder). > > Tests for java.base have been independent from each other. The new tests are > therefore created to have no dependency on each other or sharing any code. > > JBS: https://bugs.openjdk.java.net/browse/JDK-8137326 > webrev: http://cr.openjdk.java.net/~joehw/jdk11/8137326/webrev/ > > Thanks, > Joe From srinivas.dama at oracle.com Tue Jan 30 09:17:21 2018 From: srinivas.dama at oracle.com (Srinivas Dama) Date: Tue, 30 Jan 2018 01:17:21 -0800 (PST) Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) In-Reply-To: <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> References: <3558275d-6cd3-47bc-8203-da263835f747@default> <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> Message-ID: <31fd76f5-fba8-418e-9713-e0ae5b5bc601@default> Hi, Please review the revised webrev at http://cr.openjdk.java.net/~sdama/8011697/webrev.01/ for https://bugs.openjdk.java.net/browse/JDK-8011697 Regards, Srinivas -----Original Message----- From: Alan Bateman Sent: Monday, December 11, 2017 10:05 PM To: Srinivas Dama ; core-libs-dev at openjdk.java.net Subject: Re: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) On 11/12/2017 15:26, Srinivas Dama wrote: > Hi, > > Please review http://cr.openjdk.java.net/~sdama/8011697/webrev.00/ > for https://bugs.openjdk.java.net/browse/JDK-8011697 > > Fix is to make sure ScriptEngineManager always returns a particular engine on all platforms consistently. > I assume using a TreeSet would work too. A different approach is replace engineSpis with nameToFactory and extensionToFactory maps so that the getEngineByXXX methods don't need to a sequential search. -Alan From claes.redestad at oracle.com Tue Jan 30 10:12:58 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 30 Jan 2018 11:12:58 +0100 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: <54b64c89-3cfe-c4f6-e343-cb9c42f814d3@CoSoCo.de> References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> <54b64c89-3cfe-c4f6-e343-cb9c42f814d3@CoSoCo.de> Message-ID: <3bb16c35-183a-6292-d7a6-7b8b73606969@oracle.com> Hi, not sure what you're suggesting, exactly, but it seems Charset coders generate String literals which are then unpacked to char[]/byte[]'s in static blocks. While constructing arrays from String literals might have been a startup optimization in the past, it is now mainly a workaround to method bytecode limits, see https://bugs.openjdk.java.net/browse/JDK-8185104 /Claes On 2018-01-30 01:10, Ulf Zibis wrote: > Hi, > > you may can construct the lookup table as a static String constant to > slim down the footprint and treat it as a byte[] as we do in the > Charset coders. > > -Ulf > > > Am 29.01.2018 um 22:04 schrieb Claes Redestad: >> I'll experiment and analyze a bit more tomorrow to see if I can find a >> good explanation for the observed difference and/or a solution that >> allows us to slim down the lookup array. > From chris.hegarty at oracle.com Tue Jan 30 10:21:18 2018 From: chris.hegarty at oracle.com (Chris Hegarty) Date: Tue, 30 Jan 2018 10:21:18 +0000 Subject: [8u-dev] RFR - 8156824: com.sun.jndi.ldap.pool.PoolCleaner should clear its context class loader In-Reply-To: <20180126164023.GA6402@vimes> References: <20180126164023.GA6402@vimes> Message-ID: <43709FF2-EF2F-4118-89CC-B8D601E74E56@oracle.com> > On 26 Jan 2018, at 16:40, Rob McKenna wrote: > > Hi folks, > > I'd like to backport this change to 8u-dev. > > The changes are straightforward enough but 8u needs some changes that > were made to InnocuousThread. (strictly it doesn't need all of the > changes I've made but I've made an effort to make the code look as close > to 9 as possible) > > bug: https://bugs.openjdk.java.net/browse/JDK-8156824 > 9u changeset: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/22e704dfa05c > 8u webrev: http://cr.openjdk.java.net/~robm/8156824/webrev.01/ The changes looks ok to me. Thanks Rob. -Chris. From claes.redestad at oracle.com Tue Jan 30 10:40:02 2018 From: claes.redestad at oracle.com (Claes Redestad) Date: Tue, 30 Jan 2018 11:40:02 +0100 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> Message-ID: The ASM is harder than usual to follow and compare since everything is inlined aggressively, but it seems that since CharacterDataLatin1 is only invoked for 0 <= ch < 256 (invariant established in CharacterData.of(int ch)) then the compiler is able to elide bounds check entirely when the byte[] is also 256 elements. Shrinking the array adds more branches and grows the compiled code size for UUID.fromString from 751 to 1341 bytes, so it seems that even from a footprint perspective then keeping the array at 256 elements is a win. :-) /Claes On 2018-01-29 22:04, Claes Redestad wrote: > Right, I can't really explain why, but the effect is very visible and > reproducible in microbenchmarks. Differences in generated ASM might > be indicating that the inlining behavior changes enough to shift the > result around; maybe a job for @ForceInline? > > I'll experiment and analyze a bit more tomorrow to see if I can find a > good explanation for the observed difference and/or a solution that > allows us to slim down the lookup array. > > /Claes > > On 2018-01-29 20:38, Paul Sandoz wrote: >> Smaller in only the upper bound? I would an explicit upper bounds >> check would dominate that of the bounds check for the array itself. >> >> Paul. >> >>> On Jan 29, 2018, at 11:37 AM, Claes Redestad >>> > wrote: >>> >>> I ran with a smaller byte[] initially and saw about a 10% >>> improvement from removing the branch, which is why I felt the >>> superfluous bytes were motivated. >>> >>> /Claes >>> >>> Paul Sandoz > >>> skrev: (29 januari 2018 19:14:44 CET) >>> >>> >>> ??????? On Jan 29, 2018, at 9:44 AM, Martin Buchholz >>> ??????? > wrote: >>> ??????? Thanks. I might try to shrink the size of the static array, >>> ??????? by only storing values between '0' and 'z', at the cost of >>> ??????? slightly greater lookup costs for each char. >>> >>> ??? I was wondering the same, or just clip the end of the array to?z' >>> >>> ??? if (ch <= ?z? && radix ?) { // Might even fold the upper bounds >>> check for DIGITS >>> ?????? value = DIGITS[ch]; >>> ?????? ... >>> ??? } >>> >>> ??? Paul. >>> >>> ??????? On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad >>> ??????? >> ??????? > wrote: >>> >>> ??????????? Hi, for the latin1 block of CharacterData we can improve >>> ??????????? the digit(int, int) implementation by providing an >>> ??????????? optimized lookup table. This improves microbenchmarks >>> ??????????? exercising Integer.parseInt, Long.parseLong and >>> ??????????? UUID.fromString etc by around 50%for typical inputs. >>> ??????????? Webrev: >>> http://cr.openjdk.java.net/~redestad/8196331/open.00/ >>> >>> ??????????? Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 The >>> ??????????? lookup array is pre-calculated to minimize startup impact >>> ??????????? (adds 1,027 bytecodes executed during initialization) >>> /Claes >>> >>> >>> -- >>> Sent from my Android device with K-9 Mail. Please excuse my brevity. >> > From ben_walsh at uk.ibm.com Tue Jan 30 13:55:14 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Tue, 30 Jan 2018 13:55:14 +0000 Subject: [PATCH] GPU Exploitation Infrastructure Message-ID: This patch provides the infrastructure to enable the exploitation of a GPU by the compiler to accelerate certain suitable Java constructs. When enabled, a suitable compiler can attempt to accelerate the following Java constructs by launching the corresponding lambda expression on the GPU: IntStream.range().parallel().forEach() IntStream.rangeClosed().parallel().forEach() where: defines upper and lower bounds is a correctly defined lambda expression As it stands, with the HotSpot compiler, this patch performs a "no op" in the newly added in-built native library method. This can be extended so that the HotSpot compiler attempts the acceleration detailed above instead. I would like to pair with a sponsor to contribute this patch ... -------------------------------------------------------------------------------------------------------------- diff -r fd237da7a113 make/hotspot/symbols/symbols-unix --- a/make/hotspot/symbols/symbols-unix Mon Jan 22 23:06:29 2018 -0800 +++ b/make/hotspot/symbols/symbols-unix Tue Jan 30 10:07:18 2018 +0000 @@ -1,5 +1,5 @@ # -# Copyright (c) 2016, 2017, Oracle and/or its affiliates. All rights reserved. +# Copyright (c) 2016, 2018, Oracle and/or its affiliates. All rights reserved. # DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. # # This code is free software; you can redistribute it and/or modify it @@ -187,3 +187,6 @@ JVM_AddReadsModule JVM_DefineModule JVM_SetBootLoaderUnnamedModule + +# GPU exploitation support +Java_java_util_stream_IntPipeline_promoteGPUCompile diff -r fd237da7a113 src/hotspot/share/include/jvm.h --- a/src/hotspot/share/include/jvm.h Mon Jan 22 23:06:29 2018 -0800 +++ b/src/hotspot/share/include/jvm.h Tue Jan 30 10:07:18 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1997, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -1189,6 +1189,12 @@ JNIEXPORT jstring JNICALL JVM_GetTemporaryDirectory(JNIEnv *env); +/* + * GPU exploitation support + */ +JNIEXPORT void JNICALL +Java_java_util_stream_IntPipeline_promoteGPUCompile(JNIEnv *env, jobject obj); + /* Generics reflection support. * * Returns information about the given class's EnclosingMethod diff -r fd237da7a113 src/hotspot/share/prims/jvm.cpp --- a/src/hotspot/share/prims/jvm.cpp Mon Jan 22 23:06:29 2018 -0800 +++ b/src/hotspot/share/prims/jvm.cpp Tue Jan 30 10:07:18 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 1997, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 1997, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -3661,3 +3661,11 @@ JVM_ENTRY_NO_ENV(jint, JVM_FindSignal(const char *name)) return os::get_signal_number(name); JVM_END + + +// GPU exploitation support stub ///////////////////////////////////////////////////////////////////// + +JVM_ENTRY(void, Java_java_util_stream_IntPipeline_promoteGPUCompile(JNIEnv *env, jobject thisObj)) + JVMWrapper("Java_java_util_stream_IntPipeline_promoteGPUCompile"); + return; +JVM_END diff -r fd237da7a113 src/java.base/share/classes/java/util/stream/AbstractPipeline.java --- a/src/java.base/share/classes/java/util/stream/AbstractPipeline.java Mon Jan 22 23:06:29 2018 -0800 +++ b/src/java.base/share/classes/java/util/stream/AbstractPipeline.java Tue Jan 30 10:07:18 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2012, 2016, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -373,6 +373,14 @@ return sourceStage.parallel; } + /** + * Returns the sourceSpliterator + * + * @return the sourceSpliterator + */ + final Spliterator getSourceSpliterator() { + return sourceSpliterator; + } /** * Returns the composition of stream flags of the stream source and all diff -r fd237da7a113 src/java.base/share/classes/java/util/stream/IntPipeline.java --- a/src/java.base/share/classes/java/util/stream/IntPipeline.java Mon Jan 22 23:06:29 2018 -0800 +++ b/src/java.base/share/classes/java/util/stream/IntPipeline.java Tue Jan 30 10:07:18 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2012, 2017, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -433,10 +433,33 @@ } // Terminal ops from IntStream + private static boolean tryGPU = false; + protected static native void promoteGPUCompile(); @Override public void forEach(IntConsumer action) { - evaluate(ForEachOps.makeInt(action, false)); + boolean instanceOfRangeIntSpliterator = getSourceSpliterator() instanceof Streams.RangeIntSpliterator; + if (instanceOfRangeIntSpliterator) { + Streams.RangeIntSpliterator intRange = (Streams.RangeIntSpliterator)getSourceSpliterator(); + int last = intRange.upTo; + if (intRange.upTo != Integer.MAX_VALUE && intRange.last == 1){ + last = intRange.upTo + 1; + } + // tryGPU will be set at runtime using a suitable JVM specific method option + if (tryGPU) { + for (int i = intRange.from; i < last; i++){ + action.accept(i); + } + if (intRange.upTo == Integer.MAX_VALUE && intRange.last == 1){ + action.accept(Integer.MAX_VALUE); + } + return; + } + } + evaluate(ForEachOps.makeInt(action, false)); + if (instanceOfRangeIntSpliterator){ + promoteGPUCompile(); + } } @Override diff -r fd237da7a113 src/java.base/share/classes/java/util/stream/Streams.java --- a/src/java.base/share/classes/java/util/stream/Streams.java Mon Jan 22 23:06:29 2018 -0800 +++ b/src/java.base/share/classes/java/util/stream/Streams.java Tue Jan 30 10:07:18 2018 +0000 @@ -1,5 +1,5 @@ /* - * Copyright (c) 2012, 2013, Oracle and/or its affiliates. All rights reserved. + * Copyright (c) 2012, 2018, Oracle and/or its affiliates. All rights reserved. * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER. * * This code is free software; you can redistribute it and/or modify it @@ -55,12 +55,12 @@ // Can never be greater that upTo, this avoids overflow if upper bound // is Integer.MAX_VALUE // All elements are traversed if from == upTo & last == 0 - private int from; - private final int upTo; + int from; + final int upTo; // 1 if the range is closed and the last element has not been traversed // Otherwise, 0 if the range is open, or is a closed range and all // elements have been traversed - private int last; + int last; RangeIntSpliterator(int from, int upTo, boolean closed) { this(from, upTo, closed ? 1 : 0); -------------------------------------------------------------------------------------------------------------- Regards, Ben Walsh Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Alan.Bateman at oracle.com Tue Jan 30 14:31:31 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 30 Jan 2018 14:31:31 +0000 Subject: RFR JDK-8176379: java.util.Base64 mime encoder behaves incorrectly if initialized with a line length of size 1-3 In-Reply-To: <5A6F5F6E.3020509@oracle.com> References: <5A6BBA7E.3090001@oracle.com> <6DB12EB2-4A10-4091-A811-FCEA8B2B67BB@oracle.com> <5A6F5F6E.3020509@oracle.com> Message-ID: On 29/01/2018 17:52, Xueming Shen wrote: > > Actually I have a dedicated test case for "mime encoder maxlen". > *test/jdk/java/util/Base64/Base64GetEncoderTest.java* > I have updated it to test the negative and the < 4 maxlen. > > http://cr.openjdk.java.net/~sherman/8176379/webrev/ > > webrev has been updated according. Looks good. -Alan From Alan.Bateman at oracle.com Tue Jan 30 14:58:49 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 30 Jan 2018 14:58:49 +0000 Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) In-Reply-To: <31fd76f5-fba8-418e-9713-e0ae5b5bc601@default> References: <3558275d-6cd3-47bc-8203-da263835f747@default> <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> <31fd76f5-fba8-418e-9713-e0ae5b5bc601@default> Message-ID: <960de74d-0b48-2c1d-82ce-85cf7bf00d21@oracle.com> On 30/01/2018 09:17, Srinivas Dama wrote: > Hi, > > Please review the revised webrev at http://cr.openjdk.java.net/~sdama/8011697/webrev.01/ for https://bugs.openjdk.java.net/browse/JDK-8011697 > > The updated patch to ScriptEngineManager looks okay. Can you replace the copyright header in the tests with the GPL version before you push this? We don't use the GPL + "Classpath" exception header in tests. -Alan From Alan.Bateman at oracle.com Tue Jan 30 15:02:22 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 30 Jan 2018 15:02:22 +0000 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: <01CCBA7D-1815-4A78-A803-A9D18E437B34@oracle.com> References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> <01CCBA7D-1815-4A78-A803-A9D18E437B34@oracle.com> Message-ID: <0be48e06-637c-eb04-bb06-0bf0524494f9@oracle.com> On 29/01/2018 22:55, Brian Burkhalter wrote: > I made a few minor changes to the CSR [1] verbiage. Probably the nullWriter() documentation should also be updated to explicitly mention all variants of both append() and write(). > It could although limiting it to the abstract methods should be fine too as the append methods are specified in terms of the write methods. One other micro detail is that Reader/Writer support implementations specifying the object to use for synchronization. This isn't interesting for the Reader/Writer objects returned by the proposed static methods so a one-line that says that the object used to synchronize operations is not specified might help. It would avoid something synchronizing on the reader or writer and expecting operations read/write methods to block. -Alan From xueming.shen at oracle.com Tue Jan 30 15:00:26 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 30 Jan 2018 07:00:26 -0800 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> Message-ID: +1 On Jan 30, 2018, at 2:40 AM, Claes Redestad wrote: The ASM is harder than usual to follow and compare since everything is inlined aggressively, but it seems that since CharacterDataLatin1 is only invoked for 0 <= ch < 256 (invariant established in CharacterData.of(int ch)) then the compiler is able to elide bounds check entirely when the byte[] is also 256 elements. Shrinking the array adds more branches and grows the compiled code size for UUID.fromString from 751 to 1341 bytes, so it seems that even from a footprint perspective then keeping the array at 256 elements is a win. :-) /Claes > On 2018-01-29 22:04, Claes Redestad wrote: > Right, I can't really explain why, but the effect is very visible and > reproducible in microbenchmarks. Differences in generated ASM might > be indicating that the inlining behavior changes enough to shift the > result around; maybe a job for @ForceInline? > > I'll experiment and analyze a bit more tomorrow to see if I can find a > good explanation for the observed difference and/or a solution that > allows us to slim down the lookup array. > > /Claes > >> On 2018-01-29 20:38, Paul Sandoz wrote: >> Smaller in only the upper bound? I would an explicit upper bounds check would dominate that of the bounds check for the array itself. >> >> Paul. >> >>> On Jan 29, 2018, at 11:37 AM, Claes Redestad > wrote: >>> >>> I ran with a smaller byte[] initially and saw about a 10% improvement from removing the branch, which is why I felt the superfluous bytes were motivated. >>> >>> /Claes >>> >>> Paul Sandoz > skrev: (29 januari 2018 19:14:44 CET) >>> >>> >>> On Jan 29, 2018, at 9:44 AM, Martin Buchholz >>> > wrote: >>> Thanks. I might try to shrink the size of the static array, >>> by only storing values between '0' and 'z', at the cost of >>> slightly greater lookup costs for each char. >>> >>> I was wondering the same, or just clip the end of the array to?z' >>> >>> if (ch <= ?z? && radix ?) { // Might even fold the upper bounds check for DIGITS >>> value = DIGITS[ch]; >>> ... >>> } >>> >>> Paul. >>> >>> On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad >>> >> > wrote: >>> >>> Hi, for the latin1 block of CharacterData we can improve >>> the digit(int, int) implementation by providing an >>> optimized lookup table. This improves microbenchmarks >>> exercising Integer.parseInt, Long.parseLong and >>> UUID.fromString etc by around 50%for typical inputs. >>> Webrev: >>> http://cr.openjdk.java.net/~redestad/8196331/open.00/ >>> >>> Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 The >>> lookup array is pre-calculated to minimize startup impact >>> (adds 1,027 bytecodes executed during initialization) /Claes >>> >>> >>> -- >>> Sent from my Android device with K-9 Mail. Please excuse my brevity. >> > From Roger.Riggs at Oracle.com Tue Jan 30 15:05:24 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 30 Jan 2018 10:05:24 -0500 Subject: RFR JDK-8176379: java.util.Base64 mime encoder behaves incorrectly if initialized with a line length of size 1-3 In-Reply-To: References: <5A6BBA7E.3090001@oracle.com> <6DB12EB2-4A10-4091-A811-FCEA8B2B67BB@oracle.com> <5A6F5F6E.3020509@oracle.com> Message-ID: Hi, Base64GetEncoderTest does not need to test the negative values more than once. Otherwise, looks fine. Roger On 1/30/2018 9:31 AM, Alan Bateman wrote: > On 29/01/2018 17:52, Xueming Shen wrote: >> >> Actually I have a dedicated test case for "mime encoder maxlen". >> *test/jdk/java/util/Base64/Base64GetEncoderTest.java* >> I have updated it to test the negative and the < 4 maxlen. >> >> http://cr.openjdk.java.net/~sherman/8176379/webrev/ >> >> webrev has been updated according. > Looks good. > > -Alan From sundararajan.athijegannathan at oracle.com Tue Jan 30 15:22:50 2018 From: sundararajan.athijegannathan at oracle.com (Sundararajan Athijegannathan) Date: Tue, 30 Jan 2018 20:52:50 +0530 Subject: RFR: 8011697(ScriptEngine "js" randomly means either "rhino" or "nashorn", but should instead select one) In-Reply-To: <960de74d-0b48-2c1d-82ce-85cf7bf00d21@oracle.com> References: <3558275d-6cd3-47bc-8203-da263835f747@default> <86f58aa7-2805-7c4b-4130-da1b520e47da@oracle.com> <31fd76f5-fba8-418e-9713-e0ae5b5bc601@default> <960de74d-0b48-2c1d-82ce-85cf7bf00d21@oracle.com> Message-ID: <5A708DCA.5030006@oracle.com> +1 (with the copyright header change suggested by Alan) -Sundar On 30/01/18, 8:28 PM, Alan Bateman wrote: > On 30/01/2018 09:17, Srinivas Dama wrote: >> Hi, >> >> Please review the revised webrev at >> http://cr.openjdk.java.net/~sdama/8011697/webrev.01/ for >> https://bugs.openjdk.java.net/browse/JDK-8011697 >> >> > The updated patch to ScriptEngineManager looks okay. > > Can you replace the copyright header in the tests with the GPL version > before you push this? We don't use the GPL + "Classpath" exception > header in tests. > > -Alan From Alan.Bateman at oracle.com Tue Jan 30 15:35:21 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Tue, 30 Jan 2018 15:35:21 +0000 Subject: [PATCH] GPU Exploitation Infrastructure In-Reply-To: References: Message-ID: <77c394b7-2105-1145-b26d-d53718cda955@oracle.com> On 30/01/2018 13:55, Ben Walsh wrote: > This patch provides the infrastructure to enable the exploitation of a GPU > by the compiler to accelerate certain suitable Java constructs. > > When enabled, a suitable compiler can attempt to accelerate the following > Java constructs by launching the corresponding lambda expression on the > GPU: > > IntStream.range().parallel().forEach() > IntStream.rangeClosed().parallel().forEach() > > where: > > defines upper and lower bounds > is a correctly defined lambda expression > > As it stands, with the HotSpot compiler, this patch performs a "no op" in > the newly added in-built native library method. > This can be extended so that the HotSpot compiler attempts the > acceleration detailed above instead. > > I would like to pair with a sponsor to contribute this patch ... The function prototypes in jvm.h are JVM_* functions rather than JNI native method names. You may want to look at adding a JNI function to libjava with a wrapper or else use RegisterNatives to map the native method name to a JVM_* function. BTW: Is this something for one the project repos (Valhalla or Panama) rather the main line (jdk/jdk)? -Alan From paul.sandoz at oracle.com Tue Jan 30 16:29:13 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Tue, 30 Jan 2018 08:29:13 -0800 Subject: RFR: 8196331: Optimize Character.digit for latin1 input In-Reply-To: References: <8335FA18-55E9-4E7E-B065-1DF422BC5B4A@oracle.com> <3542BF87-039B-461D-9696-FDCB0858EC31@oracle.com> Message-ID: <129B8086-BF87-4436-9997-B5CE8E5A64DE@oracle.com> OK! A surprising increase in code size. Thanks for checking. Paul. > On Jan 30, 2018, at 2:40 AM, Claes Redestad wrote: > > The ASM is harder than usual to follow and compare since everything is > inlined aggressively, but it seems that since CharacterDataLatin1 is only > invoked for 0 <= ch < 256 (invariant established in CharacterData.of(int ch)) > then the compiler is able to elide bounds check entirely when the byte[] is > also 256 elements. > > Shrinking the array adds more branches and grows the compiled code size > for UUID.fromString from 751 to 1341 bytes, so it seems that even from a > footprint perspective then keeping the array at 256 elements is a win. :-) > > /Claes > > On 2018-01-29 22:04, Claes Redestad wrote: >> Right, I can't really explain why, but the effect is very visible and >> reproducible in microbenchmarks. Differences in generated ASM might >> be indicating that the inlining behavior changes enough to shift the >> result around; maybe a job for @ForceInline? >> >> I'll experiment and analyze a bit more tomorrow to see if I can find a >> good explanation for the observed difference and/or a solution that >> allows us to slim down the lookup array. >> >> /Claes >> >> On 2018-01-29 20:38, Paul Sandoz wrote: >>> Smaller in only the upper bound? I would an explicit upper bounds check would dominate that of the bounds check for the array itself. >>> >>> Paul. >>> >>>> On Jan 29, 2018, at 11:37 AM, Claes Redestad > wrote: >>>> >>>> I ran with a smaller byte[] initially and saw about a 10% improvement from removing the branch, which is why I felt the superfluous bytes were motivated. >>>> >>>> /Claes >>>> >>>> Paul Sandoz > skrev: (29 januari 2018 19:14:44 CET) >>>> >>>> >>>> On Jan 29, 2018, at 9:44 AM, Martin Buchholz >>>> > wrote: >>>> Thanks. I might try to shrink the size of the static array, >>>> by only storing values between '0' and 'z', at the cost of >>>> slightly greater lookup costs for each char. >>>> >>>> I was wondering the same, or just clip the end of the array to?z' >>>> >>>> if (ch <= ?z? && radix ?) { // Might even fold the upper bounds check for DIGITS >>>> value = DIGITS[ch]; >>>> ... >>>> } >>>> >>>> Paul. >>>> >>>> On Mon, Jan 29, 2018 at 3:15 AM, Claes Redestad >>>> >>> > wrote: >>>> >>>> Hi, for the latin1 block of CharacterData we can improve >>>> the digit(int, int) implementation by providing an >>>> optimized lookup table. This improves microbenchmarks >>>> exercising Integer.parseInt, Long.parseLong and >>>> UUID.fromString etc by around 50%for typical inputs. >>>> Webrev: >>>> http://cr.openjdk.java.net/~redestad/8196331/open.00/ >>>> >>>> Bug: https://bugs.openjdk.java.net/browse/JDK-8196331 The >>>> lookup array is pre-calculated to minimize startup impact >>>> (adds 1,027 bytecodes executed during initialization) /Claes >>>> >>>> >>>> -- >>>> Sent from my Android device with K-9 Mail. Please excuse my brevity. >>> >> > From adam.petcher at oracle.com Tue Jan 30 16:52:01 2018 From: adam.petcher at oracle.com (Adam Petcher) Date: Tue, 30 Jan 2018 11:52:01 -0500 Subject: RFR 8181594: Efficient and constant-time modular arithmetic In-Reply-To: References: Message-ID: <7bd6c1ae-4173-4384-fc57-66d1b185ba56@oracle.com> +core-libs-dev On 1/26/2018 4:06 PM, Adam Petcher wrote: > JBS: https://bugs.openjdk.java.net/browse/JDK-8181594 > Webrev: http://cr.openjdk.java.net/~apetcher/8181594/webrev.00/ > > This is a code review for the field arithmetic that will be used in > implementations of X25519/X448 key agreement, the Poly1305 > authenticator, and EdDSA signatures. I believe that the library has > all the features necessary for X25519/X448 and Poly1305, and I expect > at most a couple of minor enhancements will be required to support > EdDSA. There is no public API for this library, so we can change it in > the future to suit the needs of new algorithms without breaking > compatibility with external code. Still, I made an attempt to clearly > structure and document the (internal) API, and I want to make sure it > is understandable and easy to use. > > This is not a general-purpose modular arithmetic library. It will only > work well in circumstances where the sequence of operations is > restricted, and where the prime that defines the field has some useful > structure. Moreover, each new field will require some field-specific > code that takes into account the structure of the prime and the way > the field is used in the application. The initial implementation > includes a field for Poly1305 and the fields for X25519/X448 which > should also work for EdDSA. > > The benefits of using this library are that it is much more efficient > than using similar operations in BigInteger. Also, many operations are > branch-free, making them suitable for use in a side-channel resistant > implementation that does not branch on secrets. > > To provide some context, I have attached a code snippet describing how > this library can be used. The snippet is the constant-time Montgomery > ladder from my X25519/X448 implementation, which I expect to be out > for review soon. X25519/X448 only uses standard arithmetic operations, > and the more unusual features (e.g. add modulo a power of 2) are > needed by Poly1305. > > The field arithmetic (for all fields) is implemented using a 32-bit > representation similar to the one described in the Ed448 paper[1] (in > the "Implementation on 32-bit platforms" section). Though my > implementation uses signed limbs, and grade-school multiplication > instead of Karatsuba. The argument for correctness is essentially the > same for all three fields: the magnitude of each 64-bit limb is at > most 2^(k-1) after reduction, except for the last limb which may have > a magnitude of up to 2^k. The values of k are between 26 to 28 > (depending on the field), and we can calculate that the maximum > magnitude for any limb during an add-multiply-carry-reduce sequence is > always less than 2^63. Therefore, no overflow occurs and all > operations are correct. > > Process note: this enhancement is part of JEP 324 (Key Agreement with > Curve25519 and Curve448). When this code review is complete, nothing > will happen until all other work for this JEP is complete, and the JEP > is accepted as part of some release. This means that this code will be > pushed to the repo along with the X25519/X448 code that uses it. > > [1] https://eprint.iacr.org/2015/625.pdf > > > -------------- next part -------------- private IntegerModuloP_Base pointMultiply(byte[] k, IntegerModuloP u){ IntegerModuloP x_1 = u; MutableIntegerModuloP x_2 = one.mutable(); MutableIntegerModuloP z_2 = zero.mutable(); MutableIntegerModuloP x_3 = u.mutable(); MutableIntegerModuloP z_3 = one.mutable(); int swap = 0; // Variables below are reused to avoid unnecessary allocation // They will be assigned in the loop, so initial value doesn't matter MutableIntegerModuloP m1 = zero.mutable(); MutableIntegerModuloP DA = zero.mutable(); MutableIntegerModuloP E = zero.mutable(); MutableIntegerModuloP a24_times_E = zero.mutable(); for(int t = params.getBits() - 1; t >= 0; t--){ int k_t = bitAt(k, t); swap = swap ^ k_t; x_2.conditionalSwapWith(x_3, swap); z_2.conditionalSwapWith(z_3, swap); swap = k_t; // A(m1) = x_2 + z_2 m1.setValue(x_2).setSum(z_2); // D = x_3 - z_3 // DA = D * A(m1) DA.setValue(x_3).setDifference(z_3).setProduct(m1); // AA(m1) = A(m1)^2 m1.setSquare(); // B(x_2) = x_2 - z_2 x_2.setDifference(z_2); // C = x_3 + z_3 // CB(x_3) = C * B(x_2) x_3.setSum(z_3).setProduct(x_2); // BB(x_2) = B^2 x_2.setSquare(); // E = AA(m1) - BB(x_2) E.setValue(m1).setDifference(x_2); // compute a24 * E using SmallValue a24_times_E.setValue(E); a24_times_E.setProduct(a24); // assign results to x_3, z_3, x_2, z_2 // x_2 = AA(m1) * BB x_2.setProduct(m1); // z_2 = E * (AA(m1) + a24 * E) z_2.setValue(m1).setSum(a24_times_E).setProduct(E); // z_3 = x_1*(DA - CB(x_3))^2 z_3.setValue(DA).setDifference(x_3).setSquare().setProduct(x_1); // x_3 = (CB(x_3) + DA)^2 x_3.setSum(DA).setSquare(); } x_2.conditionalSwapWith(x_3, swap); z_2.conditionalSwapWith(z_3, swap); // return (x_2 * z_2^(p - 2)) return x_2.setProduct(z_2.multiplicativeInverse()); } From patrick at reini.net Tue Jan 30 16:52:01 2018 From: patrick at reini.net (Patrick Reinhart) Date: Tue, 30 Jan 2018 17:52:01 +0100 Subject: JDK-8196298: Add null Reader and Writer In-Reply-To: <0be48e06-637c-eb04-bb06-0bf0524494f9@oracle.com> References: <3cd6731c-3e52-e24e-3720-327cbbbe7b47@reini.net> <4e669f74-10d9-b9b2-ca64-805003523d88@reini.net> <29989B8B-48AB-408A-A819-F1A099A71D9A@oracle.com> <01CCBA7D-1815-4A78-A803-A9D18E437B34@oracle.com> <0be48e06-637c-eb04-bb06-0bf0524494f9@oracle.com> Message-ID: <33FEB947-39BD-41FE-B28F-2536E6BD687B@reini.net> > Am 30.01.2018 um 16:02 schrieb Alan Bateman : > > On 29/01/2018 22:55, Brian Burkhalter wrote: >> I made a few minor changes to the CSR [1] verbiage. Probably the nullWriter() documentation should also be updated to explicitly mention all variants of both append() and write(). >> > It could although limiting it to the abstract methods should be fine too as the append methods are specified in terms of the write methods. > > One other micro detail is that Reader/Writer support implementations specifying the object to use for synchronization. This isn't interesting for the Reader/Writer objects returned by the proposed static methods so a one-line that says that the object used to synchronize operations is not specified might help. It would avoid something synchronizing on the reader or writer and expecting operations read/write methods to block Is that a difference from the null InputStream/OutputStream here? -Patrick From martinrb at google.com Tue Jan 30 19:33:32 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jan 2018 11:33:32 -0800 Subject: [PATCH] Inefficient ArrayList.subList().toArray() In-Reply-To: References: <520081516917760@web45g.yandex.ru> Message-ID: On Mon, Jan 29, 2018 at 1:02 PM, Martin Buchholz wrote: > I'm going to expedite this a little since we are building further changes > on top. > > 8196207: Inefficient ArrayList.subList().toArray() > http://cr.openjdk.java.net/~martin/webrevs/jdk/jsr166- > integration/ArrayList-subList-toArray/index.html > https://bugs.openjdk.java.net/browse/JDK-8196207 > 8196207 was pushed, ready for Claes or jsr166 team to add to. From david.holmes at oracle.com Tue Jan 30 21:46:06 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 31 Jan 2018 07:46:06 +1000 Subject: [PATCH] GPU Exploitation Infrastructure In-Reply-To: <77c394b7-2105-1145-b26d-d53718cda955@oracle.com> References: <77c394b7-2105-1145-b26d-d53718cda955@oracle.com> Message-ID: On 31/01/2018 1:35 AM, Alan Bateman wrote: > On 30/01/2018 13:55, Ben Walsh wrote: >> This patch provides the infrastructure to enable the exploitation of a >> GPU >> by the compiler to accelerate certain suitable Java constructs. >> >> When enabled, a suitable compiler can attempt to accelerate the following >> Java constructs by launching the corresponding lambda expression on the >> GPU: >> >> ??? IntStream.range().parallel().forEach() >> ??? IntStream.rangeClosed().parallel().forEach() >> >> ??? where: >> >> ?????? defines upper and lower bounds >> ?????? is a correctly defined lambda expression >> >> As it stands, with the HotSpot compiler, this patch performs a "no op" in >> the newly added in-built native library method. >> This can be extended so that the HotSpot compiler attempts the >> acceleration detailed above instead. >> >> I would like to pair with a sponsor to contribute this patch ... > The function prototypes in jvm.h are JVM_* functions rather than JNI > native method names. You may want to look at adding a JNI function to > libjava with a wrapper or else use RegisterNatives to map the native > method name to a JVM_* function. > > BTW: Is this something for one the project repos (Valhalla or Panama) > rather the main line (jdk/jdk)? This should be looked at in the context of Panama I think, though IIRC there was a distinct GPU related project as well? Either way GPU support in mainline doesn't seem like something to just be dropped in like this without much more higher-level discussion taking place. Plus hotspot changes - even stubs - must be reviewed on the appropriate hotspot mailing list. David > -Alan From Roger.Riggs at Oracle.com Tue Jan 30 22:08:49 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Tue, 30 Jan 2018 17:08:49 -0500 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner Message-ID: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> Please review changes to replace finalizers in socket, datagram, and multicast networking with Cleaner based release of the raw file descriptors.? Each FileDescriptor is registered for cleanup after the raw fd (or handle) is assigned.? Normal calls to close unregister the cleaner before using the current logic to close the raw fd/handle. Windows networking uses fd's with the Windows socket_ API requiring a special cased Cleanable. The tests check that the implementation objects including FileDescriptors are reclaimed and for Linux that the raw fd counts are reduced as expected. Webrev: ?? http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ Thanks, Roger From xueming.shen at oracle.com Tue Jan 30 22:59:56 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Tue, 30 Jan 2018 14:59:56 -0800 Subject: JDK-8166339, Code conversion working behavior was changed for x-IBM834 Message-ID: <5A70F8EC.4010801@oracle.com> Hi, Please help review the change for JDK-8166339. issue: https://bugs.openjdk.java.net/browse/JDK-8166339 webrev: http://cr.openjdk.java.net/~sherman/8166339/webrev This is a regression triggered by issue: https://bugs.openjdk.java.net/browse/JDK-8008386 webrev: http://cr.openjdk.java.net/~sherman/8008386/webrev which updated the doublebyte decoder implementation to handle unmappable bytes more "appropriately" ( malformed (1) or unmappable(2) ). However in case of Cp834, which is a doublebyte-only charset, the unmappable bytes should always be treated as double-byte pair, therefor always unmappable(2). Thanks Sherman From david.holmes at oracle.com Wed Jan 31 01:32:46 2018 From: david.holmes at oracle.com (David Holmes) Date: Wed, 31 Jan 2018 11:32:46 +1000 Subject: [PATCH] GPU Exploitation Infrastructure In-Reply-To: References: <77c394b7-2105-1145-b26d-d53718cda955@oracle.com> Message-ID: Project Sumatra was looking at GPU use: https://wiki.openjdk.java.net/display/Sumatra/Main It's inactive though. David On 31/01/2018 7:46 AM, David Holmes wrote: > On 31/01/2018 1:35 AM, Alan Bateman wrote: >> On 30/01/2018 13:55, Ben Walsh wrote: >>> This patch provides the infrastructure to enable the exploitation of >>> a GPU >>> by the compiler to accelerate certain suitable Java constructs. >>> >>> When enabled, a suitable compiler can attempt to accelerate the >>> following >>> Java constructs by launching the corresponding lambda expression on the >>> GPU: >>> >>> ??? IntStream.range().parallel().forEach() >>> ??? IntStream.rangeClosed().parallel().forEach() >>> >>> ??? where: >>> >>> ?????? defines upper and lower bounds >>> ?????? is a correctly defined lambda expression >>> >>> As it stands, with the HotSpot compiler, this patch performs a "no >>> op" in >>> the newly added in-built native library method. >>> This can be extended so that the HotSpot compiler attempts the >>> acceleration detailed above instead. >>> >>> I would like to pair with a sponsor to contribute this patch ... >> The function prototypes in jvm.h are JVM_* functions rather than JNI >> native method names. You may want to look at adding a JNI function to >> libjava with a wrapper or else use RegisterNatives to map the native >> method name to a JVM_* function. >> >> BTW: Is this something for one the project repos (Valhalla or Panama) >> rather the main line (jdk/jdk)? > > This should be looked at in the context of Panama I think, though IIRC > there was a distinct GPU related project as well? Either way GPU support > in mainline doesn't seem like something to just be dropped in like this > without much more higher-level discussion taking place. > > Plus hotspot changes - even stubs - must be reviewed on the appropriate > hotspot mailing list. > > David > >> -Alan From amaembo at gmail.com Wed Jan 31 03:00:14 2018 From: amaembo at gmail.com (Tagir Valeev) Date: Wed, 31 Jan 2018 10:00:14 +0700 Subject: Collections.addAll: remove outdated performance advice and add a note about atomicity Message-ID: Hello! I suggest a patch for java.util.Collections#addAll JavaDoc: --- Collections.java 2018-01-31 09:39:31.599107500 +0700 +++ Collections.java.patched 2018-01-31 09:51:11.929059600 +0700 @@ -5406,4 +5406,8 @@ * The behavior of this convenience method is identical to that of - * {@code c.addAll(Arrays.asList(elements))}, but this method is likely - * to run significantly faster under most implementations. + * {@code c.addAll(Arrays.asList(elements))} except possible + * difference in intermediate state visibility for concurrent or + * synchronized collections. Calling this method does not guarantee + * that the intermediate state (some of elements are added) is invisible, + * even if the collection itself provides such guarantee for its + * {@link Collection#addAll(Collection)} method. * First, currently it says that Collections#addAll is likely to run significantly faster. However it's only marginally faster for collections which delegate their addAll method to standard AbstractCollection#addAll implementation. Also it could be much slower for collections which have optimized addAll (like ArrayList, CopyOnWriteArrayList, ConcurrentLinkedDeque, etc.). I don't know a single example of collection where Collections#addAll is actually significantly faster. Also it says that the behavior is identical, while it's not. If, e.g. c is a collection returned from synchronizedCollection, then intermediate state of c.addAll(Arrays.asList(elements)) would not be visible under synchronized(c) in another thread. On the other hand, replacing such call with Collections.addAll(c, elements) (to make it "significantly faster") will lift this guarantee: now you can see partially added array. What do you think? Should I file an issue? With best regards, Tagir Valeev. From martinrb at google.com Wed Jan 31 03:07:47 2018 From: martinrb at google.com (Martin Buchholz) Date: Tue, 30 Jan 2018 19:07:47 -0800 Subject: Collections.addAll: remove outdated performance advice and add a note about atomicity In-Reply-To: References: Message-ID: I tried to tackle this here: http://openjdk.markmail.org/thread/eet2zd6ig3pfpv5g and it's still on my TODO list but not likely to get to top spot soon. On Tue, Jan 30, 2018 at 7:00 PM, Tagir Valeev wrote: > Hello! > > I suggest a patch for java.util.Collections#addAll JavaDoc: > > --- Collections.java 2018-01-31 09:39:31.599107500 +0700 > +++ Collections.java.patched 2018-01-31 09:51:11.929059600 +0700 > @@ -5406,4 +5406,8 @@ > * The behavior of this convenience method is identical to that of > - * {@code c.addAll(Arrays.asList(elements))}, but this method is > likely > - * to run significantly faster under most implementations. > + * {@code c.addAll(Arrays.asList(elements))} except possible > + * difference in intermediate state visibility for concurrent or > + * synchronized collections. Calling this method does not guarantee > + * that the intermediate state (some of elements are added) is > invisible, > + * even if the collection itself provides such guarantee for its > + * {@link Collection#addAll(Collection)} method. > * > > First, currently it says that Collections#addAll is likely to run > significantly faster. However it's only marginally faster for > collections which delegate their addAll method to standard > AbstractCollection#addAll implementation. Also it could be much slower > for collections which have optimized addAll (like ArrayList, > CopyOnWriteArrayList, ConcurrentLinkedDeque, etc.). I don't know a > single example of collection where Collections#addAll is actually > significantly faster. Also it says that the behavior is identical, > while it's not. If, e.g. c is a collection returned from > synchronizedCollection, then intermediate state of > c.addAll(Arrays.asList(elements)) would not be visible under > synchronized(c) in another thread. On the other hand, replacing such > call with Collections.addAll(c, elements) (to make it "significantly > faster") will lift this guarantee: now you can see partially added > array. > > What do you think? Should I file an issue? > > With best regards, > Tagir Valeev. > From amy.lu at oracle.com Wed Jan 31 04:10:32 2018 From: amy.lu at oracle.com (Amy Lu) Date: Wed, 31 Jan 2018 12:10:32 +0800 Subject: [JDK 11] RFR 8196211: Move two sun/nio/cs tests into OpenJDK In-Reply-To: References: <74868834-8095-d530-ff16-66ec257f24ee@oracle.com> <4BF3C2BF-1D61-440F-BFC3-18B2A47234DE@oracle.com> <19a79a9f-d49f-8163-e486-2963b7a7d5da@oracle.com> Message-ID: <1b992997-7bc1-e1f8-0a2f-32fd635015bd@oracle.com> Thank you Paul. Pushed as suggested :-) Thanks, Amy On 30/01/2018 12:37 AM, Paul Sandoz wrote: > That?s better :-) > > If you wanna make it static i suggest: > > ? s/inputFileName/INPUT_FILE_NAME > > although there is little benefit here since all access is local to > main. Up to you to make local or keep static. No need for another review. > > Paul. > >> On Jan 29, 2018, at 4:17 AM, Amy Lu > > wrote: >> >> Updated on the inputFileName. >> Please review:http://cr.openjdk.java.net/~amlu/8196211/webrev.01/ >> >> Thanks, >> Amy >> On 27/01/2018 2:32 AM, Paul Sandoz wrote: >>> Hi, >>> >>> Quick observation. >>> >>> EUCTWBufferBoundaryDecodeTest uses a different data file on windows and it uses the line separator as a trigger. Is it possible to better formalize this by passing in the argument for the file via jtreg? >>> >>> Paul. >>> >>> >>>> On Jan 25, 2018, at 11:45 PM, Amy Lu wrote: >>>> >>>> Please review the patch to move two sun/nio/cs tests into OpenJDK. >>>> >>>> bug:https://bugs.openjdk.java.net/browse/JDK-8196211 >>>> webrev:http://cr.openjdk.java.net/~amlu/8196211/webrev.00/ >>>> >>>> Thanks, >>>> Amy >> > From dbrosius at mebigfatguy.com Wed Jan 31 05:15:19 2018 From: dbrosius at mebigfatguy.com (Dave Brosius) Date: Wed, 31 Jan 2018 00:15:19 -0500 Subject: Possible addition of a default 'getOne' method on Iterable? In-Reply-To: References: Message-ID: <15512374-c444-0a68-d3cb-ee1fc02267ec@mebigfatguy.com> Greetings, sorry if this has been asked before, but has there been any consideration for adding a default T getOne() { ??? Iterator it = iterator(); ??? if (!it.hasNext()) { ??????? throw new NoSuchElementException(); ??? } ??? return it.next(); } on the Iterable interface? It is often the case you have a collection of some sort (un indexed, in this case), where you know there is only one value in the collection, or you know for some attribute of all the objects in the Iterable, all objects can be thought of as the same, and so you just want to get any of the elements. Having to craft this iterator code is annoying, and it would be much nicer to be able to do String s = mySet.getOne(); In addition to this, it is likely that most collections could implement getOne() more optimally than using the standard iterator approach. Of course i am not stuck on the choice of the name 'getOne' anything would do. examplar() ?? As we know, naming is always the hardest part. thoughts? dave From orionllmain at gmail.com Wed Jan 31 05:28:30 2018 From: orionllmain at gmail.com (Zheka Kozlov) Date: Wed, 31 Jan 2018 12:28:30 +0700 Subject: Possible addition of a default 'getOne' method on Iterable? In-Reply-To: <15512374-c444-0a68-d3cb-ee1fc02267ec@mebigfatguy.com> References: <15512374-c444-0a68-d3cb-ee1fc02267ec@mebigfatguy.com> Message-ID: Isn't iterable.getOne() the same as iterable.iterator().next()? 2018-01-31 12:15 GMT+07:00 Dave Brosius : > Greetings, > > > sorry if this has been asked before, but has there been any consideration > for adding a > > default T getOne() { > > Iterator it = iterator(); > if (!it.hasNext()) { > throw new NoSuchElementException(); > } > > return it.next(); > } > > > on the Iterable interface? > > > It is often the case you have a collection of some sort (un indexed, in > this case), where you know there is only one value in the collection, or > you know for some attribute of all the objects in the Iterable, all objects > can be thought of as the same, and so you just want to get any of the > elements. > > Having to craft this iterator code is annoying, and it would be much nicer > to be able to do > > String s = mySet.getOne(); > > In addition to this, it is likely that most collections could implement > getOne() more optimally than using the standard iterator approach. > > Of course i am not stuck on the choice of the name 'getOne' anything would > do. examplar() ? As we know, naming is always the hardest part. > > thoughts? > dave > > From dbrosius at mebigfatguy.com Wed Jan 31 06:09:44 2018 From: dbrosius at mebigfatguy.com (Dave Brosius) Date: Wed, 31 Jan 2018 01:09:44 -0500 Subject: Possible addition of a default 'getOne' method on Iterable? In-Reply-To: References: <15512374-c444-0a68-d3cb-ee1fc02267ec@mebigfatguy.com> Message-ID: <1e3e2d80-c719-eb54-e5dd-7c56edff798b@mebigfatguy.com> Basically, but it's annoying and ugly to write most likely sub-optimal has problems with things like synchronized collections On 01/31/2018 12:28 AM, Zheka Kozlov wrote: > Isn't iterable.getOne() the same as iterable.iterator().next()? > > 2018-01-31 12:15 GMT+07:00 Dave Brosius >: > > Greetings, > > > sorry if this has been asked before, but has there been any > consideration for adding a > > default T getOne() { > > ??? Iterator it = iterator(); > ??? if (!it.hasNext()) { > ??????? throw new NoSuchElementException(); > ??? } > > ??? return it.next(); > } > > > on the Iterable interface? > > > It is often the case you have a collection of some sort (un > indexed, in this case), where you know there is only one value in > the collection, or you know for some attribute of all the objects > in the Iterable, all objects can be thought of as the same, and so > you just want to get any of the elements. > > Having to craft this iterator code is annoying, and it would be > much nicer to be able to do > > String s = mySet.getOne(); > > In addition to this, it is likely that most collections could > implement getOne() more optimally than using the standard iterator > approach. > > Of course i am not stuck on the choice of the name 'getOne' > anything would do. examplar() ?? As we know, naming is always the > hardest part. > > thoughts? > dave > > From joe.darcy at oracle.com Wed Jan 31 07:59:41 2018 From: joe.darcy at oracle.com (joe darcy) Date: Tue, 30 Jan 2018 23:59:41 -0800 Subject: JDK 11 RFR of 8194070 : Update various tests to pass under JDK 11 Message-ID: <2465ca6a-ed16-a26c-b21a-79f7c6f214b5@oracle.com> Hello, Please review the changes to address ??? 8194070 : Update various tests to pass under JDK 11 ??? http://cr.openjdk.java.net/~darcy/8194070.2/ which are updates to the various and sundry multi-release jar files tests so that they can after both before and after the JDK version is updated from 10 to 11. To summarize the changes, in different directories the multi-release jar files tests use the shared CreateMultiReleaseTestJars.java library. That library created fixed contents for versions 8, 9, and 10 in a multi-release jar files. Some tests looked for fixed contents in release 8, 9, and 10, but also for content under Runtime.version().major(), or equivalent, which would stop working when the version went to 11. Therefore, some aspects of the overall testing of multi-release jar files was internally inconsistent. This lead to the removal of the"-Djdk.util.jar.version=10" @run lines from MultiReleaseJarHttpProperties.java and MultiReleaseJarProperties.java. I changed CreateMultiReleaseTestJars to created content for 8, 9, and the current version and made corresponding updates in the tests using the library. The general set of multi-release jar file tests could be made more comprehensive, but I'll leave that for, hopefully others to do, as future work. The obvious delta to update MVJarSigningTest to JDK 11 did not create a working test; I'd prefer some one else track down which part of the supporting code also needs to be modified to work against the current version properly. The changed tests pass under a current JDK 11 build as well as an internal JDK 11 build where the version has been updated to 11. Thanks, -Joe From sean.coffey at oracle.com Wed Jan 31 09:07:08 2018 From: sean.coffey at oracle.com (=?UTF-8?Q?Se=c3=a1n_Coffey?=) Date: Wed, 31 Jan 2018 09:07:08 +0000 Subject: JDK-8166339,Code conversion working behavior was changed for x-IBM834 In-Reply-To: <5A70F8EC.4010801@oracle.com> References: <5A70F8EC.4010801@oracle.com> Message-ID: Looks fine to me Sherman. regards, Sean. On 30/01/2018 22:59, Xueming Shen wrote: > Hi, > > Please help review the change for JDK-8166339. > > issue: https://bugs.openjdk.java.net/browse/JDK-8166339 > webrev: http://cr.openjdk.java.net/~sherman/8166339/webrev > > This is a regression triggered by > > issue: https://bugs.openjdk.java.net/browse/JDK-8008386 > webrev: http://cr.openjdk.java.net/~sherman/8008386/webrev > > which updated the doublebyte decoder implementation to handle unmappable > bytes more "appropriately" ( malformed (1) or unmappable(2) ). > > However in case of Cp834, which is a doublebyte-only charset, the > unmappable > bytes should always be treated as double-byte pair, therefor always > unmappable(2). > > Thanks > Sherman From ben_walsh at uk.ibm.com Wed Jan 31 13:16:21 2018 From: ben_walsh at uk.ibm.com (Ben Walsh) Date: Wed, 31 Jan 2018 13:16:21 +0000 Subject: [PATCH] GPU Exploitation Infrastructure In-Reply-To: References: <77c394b7-2105-1145-b26d-d53718cda955@oracle.com> Message-ID: Thank you Alan and David. In light of your responses, I'll take this to the Valhalla project. Thanks, Ben From: David Holmes To: Alan Bateman , Ben Walsh , core-libs-dev at openjdk.java.net Date: 31/01/2018 01:33 Subject: Re: [PATCH] GPU Exploitation Infrastructure Project Sumatra was looking at GPU use: https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.openjdk.java.net_display_Sumatra_Main&d=DwIDaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=0rTKw9slljsdE7-sx--lzTNyOod7e8UQH1LYkfqUvuI&m=vD7LxfdLPrTxzzKSXLaOfQRSdWYQbh-se0qfBfp4nNw&s=AHBCwadIN7Q_j6x0xUbIaPcF0uLyH4HgNqUg4j8YaV0&e= It's inactive though. David On 31/01/2018 7:46 AM, David Holmes wrote: > On 31/01/2018 1:35 AM, Alan Bateman wrote: >> On 30/01/2018 13:55, Ben Walsh wrote: >>> This patch provides the infrastructure to enable the exploitation of >>> a GPU >>> by the compiler to accelerate certain suitable Java constructs. >>> >>> When enabled, a suitable compiler can attempt to accelerate the >>> following >>> Java constructs by launching the corresponding lambda expression on the >>> GPU: >>> >>> IntStream.range().parallel().forEach() >>> IntStream.rangeClosed().parallel().forEach() >>> >>> where: >>> >>> defines upper and lower bounds >>> is a correctly defined lambda expression >>> >>> As it stands, with the HotSpot compiler, this patch performs a "no >>> op" in >>> the newly added in-built native library method. >>> This can be extended so that the HotSpot compiler attempts the >>> acceleration detailed above instead. >>> >>> I would like to pair with a sponsor to contribute this patch ... >> The function prototypes in jvm.h are JVM_* functions rather than JNI >> native method names. You may want to look at adding a JNI function to >> libjava with a wrapper or else use RegisterNatives to map the native >> method name to a JVM_* function. >> >> BTW: Is this something for one the project repos (Valhalla or Panama) >> rather the main line (jdk/jdk)? > > This should be looked at in the context of Panama I think, though IIRC > there was a distinct GPU related project as well? Either way GPU support > in mainline doesn't seem like something to just be dropped in like this > without much more higher-level discussion taking place. > > Plus hotspot changes - even stubs - must be reviewed on the appropriate > hotspot mailing list. > > David > >> -Alan Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU From Roger.Riggs at Oracle.com Wed Jan 31 15:52:10 2018 From: Roger.Riggs at Oracle.com (Roger Riggs) Date: Wed, 31 Jan 2018 10:52:10 -0500 Subject: RFR 8195059: Update java.net Socket and DatagramSocket implementations to use Cleaner In-Reply-To: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> References: <83a9e988-b2f0-93d1-bf47-f4819376e197@Oracle.com> Message-ID: <66f9e158-ab86-2150-8017-1734b8892b99@Oracle.com> Adding net-dev at openjdk.java.net On 1/30/2018 5:08 PM, Roger Riggs wrote: > Please review changes to replace finalizers in socket, datagram, and > multicast networking > with Cleaner based release of the raw file descriptors.? Each > FileDescriptor is registered > for cleanup after the raw fd (or handle) is assigned.? Normal calls to > close unregister the > cleaner before using the current logic to close the raw fd/handle. > Windows networking > uses fd's with the Windows socket_ API requiring a special cased > Cleanable. > > The tests check that the implementation objects including > FileDescriptors are reclaimed > and for Linux that the raw fd counts are reduced as expected. > > Webrev: > ?? http://cr.openjdk.java.net/~rriggs/webrev-net-cleanup-8195059/ > > Thanks, Roger > > From Alan.Bateman at oracle.com Wed Jan 31 19:04:13 2018 From: Alan.Bateman at oracle.com (Alan Bateman) Date: Wed, 31 Jan 2018 19:04:13 +0000 Subject: JDK 11 RFR of 8194070 : Update various tests to pass under JDK 11 In-Reply-To: <2465ca6a-ed16-a26c-b21a-79f7c6f214b5@oracle.com> References: <2465ca6a-ed16-a26c-b21a-79f7c6f214b5@oracle.com> Message-ID: On 31/01/2018 07:59, joe darcy wrote: > Hello, > > Please review the changes to address > > ??? 8194070 : Update various tests to pass under JDK 11 > ??? http://cr.openjdk.java.net/~darcy/8194070.2/ > > which are updates to the various and sundry multi-release jar files > tests so that they can after both before and after the JDK version is > updated from 10 to 11. I was initially uneasy about dropping of the version=10 runs from MultiReleaseJarHttpProperties.java and MultiReleaseJarProperties.java but I think the rational is sound. So I think the patch looks good to me. -Alan From paul.sandoz at oracle.com Wed Jan 31 19:04:45 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 31 Jan 2018 11:04:45 -0800 Subject: JDK 11 RFR of 8194070 : Update various tests to pass under JDK 11 In-Reply-To: <2465ca6a-ed16-a26c-b21a-79f7c6f214b5@oracle.com> References: <2465ca6a-ed16-a26c-b21a-79f7c6f214b5@oracle.com> Message-ID: Looks ok, Paul. > On Jan 30, 2018, at 11:59 PM, joe darcy wrote: > > Hello, > > Please review the changes to address > > 8194070 : Update various tests to pass under JDK 11 > http://cr.openjdk.java.net/~darcy/8194070.2/ > > which are updates to the various and sundry multi-release jar files tests so that they can after both before and after the JDK version is updated from 10 to 11. > > To summarize the changes, in different directories the multi-release jar files tests use the shared CreateMultiReleaseTestJars.java library. That library created fixed contents for versions 8, 9, and 10 in a multi-release jar files. Some tests looked for fixed contents in release 8, 9, and 10, but also for content under Runtime.version().major(), or equivalent, which would stop working when the version went to 11. Therefore, some aspects of the overall testing of multi-release jar files was internally inconsistent. > > This lead to the removal of the"-Djdk.util.jar.version=10" @run lines from MultiReleaseJarHttpProperties.java and MultiReleaseJarProperties.java. > > I changed CreateMultiReleaseTestJars to created content for 8, 9, and the current version and made corresponding updates in the tests using the library. The general set of multi-release jar file tests could be made more comprehensive, but I'll leave that for, hopefully others to do, as future work. > > The obvious delta to update MVJarSigningTest to JDK 11 did not create a working test; I'd prefer some one else track down which part of the supporting code also needs to be modified to work against the current version properly. > > The changed tests pass under a current JDK 11 build as well as an internal JDK 11 build where the version has been updated to 11. > > Thanks, > > -Joe > From xueming.shen at oracle.com Wed Jan 31 18:49:05 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 31 Jan 2018 10:49:05 -0800 Subject: JDK 11 RFR of 8194070 : Update various tests to pass under JDK 11 In-Reply-To: <2465ca6a-ed16-a26c-b21a-79f7c6f214b5@oracle.com> References: <2465ca6a-ed16-a26c-b21a-79f7c6f214b5@oracle.com> Message-ID: <5A720FA1.3030900@oracle.com> +1 On 01/30/2018 11:59 PM, joe darcy wrote: > Hello, > > Please review the changes to address > > 8194070 : Update various tests to pass under JDK 11 > http://cr.openjdk.java.net/~darcy/8194070.2/ > > which are updates to the various and sundry multi-release jar files tests so that they can after both before and after the JDK version is updated from 10 to 11. > > To summarize the changes, in different directories the multi-release jar files tests use the shared CreateMultiReleaseTestJars.java library. That library created fixed contents for versions 8, 9, and 10 in a multi-release jar files. Some tests looked for fixed contents in release 8, 9, and 10, but also for content under Runtime.version().major(), or equivalent, which would stop working when the version went to 11. Therefore, some aspects of the overall testing of multi-release jar files was internally inconsistent. > > This lead to the removal of the"-Djdk.util.jar.version=10" @run lines from MultiReleaseJarHttpProperties.java and MultiReleaseJarProperties.java. > > I changed CreateMultiReleaseTestJars to created content for 8, 9, and the current version and made corresponding updates in the tests using the library. The general set of multi-release jar file tests could be made more comprehensive, but I'll leave that for, hopefully others to do, as future work. > > The obvious delta to update MVJarSigningTest to JDK 11 did not create a working test; I'd prefer some one else track down which part of the supporting code also needs to be modified to work against the current version properly. > > The changed tests pass under a current JDK 11 build as well as an internal JDK 11 build where the version has been updated to 11. > > Thanks, > > -Joe > From xueming.shen at oracle.com Wed Jan 31 19:54:02 2018 From: xueming.shen at oracle.com (Xueming Shen) Date: Wed, 31 Jan 2018 11:54:02 -0800 Subject: RFR JDK-8193802: NullPointerException from JarFileSystem.getVersionMap() Message-ID: <5A721EDA.2020406@oracle.com> Hi Please help review the change for JDK-8193802. Issue: https://bugs.openjdk.java.net/browse/JDK-8193802 Webrev: http://cr.openjdk.java.net/~sherman/8193802/webrev To add a "null" check when building mr lookup tables. The use scenario here is that the jar file and the jarfs env are configured to a mr jarfs, but the jar file itself actually does not have the corresponding directory "meta-inf/versions". Thanks, Sherman From mandy.chung at oracle.com Wed Jan 31 21:10:52 2018 From: mandy.chung at oracle.com (mandy chung) Date: Wed, 31 Jan 2018 13:10:52 -0800 Subject: RFR JDK-8193802: NullPointerException from JarFileSystem.getVersionMap() In-Reply-To: <5A721EDA.2020406@oracle.com> References: <5A721EDA.2020406@oracle.com> Message-ID: <8f90d7f1-b8e0-ee23-e215-8fb61e2dd741@oracle.com> Looks fine to me. Mandy On 1/31/18 11:54 AM, Xueming Shen wrote: > Hi > > Please help review the change for JDK-8193802. > > Issue: https://bugs.openjdk.java.net/browse/JDK-8193802 > Webrev: http://cr.openjdk.java.net/~sherman/8193802/webrev > > To add a "null" check when building mr lookup tables. The use scenario > here is > that the jar file and the jarfs env are configured to a mr jarfs, but > the jar file itself > actually does not have the corresponding directory "meta-inf/versions". > > Thanks, > Sherman From paul.sandoz at oracle.com Wed Jan 31 22:02:13 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 31 Jan 2018 14:02:13 -0800 Subject: Possible addition of a default 'getOne' method on Iterable? In-Reply-To: <1e3e2d80-c719-eb54-e5dd-7c56edff798b@mebigfatguy.com> References: <15512374-c444-0a68-d3cb-ee1fc02267ec@mebigfatguy.com> <1e3e2d80-c719-eb54-e5dd-7c56edff798b@mebigfatguy.com> Message-ID: <00F7B3CA-178D-4087-8A3D-A42F5D335EE7@oracle.com> See also Stream.findFirst and findAny. Stream is the place we have and I think we should primarily focus effort on rather than Iterable for such features. I believe a related feature some have requested is the ability to obtain an only, or at most, one element. On Stream this might manifest as: T findOne(); Optional findAtMostOne(); // See Guava?s MoreCollectors/onlyElement,toOptional [1] which might differ in terms of exceptions thrown compared to other terminal operations. Paul. [1] https://google.github.io/guava/releases/21.0/api/docs/com/google/common/collect/MoreCollectors.html#onlyElement-- > On Jan 30, 2018, at 10:09 PM, Dave Brosius wrote: > > Basically, but it's > > annoying and ugly to write > > most likely sub-optimal > > has problems with things like synchronized collections > > > On 01/31/2018 12:28 AM, Zheka Kozlov wrote: >> Isn't iterable.getOne() the same as iterable.iterator().next()? >> >> 2018-01-31 12:15 GMT+07:00 Dave Brosius >: >> >> Greetings, >> >> >> sorry if this has been asked before, but has there been any >> consideration for adding a >> >> default T getOne() { >> >> Iterator it = iterator(); >> if (!it.hasNext()) { >> throw new NoSuchElementException(); >> } >> >> return it.next(); >> } >> >> >> on the Iterable interface? >> >> >> It is often the case you have a collection of some sort (un >> indexed, in this case), where you know there is only one value in >> the collection, or you know for some attribute of all the objects >> in the Iterable, all objects can be thought of as the same, and so >> you just want to get any of the elements. >> >> Having to craft this iterator code is annoying, and it would be >> much nicer to be able to do >> >> String s = mySet.getOne(); >> >> In addition to this, it is likely that most collections could >> implement getOne() more optimally than using the standard iterator >> approach. >> >> Of course i am not stuck on the choice of the name 'getOne' >> anything would do. examplar() ? As we know, naming is always the >> hardest part. >> >> thoughts? >> dave >> >> > From paul.sandoz at oracle.com Wed Jan 31 23:15:36 2018 From: paul.sandoz at oracle.com (Paul Sandoz) Date: Wed, 31 Jan 2018 15:15:36 -0800 Subject: [11] RFR 8195694: ConstantBootstraps.invoke does not preserve variable arity Message-ID: <6A5707B7-518C-43C2-98C9-6F52AAF3FE6F@oracle.com> Hi, Please review this fix to the invoke BSM so that it preserves variable arity, if any: http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195694-constant-bsms-invoke-arity/webrev/ This will be pushed to the hs repo. Thanks, Paul. From john.r.rose at oracle.com Wed Jan 31 23:23:54 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 31 Jan 2018 15:23:54 -0800 Subject: [11] RFR 8195694: ConstantBootstraps.invoke does not preserve variable arity In-Reply-To: <6A5707B7-518C-43C2-98C9-6F52AAF3FE6F@oracle.com> References: <6A5707B7-518C-43C2-98C9-6F52AAF3FE6F@oracle.com> Message-ID: If you remove the old asType call it?s good! > On Jan 31, 2018, at 3:15 PM, Paul Sandoz wrote: > > Hi, > > Please review this fix to the invoke BSM so that it preserves variable arity, if any: > > http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195694-constant-bsms-invoke-arity/webrev/ > > This will be pushed to the hs repo. > > Thanks, > Paul. From john.r.rose at oracle.com Wed Jan 31 23:49:40 2018 From: john.r.rose at oracle.com (John Rose) Date: Wed, 31 Jan 2018 15:49:40 -0800 Subject: [11] RFR 8195694: ConstantBootstraps.invoke does not preserve variable arity In-Reply-To: References: <6A5707B7-518C-43C2-98C9-6F52AAF3FE6F@oracle.com> Message-ID: <476DA321-6EAB-4C27-A5CE-5776E29F7646@oracle.com> On second thought, you should also use invokeWithArguments to support jumbo arities. This tricky idiom should be put into a utility method, package private for starters. A version of it also appears in BSM invocation code. > On Jan 31, 2018, at 3:23 PM, John Rose wrote: > > If you remove the old asType call it?s good! > >> On Jan 31, 2018, at 3:15 PM, Paul Sandoz wrote: >> >> Hi, >> >> Please review this fix to the invoke BSM so that it preserves variable arity, if any: >> >> http://cr.openjdk.java.net/~psandoz/jdk/JDK-8195694-constant-bsms-invoke-arity/webrev/ >> >> This will be pushed to the hs repo. >> >> Thanks, >> Paul. >